Article catalog

preface

Projects usually choose Go because of its concurrency. The Go team has spent a lot of effort to make concurrency in Go   It becomes cheap (in terms of hardware resources) and efficient, but it is possible to write code using the concurrency feature of Go, which is neither reliable nor feasible. I want to leave you some suggestions to avoid some pitfalls caused by the concurrency feature of Go

Tip: the following is the main content of this article. The following cases can be used for reference

1, Keep yourself busy or finish your work by yourself

Keep yourself busy or do the work yourself

Let's look at the first example

package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintln(w, "Hello")
	})

    //Start a process to handle http listening
	go func() {
		if err := http.ListenAndServe(":8080", nil); err != nil {
			log.Fatal(err) // Exit immediately if an error occurs 
		}
	}()
    

    //for loop
	for {
	}
}

This is a simple http server. The for loop wastes CPU resources infinitely  

2, Leave concurrency or (option) to the caller

1. Example 1

func debug(){

	go http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux)
}

func app(){
	
	go func() {
		mux := http.NewServeMux()
		mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
			fmt.Fprintln(resp, "Hello, QCon!")
		})
		http.ListenAndServe("0.0.0.0:8080", mux)
	}()
	
}

func main() {

	 debug()  // debug
	 app()    // app traffic

	select{}
}

Look at the example above

Two monitoring services have been opened. 8080 is online access and 8001 is debug ged

Question 1. There is an accident on the line. I want to debug the results. I don't know when to quit. Is it stupid x now?

Question 2. Does the main function perform a select {} function and block all the time, just like a stupid x!

Q3. If these two services are in your other packages and no comments are written, will others find that this is a collaborative process?

    example   two

func debug(){

	 http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux)
}

func app(){

	mux := http.NewServeMux()
	mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(resp, "Hello, QCon!")
	})
	http.ListenAndServe("0.0.0.0:8080", mux)

}

func main() {

	 go debug()  // debug
	 go app()    // app traffic

	select{}
}

This example follows the principle of leaving concurrency to the caller

Just as the function in Go leaves concurrency to the caller, the application should leave the monitoring of its state to the program calling them, and if the call fails, the application should restart them. Don't let your application restart itself. It's a process that's best handled from outside the application

Question 1. There is an accident on the line. I want to debug the results. I don't know when to quit. Is it stupid x now?

Question 2. Is it embarrassing if the main function performs a select {} and keeps blocking

Example 3

func debug(){

	 if err := http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux); err != nil{
		 log.Fatal(err)
	 }
}

func app(){

	mux := http.NewServeMux()
	mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(resp, "Hello, QCon!")
	})
	if err := http.ListenAndServe("0.0.0.0:8080", mux); err != nil {
		log.Fatal(err)
	}

}

func main() {

	 go debug()  // debug
	 go app()    // app traffic

	select{}
}

In this example, if two listeners return err, the other will terminate immediately.

What we really want is to pass any errors back to the originator of goroutine so that it can know why goroutine stopped and shut down the process cleanly

Existing problems

1.log.fatal   The main function cannot know what cannot be captured.

2.select is still waiting

3, Don't start a goroutine without knowing when it will stop

Example 3

func serve(addr string, handler http.Handler, stop <-chan struct{}) error {
	s := http.Server{
		Addr:    addr,
		Handler: handler,
	}
    
   //Open a co process and accept the closing signal for closing
	go func() {
		<-stop // wait for stop signal
		s.Shutdown(context.Background())
	}()

	return s.ListenAndServe()
}

func serveApp(stop <-chan struct{}) error {
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(resp, "Hello, QCon!")
	})
	return serve("0.0.0.0:8080", mux, stop)
}

func serveDebug(stop <-chan struct{}) error {
	return serve("127.0.0.1:8001", http.DefaultServeMux, stop)
}

func main() {
    //done records the error returned
	done := make(chan error, 2)
    //Define termination signal
	stop := make(chan struct{})
   //Open debug
	go func() {
		done <- serveDebug(stop)
	}()
    //Open the main program
	go func() {
		done <- serveApp(stop)
	}()

	var stopped bool  //Only record once
    
    //Because there are two co processes, all cycles 2
	for i := 0; i < cap(done); i++ {
        //When one of them returns error, an error is printed
		if err := <-done; err != nil {
			fmt.Println("error: %v", err)
		}
        //And notify and assist in closing
		if !stopped {
			stopped = true
			close(stop)
		}
	}
}

Every time we receive a value on the done channel, we close the stop channel, which causes all goroutines to wait for the channel to close their http. This in turn causes all remaining lists and service goroutine to return. Once all goroutines we started stop, main.main returns and the process stops completely.

There are still some defects in this code. There is no error capture in the server   The Shutdown error may still fail to stop

four   goroutine leak

Example 1

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/pprof"
)

func serve(addr string, handler http.Handler, stop <-chan struct{}) error {
	s := http.Server{
		Addr:    addr,
		Handler: handler,
	}

	//Open a co process and accept the closing signal for closing
	go func() {
		<-stop // wait for stop signal
		s.Shutdown(context.Background())
	}()

	return s.ListenAndServe()
}

func serveApp(stop <-chan struct{}) error {
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(resp, "Hello, QCon!")
	})
	mux.HandleFunc("/received", func(resp http.ResponseWriter, req *http.Request) {
		
		//Leak code
		ch := make(chan int)
		//This process has been waiting for the value sent by ch, but no one will send it to cause leakage
		go func() {
			val := <-ch
			fmt.Println("We received a value:", val)
		}()
		
		//Requesting this link will return directly
		fmt.Fprintln(resp, "Hello, received!")
	})
	return serve(":8080", mux, stop)
}

func serveDebug(stop <-chan struct{}) error {
	pprofHandler := http.NewServeMux()
	pprofHandler.Handle("/debug/pprof/", http.HandlerFunc(pprof.Index))
	return serve(":8001", pprofHandler, stop)
}

func main() {
	//done records the error returned
	done := make(chan error, 2)
	//Define termination signal
	stop := make(chan struct{})
	//Open debug
	go func() {
		done <- serveDebug(stop)
	}()
	//Open the main program
	go func() {
		done <- serveApp(stop)
	}()

	var stopped bool //Only record once

	//Because there are two co processes, all cycles 2
	for i := 0; i < cap(done); i++ {
		//When one of them returns error, an error is printed
		if err := <-done; err != nil {
			fmt.Println("error: %v", err)
		}
		//And notify and assist in closing
		if !stopped {
			stopped = true
			close(stop)
		}
	}
}

/received requests this url to test for leaks

Here is a comparison between requesting this link many times and the original

  When this url is not requested, the number of processes is only 8

  However, if multiple requests are made, many coordination processes cannot exit, resulting in leakage

Ensure that the goroutine created has been completed  

package main

import (
	"context"
	"fmt"
	"net/http"
	"net/http/pprof"
	"time"
)

func serve(addr string, handler http.Handler, stop <-chan struct{}) error {
	s := http.Server{
		Addr:    addr,
		Handler: handler,
	}

	//Open a co process and accept the closing signal for closing
	go func() {
		<-stop // wait for stop signal
		s.Shutdown(context.Background())
	}()

	return s.ListenAndServe()
}

func serveApp(stop <-chan struct{}) error {
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) {
		fmt.Fprintln(resp, "Hello, QCon!")
	})
	mux.HandleFunc("/received", func(resp http.ResponseWriter, req *http.Request) {

		//Leak code
		ch := make(chan int)
		//This process has been waiting for the value sent by ch, but no one will send it to cause leakage
		go func() {
			val := <-ch
			fmt.Println("We received a value:", val)
		}()

		//Requesting this link will return directly
		fmt.Fprintln(resp, "Hello, received!")
	})

	mux.HandleFunc("/index", func(resp http.ResponseWriter, req *http.Request) {
		
		go func() {
			 time.Sleep(time.Second * 5)  //Report information
			fmt.Println("Request reporting information")
		}()

		//Requesting this link will return directly
		fmt.Fprintln(resp, "Hello, index!")
	})
	return serve(":8080", mux, stop)
}

func serveDebug(stop <-chan struct{}) error {
	pprofHandler := http.NewServeMux()
	pprofHandler.Handle("/debug/pprof/", http.HandlerFunc(pprof.Index))
	return serve(":8001", pprofHandler, stop)
}

func main() {
	//done records the error returned
	done := make(chan error, 2)
	//Define termination signal
	stop := make(chan struct{})
	//Open debug
	go func() {
		done <- serveDebug(stop)
	}()
	//Open the main program
	go func() {
		done <- serveApp(stop)
	}()

	var stopped bool //Only record once

	//Because there are two co processes, all cycles 2
	for i := 0; i < cap(done); i++ {
		//When one of them returns error, an error is printed
		if err := <-done; err != nil {
			fmt.Println("error: %v", err)
		}
		//And notify and assist in closing
		if !stopped {
			stopped = true
			close(stop)
		}
	}
}

This example adds an index url to report information

Question 1. If this report is kept waiting, it will cause leakage  

Question 2. You never know when to stop

summary

1. Please give the caller the option of whether to call concurrently.
2. Please be responsible for the cooperation process. 2.1 never start a process where you don't know when to quit. 2.2 control the life cycle of this collaboration process and know when to exit. Whether chen or context  
3. Try to avoid starting the collaboration process in the request. The reported execution tasks can be delivered to the message queue or self-defined worker s to avoid the disclosure of the collaboration process  

Some big guy literature

Practical Go: Real world advice for writing maintainable Go programs

Concurrency Trap #2: Incomplete Work

https://www.ardanlabs.com/blog/2018/11/goroutine-leaks-the-forgotten-sender.html

https://www.ardanlabs.com/blog/2014/01/concurrency-goroutines-and-gomaxprocs.html