Article catalog
preface
Projects usually choose Go because of its concurrency. The Go team has spent a lot of effort to make concurrency in Go It becomes cheap (in terms of hardware resources) and efficient, but it is possible to write code using the concurrency feature of Go, which is neither reliable nor feasible. I want to leave you some suggestions to avoid some pitfalls caused by the concurrency feature of Go
Tip: the following is the main content of this article. The following cases can be used for reference
1, Keep yourself busy or finish your work by yourself
Keep yourself busy or do the work yourself
Let's look at the first example
package main import ( "fmt" "log" "net/http" ) func main() { http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello") }) //Start a process to handle http listening go func() { if err := http.ListenAndServe(":8080", nil); err != nil { log.Fatal(err) // Exit immediately if an error occurs } }() //for loop for { } }
This is a simple http server. The for loop wastes CPU resources infinitely
2, Leave concurrency or (option) to the caller
1. Example 1
func debug(){ go http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux) } func app(){ go func() { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) http.ListenAndServe("0.0.0.0:8080", mux) }() } func main() { debug() // debug app() // app traffic select{} }
Look at the example above
Two monitoring services have been opened. 8080 is online access and 8001 is debug ged
Question 1. There is an accident on the line. I want to debug the results. I don't know when to quit. Is it stupid x now?
Question 2. Does the main function perform a select {} function and block all the time, just like a stupid x!
Q3. If these two services are in your other packages and no comments are written, will others find that this is a collaborative process?
example two
func debug(){ http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux) } func app(){ mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) http.ListenAndServe("0.0.0.0:8080", mux) } func main() { go debug() // debug go app() // app traffic select{} }
This example follows the principle of leaving concurrency to the caller
Just as the function in Go leaves concurrency to the caller, the application should leave the monitoring of its state to the program calling them, and if the call fails, the application should restart them. Don't let your application restart itself. It's a process that's best handled from outside the application
Question 1. There is an accident on the line. I want to debug the results. I don't know when to quit. Is it stupid x now?
Question 2. Is it embarrassing if the main function performs a select {} and keeps blocking
Example 3
func debug(){ if err := http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux); err != nil{ log.Fatal(err) } } func app(){ mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) if err := http.ListenAndServe("0.0.0.0:8080", mux); err != nil { log.Fatal(err) } } func main() { go debug() // debug go app() // app traffic select{} }
In this example, if two listeners return err, the other will terminate immediately.
What we really want is to pass any errors back to the originator of goroutine so that it can know why goroutine stopped and shut down the process cleanly
Existing problems
1.log.fatal The main function cannot know what cannot be captured.
2.select is still waiting
3, Don't start a goroutine without knowing when it will stop
Example 3
func serve(addr string, handler http.Handler, stop <-chan struct{}) error { s := http.Server{ Addr: addr, Handler: handler, } //Open a co process and accept the closing signal for closing go func() { <-stop // wait for stop signal s.Shutdown(context.Background()) }() return s.ListenAndServe() } func serveApp(stop <-chan struct{}) error { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) return serve("0.0.0.0:8080", mux, stop) } func serveDebug(stop <-chan struct{}) error { return serve("127.0.0.1:8001", http.DefaultServeMux, stop) } func main() { //done records the error returned done := make(chan error, 2) //Define termination signal stop := make(chan struct{}) //Open debug go func() { done <- serveDebug(stop) }() //Open the main program go func() { done <- serveApp(stop) }() var stopped bool //Only record once //Because there are two co processes, all cycles 2 for i := 0; i < cap(done); i++ { //When one of them returns error, an error is printed if err := <-done; err != nil { fmt.Println("error: %v", err) } //And notify and assist in closing if !stopped { stopped = true close(stop) } } }
Every time we receive a value on the done channel, we close the stop channel, which causes all goroutines to wait for the channel to close their http. This in turn causes all remaining lists and service goroutine to return. Once all goroutines we started stop, main.main returns and the process stops completely.
There are still some defects in this code. There is no error capture in the server The Shutdown error may still fail to stop
four goroutine leak
Example 1
package main import ( "context" "fmt" "net/http" "net/http/pprof" ) func serve(addr string, handler http.Handler, stop <-chan struct{}) error { s := http.Server{ Addr: addr, Handler: handler, } //Open a co process and accept the closing signal for closing go func() { <-stop // wait for stop signal s.Shutdown(context.Background()) }() return s.ListenAndServe() } func serveApp(stop <-chan struct{}) error { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) mux.HandleFunc("/received", func(resp http.ResponseWriter, req *http.Request) { //Leak code ch := make(chan int) //This process has been waiting for the value sent by ch, but no one will send it to cause leakage go func() { val := <-ch fmt.Println("We received a value:", val) }() //Requesting this link will return directly fmt.Fprintln(resp, "Hello, received!") }) return serve(":8080", mux, stop) } func serveDebug(stop <-chan struct{}) error { pprofHandler := http.NewServeMux() pprofHandler.Handle("/debug/pprof/", http.HandlerFunc(pprof.Index)) return serve(":8001", pprofHandler, stop) } func main() { //done records the error returned done := make(chan error, 2) //Define termination signal stop := make(chan struct{}) //Open debug go func() { done <- serveDebug(stop) }() //Open the main program go func() { done <- serveApp(stop) }() var stopped bool //Only record once //Because there are two co processes, all cycles 2 for i := 0; i < cap(done); i++ { //When one of them returns error, an error is printed if err := <-done; err != nil { fmt.Println("error: %v", err) } //And notify and assist in closing if !stopped { stopped = true close(stop) } } }
/received requests this url to test for leaks
Here is a comparison between requesting this link many times and the original
When this url is not requested, the number of processes is only 8
However, if multiple requests are made, many coordination processes cannot exit, resulting in leakage
Ensure that the goroutine created has been completed
package main import ( "context" "fmt" "net/http" "net/http/pprof" "time" ) func serve(addr string, handler http.Handler, stop <-chan struct{}) error { s := http.Server{ Addr: addr, Handler: handler, } //Open a co process and accept the closing signal for closing go func() { <-stop // wait for stop signal s.Shutdown(context.Background()) }() return s.ListenAndServe() } func serveApp(stop <-chan struct{}) error { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) mux.HandleFunc("/received", func(resp http.ResponseWriter, req *http.Request) { //Leak code ch := make(chan int) //This process has been waiting for the value sent by ch, but no one will send it to cause leakage go func() { val := <-ch fmt.Println("We received a value:", val) }() //Requesting this link will return directly fmt.Fprintln(resp, "Hello, received!") }) mux.HandleFunc("/index", func(resp http.ResponseWriter, req *http.Request) { go func() { time.Sleep(time.Second * 5) //Report information fmt.Println("Request reporting information") }() //Requesting this link will return directly fmt.Fprintln(resp, "Hello, index!") }) return serve(":8080", mux, stop) } func serveDebug(stop <-chan struct{}) error { pprofHandler := http.NewServeMux() pprofHandler.Handle("/debug/pprof/", http.HandlerFunc(pprof.Index)) return serve(":8001", pprofHandler, stop) } func main() { //done records the error returned done := make(chan error, 2) //Define termination signal stop := make(chan struct{}) //Open debug go func() { done <- serveDebug(stop) }() //Open the main program go func() { done <- serveApp(stop) }() var stopped bool //Only record once //Because there are two co processes, all cycles 2 for i := 0; i < cap(done); i++ { //When one of them returns error, an error is printed if err := <-done; err != nil { fmt.Println("error: %v", err) } //And notify and assist in closing if !stopped { stopped = true close(stop) } } }
This example adds an index url to report information
Question 1. If this report is kept waiting, it will cause leakage
Question 2. You never know when to stop
summary
- Please give the caller the option of whether to call concurrently.
- Please be responsible for the cooperation process. 2.1 never start a process where you don't know when to quit. 2.2 control the life cycle of this collaboration process and know when to exit. Whether chen or context
- Try to avoid starting the collaboration process in the request. The reported execution tasks can be delivered to the message queue or self-defined worker s to avoid the disclosure of the collaboration process
Some big guy literature
Practical Go: Real world advice for writing maintainable Go programs
Concurrency Trap #2: Incomplete Work
https://www.ardanlabs.com/blog/2018/11/goroutine-leaks-the-forgotten-sender.html
https://www.ardanlabs.com/blog/2014/01/concurrency-goroutines-and-gomaxprocs.html