Review of Go Micro + Gin soft shutdown service (smooth shutdown service) at different levels I

Review of Go Micro + Gin soft shutdown service (smooth shutdown service) at different levels I

The following is a summary of my chicken's handling of a soft stop service. What's wrong? Please help me correct it

Service soft shutdown means that when closing a service, if there is a request being processed, you should wait for the request to be processed before closing the service, so as to achieve the purpose of smoothly closing the service. The basic idea is as follows:

  1. Process termination signal detected
  2. Remove the service from the registry and do not receive subsequent requests
  3. Detect if a request is being processed
  4. The service can be stopped after all the current requests have been processed

Since I am now engaged in Go development, let's talk about realizing these four big steps based on Go Micro + Gin from the perspective of Go.

Let's talk about the relevant source code of the service run under the micro web package

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-5b18bpgx-1638535990496) (C: \ users \ administrator \ appdata \ roaming \ typora \ typora user images \ image-20211202162254056. PNG)]

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-pwglj3mw-163853990498) (C: \ users \ administrator \ appdata \ roaming \ typora \ user images \ image-20211202162725475. PNG)]

The run method mainly does the following things from top to bottom:

  1. Call the start method to start the service
  2. Call the register method to register the service information with the registry
  3. Create a channel ex and call the s.run(ex) method to start cyclic registration. The run method code is relatively short, as follows. When RegisterInterval is set, start to establish scheduled tasks and register regularly
func (s *service) run(exit chan bool) {
	if s.opts.RegisterInterval <= time.Duration(0) {
		return
	}

	t := time.NewTicker(s.opts.RegisterInterval)

	for {
		select {
		case <-t.C:
			s.register()
		case <-exit:
			t.Stop()
			return
		}
	}
}

4. Monitor the process exit signal

5. deregister de registers and removes the node from the registry

6. stop service

Expand the start method:

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-tkbtljbu-1638535990498) (C: \ users \ administrator \ appdata \ roaming \ typora \ user images \ image-20211203155516337. PNG)]

This method mainly executes the customized pre enhancement function, creates a network listener, constructs a service object, calls the serve method to receive and process requests, then executes the custom post enhancement function, and puts the possible error s when closing the listener into exit. At this point, the start function is completed and the service has been started. Then continue to execute methods such as registration. Then block through select {} until the process exit signal or context cancel is received.

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-khckvvc5-1638535990499) (C: \ users \ administrator \ appdata \ roaming \ typora \ user images \ image-20211203160259067. PNG)]

The stop method is very simple, but it will also execute a pre enhancement function and a post enhancement function respectively, which is one of the key points to realize soft shutdown

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-99a3tbtu-163853590500) (C: \ users \ administrator \ appdata \ roaming \ typora \ user images \ image-20211203161352472. PNG)]

At this point, the whole process from start to stop of the service under the micro web package is probably completed. Generally, WEB packages are used at the API level in conjunction with the Gin framework. Now let's talk about how to realize soft shutdown at this level:

Process termination signal detected

In go, you can use the signal.Notify method to monitor the signals of the process. As for the constant values of each signal under the syscall package, it will not be described here. In the start method mentioned earlier, the micro monitors the signal itself:

ch := make(chan os.Signal, 1)
if s.opts.Signal {
    signal.Notify(ch, syscall.SIGTERM, syscall.SIGINT)
}
select {
    // wait on kill signal
    case sig := <-ch:
    if logger.V(logger.InfoLevel, log) {
        log.Infof("Received signal %s", sig)
    }
    // Wait for context cancel
    case <-s.opts.Context.Done():
    if logger.V(logger.InfoLevel, log) {
        log.Info("Received context shutdown")
    }
}

This means that we don't need to deal with this step. Of course, we can also set Signal in options to prevent micro from listening internally and choose to listen by ourselves, as follows:

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	service := web.NewService()
	service.Init(
		func(o *web.Options) {
			o.BeforeStop = append(o.BeforeStop, BeforeStop)
		},
	)
	go func() {
		if err := service.Run(); err != nil {
			fmt.Println("err:", err)
		}
		fmt.Println("service exit")
	}()
	var state int32 = 1
	sc := make(chan os.Signal)
	signal.Notify(sc, syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
	select {
	case sig := <-sc:
        //You can manually remove nodes from the registry by referring to the deregister method in the start method. There is no detailed description here, because I don't have wheels in this way. Why don't I have to make them myself
		BeforeStop()
		atomic.StoreInt32(&state, 0)
		fmt.Printf("Get exit signal[%s]", sig.String())
	}
	fmt.Println("cancel implement")
	cancel()
	if callFunc != nil {
		callFunc()
	}
	fmt.Printf("Service exit")
	os.Exit(int(atomic.LoadInt32(&state)))
	return
}

Remove the service from the registry and do not receive subsequent requests

Generally, the registry will provide a method to remove nodes. The implementation of micro is as follows. Note: if you manually remove the node, you must put service.Run() in a new collaboration, otherwise this section is useless. Because the run method will select {} block, after the select, the micro will remove the node itself. If it is placed before run, the node has not been registered.

r := service.Options().Service.Client().Options().Registry
n := &registry.Node{
    Id: service.Options().Id,
}
var ns []*registry.Node
ns = append(ns, n)
r.Deregister(&registry.Service{
    Name: service.Options().Name,
    Nodes: ns,
})

Because bloggers directly rely on the internal monitoring signal of micro, they can easily rely on the internal anti registration of micro. After all, why not use wheels.

Detect if a request is being processed

At the beginning, I wanted to implement this step through interceptors. Later, I found that Gin supports custom middleware. At the same time, the middle added through the Use method will be included in the processing link of each request, so it will take off in situ.

Therefore, as long as a middleware is defined and added to the engine of gin, each request will naturally trigger the middleware, so it is easy to record the request. The middleware is implemented as follows:

router := gin.Default()
router.Use(Logger())
var RequestNumber atomic.Int32

func Logger() gin.HandlerFunc {
	return func(c *gin.Context) {
		fmt.Println("-----Request to come in-----")
        //Number of requests + 1
		RequestNumber.Inc()
        //Execute next node
		c.Next()
        //Number of execution completion requests - 1
		fmt.Println("----Request complete----", c.Writer.Status())
		RequestNumber.Dec()
	}
}

The service can be stopped after all the current requests have been processed

At this time, the node is no longer in the registry, and subsequent requests cannot come in. At this time, you can exit the process by waiting for all current requests to be processed. Are you familiar with the operation before exiting? Yes, it is the pre enhancement function in the stop method mentioned earlier. It is said that this set of pre and post enhancements of micro and a series of middleware wrappers make it very convenient for us to expand.

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	service := web.NewService()
	service.Init(
		func(o *web.Options) {
			o.BeforeStop = append(o.BeforeStop, BeforeStop)
		},
	)
    if err := service.Run(); err != nil {
		log.WithError(err).Error("service.Run")
	}
	cancel()
	return
}  
func BeforeStop() error {
	log.Info("Current number of requests:", handler.RequestNumber)
	for {
		log.Info("Current number of requests:", handler.RequestNumber)
		if handler.RequestNumber.Load() == 0 {
			return nil
		}
		time.Sleep(1 * time.Second)
	}
}

The above is the implementation idea and a scheme of soft stop service under go micro web package. To sum up:

1. The middleware using Gin will participate in each request processing link to record requests, that is, the Logger() mentioned above

2. In the run method, the process exit signal will be monitored, and the select {} block will not go down until the process exit or contextcancel() is received. Then call deregister() to remove the service node from the registry. This means that we don't need to deal with listening signals and removing service nodes by ourselves.

3. Judge whether there are still requests that have not been processed. Use the BeforeStop function array provided by micro to judge whether all the current requests have been completed before the stop method is executed.

Reference link:

Go Micro documentation: https://learnku.com/docs/go-micro/2.x/packer/8501

Go Micro wrappers: https://github.com/microhq/go-plugins/tree/master/wrapper

Go Micro pit Prevention Guide: https://magodo.github.io/go-micro-tips/#wrapper

Tags: Go Back-end

Posted on Fri, 03 Dec 2021 16:22:42 -0500 by mattastic