golang provides an out of the box RPC service, which is simple but not simple.
RPC introduction
Remote Procedure Call (RPC) is a kind of computer communication protocol. It allows programs running on another computer to call subroutines in another address space (generally a computer in an open network), while programmers are just like calling local programs without any additional interactive programming. RPC is a mode of CS (client server) architecture, which interacts information by sending request and receiving response.
There are many widely used RPC frameworks, such as gRPC, Thrift, Dubbo, brpc, etc. Some of the RPC frameworks here implement cross language calls, some implement service registration discovery, etc. It is much more widely used than the official RPC package we introduced today. However, through the study of net/rpc, we can make a basic understanding of an RPC framework.
The implementation of Golang
RPC is a cs architecture, so it has both client and server. Next, we first analyze the coding of communication, and then analyze the implementation of RPC from the perspective of server and client.
Communication code
In rpc implementation, golang abstracts the protocol layer. We can customize the protocol to implement our own interface. The interface of the protocol is as follows:
// Server type ServerCodec interface { ReadRequestHeader(*Request) error ReadRequestBody(interface{}) error WriteResponse(*Response, interface{}) error // Close can be called multiple times and must be idempotent. Close() error } // client type ClientCodec interface { WriteRequest(*Request, interface{}) error ReadResponseHeader(*Response) error ReadResponseBody(interface{}) error Close() error }
The package provides the implementation of encoding and decoding based on gob binary coding. Of course, we can also realize the encoding and decoding methods we want.
Server side implementation
Structure definition
type Server struct { serviceMap sync.Map // Save Service reqLock sync.Mutex // Read requested lock freeReq *Request respLock sync.Mutex // Write response lock freeResp *Response }
The server supports concurrent execution by means of mutex. Since each request and response needs to define a Request/Response object, in order to reduce memory allocation, a freeReq/freeResp linked list is used to implement two object pools.
When a Request object is needed, it is obtained from the freeReq linked list and put back into the linked list after use.
Registration of services
Service is saved in the serviceMap of the Server. The information of each service is as follows:
type service struct { name string // service name rcvr reflect.Value // service object typ reflect.Type // Service type method map[string]*methodType // Registration method }
As you can see from the above, a type and multiple methods of that type can be registered as a Service. When registering a Service, save the Service in the serviceMap through the following methods.
// Use object method name by default func (server *Server) Register(rcvr interface{}) error {} // Specify method name func (server *Server) RegisterName(name string, rcvr interface{}) error {}
Service call
First, the start of rpc service. Consistent with most network applications, after accept ing a connection, a collaboration will be started for message processing. The code is as follows:
for { conn, err := lis.Accept() if err != nil { log.Print("rpc.Serve: accept:", err.Error()) return } go server.ServeConn(conn) }
Secondly, for each connection, the server will continuously obtain the request and send the response asynchronously. The code is as follows:
for { // Read request service, mtype, req, argv, replyv, keepReading, err := server.readRequest(codec) if err != nil { if debugLog && err != io.EOF { log.Println("rpc:", err) } if !keepReading { break } if req != nil { // Send request server.sendResponse(sending, req, invalidRequest, codec, err.Error()) server.freeRequest(req) // Release req object } continue } wg.Add(1) // Process each request concurrently go service.call(server, sending, wg, mtype, req, argv, replyv, codec) }
Finally, because requests are sent asynchronously, the order of the requests and the order of the responses are not necessarily the same. Therefore, in the response message, the seq (serial number) of the request message will be carried to ensure the consistency of the message.
In addition, in order to be compatible with http services, net/rpc package also converts http protocol into rpc Protocol through the Hijack method implemented by http package. The code is as follows:
func (server *Server) ServeHTTP(w http.ResponseWriter, req *http.Request) { // The client connects through the CONNECT method // Get the tcp connection through Hijack conn, _, err := w.(http.Hijacker).Hijack() if err != nil { log.Print("rpc hijacking ", req.RemoteAddr, ": ", err.Error()) return } // Send client, support RPC Protocol io.WriteString(conn, "HTTP/1.0 "+connected+"\n\n") // Start request response for RPC server.ServeConn(conn) }
Client side implementation
The connection of the client is simpler than that of the server. We learn from three aspects: initiating connection, sending request and reading response.
RPC connections
Since this RPC supports HTTP protocol for connection upgrade, there are several connection modes.
-
Use the tcp protocol directly.
func Dial(network, address string) (*Client, error) {} -
Use the http protocol. The http protocol can specify a path or use the default rpc path.
// Default path "/_ goRPC_" func DialHTTP(network, address string) (*Client, error) {} // Use default path func DialHTTPPath(network, address, path string) (*Client, error) {}
Sending of request
The sending of RPC requests provides synchronous and asynchronous interface calls as follows:
// asynchronous func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call {} // synchronization func (client *Client) Call(serviceMethod string, args interface{}, reply interface{}) error{}
From the internal implementation, it can be seen that the return data is obtained asynchronously through Go.
Next, let's see how to send the request internally:
func (client *Client) send(call *Call) { // Under normal conditions of client seq := client.seq client.seq++ // Requested serial number client.pending[seq] = call // Code the request, including the request method and parameters. // Encode and send the request. client.request.Seq = seq client.request.ServiceMethod = call.ServiceMethod // client can initiate Request concurrently and wait for Done asynchronously err := client.codec.WriteRequest(&client.request, call.Args) // Whether there is a sending failure. If the sending is successful, it will be saved in the pending map and wait for the request result. if err != nil { client.mutex.Lock() call = client.pending[seq] delete(client.pending, seq) client.mutex.Unlock() if call != nil { call.Error = err call.done() } } }
As you can see from the above, for a client, you can send multiple requests at the same time, and then wait for the response asynchronously.
Read response
After the rpc connection is successful, a connection will be established for reading the response.
for err == nil { response = Response{} err = client.codec.ReadResponseHeader(&response) if err != nil { break } seq := response.Seq client.mutex.Lock() call := client.pending[seq] // Remove from the pending list delete(client.pending, seq) client.mutex.Unlock() // Decode body // There are many kinds of judgments here to judge whether there is any abnormality client.codec.ReadResponseBody(nil) // Finally, the asynchronous waiting request is notified, and the call is completed call.done() }
The response header is read circularly to respond to the body, and the read result is notified to the asynchronous request calling rpc to complete the read of the response once.
Simple example
Here is a simple example provided by our government to summarize rpc package learning.
Server
type Args struct { // Request parameters A, B int } type Quotient struct { // Type of a response Quo, Rem int } type Arith int // Multiplication and division are defined func (t *Arith) Multiply(args *Args, reply *int) error { *reply = args.A * args.B return nil } func (t *Arith) Divide(args *Args, quo *Quotient) error { if args.B == 0 { return errors.New("divide by zero") } quo.Quo = args.A / args.B quo.Rem = args.A % args.B return nil } func main() { serv := rpc.NewServer() arith := new(Arith) serv.Register(arith) // Service registration // Listen through http, and then do protocol conversion http.ListenAndServe("0.0.0.0:3000", serv) }
client
func main() { client, err := rpc.DialHTTP("tcp", "127.0.0.1:3000") if err != nil { log.Fatal("dialing:", err) } dones := make([]chan *rpc.Call, 0, 10) // Start the request synchronously first for i := 0; i < 10; i++ { quotient := new(Quotient) args := &Args{i + 10, i} divCall := client.Go("Arith.Divide", args, quotient, nil) dones = append(dones, divCall.Done) log.Print("send", i) } log.Print("---------------") // Asynchronous read after for idx, done := range dones { replyCall := <-done // will be equal to divCall args := replyCall.Args.(*Args) reply := replyCall.Reply.(*Quotient) log.Printf("%d / %d = %d, %d %% %d = %d\n", args.A, args.B, reply.Quo, args.A, args.B, reply.Rem) log.Print("recv", idx) } }
What can we learn
Finally, make a learning summary.
- Asynchronous operation is implemented for different requests on the unified connection, and data consistency needs to be ensured through request and response.
- Realization of an object pool by linked list
- A simple practice of implementing the hijack method in http package, upgrading to rpc Protocol through http protocol. Hijack the tcp connection of the original http protocol and use it for rpc.
- rpc implementation, through gob coding, should not support communication with other languages. We need to realize the encoding and decoding mode by ourselves.
- The implementation of rpc does not support the registration and discovery of services, so we need to maintain the service provider ourselves.