A deep understanding of the relationship between HTTP services and ontext in golang

A deep understanding of the relationship between HTTP services and ontext in golang

Question Background

In the go language of http services, we often use Context to cancel a request or read data. An accidental attempt has given me some interest in ontext. Next, around the following examples, this paper analyzes how http uses Context to control request cancellation and affect data reading.

Example

Let's start an http service and send a lot of data to each request in the following code:
srv.go:http service

package main

import (
	"fmt"
	"net/http"
)

func hello(w http.ResponseWriter, r *http.Request) {
	for i := 0; i < 100*10000; i++ {
		w.Write([]byte("hello world"))
	}
}

func main() {
	fmt.Println("listening 8888:")
	http.HandleFunc("/hello", hello)
	_ = http.ListenAndServe(":8888", nil)
}

client.go: the client that sent the request

package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"net/http"
	"time"
)

func main() {

	client := http.Client{}
	request, err := http.NewRequest(http.MethodPost, "http://127.0.0.1:8888/hello", nil)
	ctx, cancelFunc := context.WithCancel(request.Context())
	request = request.WithContext(ctx)
	if err != nil {
		return
	}
	response, err := client.Do(request)
	if err != nil {
		log.Fatal(err)
	}
	cache := make([]byte, 128)
	timer := time.NewTimer(time.Millisecond)
	go func() {
		select {
		case <-timer.C:
			cancelFunc()
		}
	}()
	for {
		read, err := response.Body.Read(cache)
		if err == nil {
			fmt.Println(string(cache[:read]))
			continue
		}
		if err == io.EOF {
			fmt.Println(string(cache[:read]))
			break
		}
		log.Fatal(err)
	}

}

The code is simple, so don't annotate it. Start the service and the client separately, and we'll get the following results:

When we see this sentence Process finished with the exit code 1, we know that the program did not exit correctly and encountered an error. The first reaction is to trace the error. Here we trace the error.

Error Tracking

First make it clear that this "context canceled" is printed by the client:

log.Fatal(err)
// This error is due to an error in reading the data in Response and is not an io.EOF error

Breakpoint Entry:

read, err := response.Body.Read(cache)

We will enter the transport.go file:

func (es *bodyEOFSignal) Read(p []byte) (n int, err error) { // This indicates that the body we read is of type bodyEOFSignal
	es.mu.Lock()
	closed, rerr := es.closed, es.rerr
	es.mu.Unlock()
	if closed {
		return 0, errReadOnClosedResBody
	}
	if rerr != nil {
		return 0, rerr
	}

	n, err = es.body.Read(p)// We've read about errors here, what's wrong here, and we'll show you later
	if err != nil {
		es.mu.Lock()
		defer es.mu.Unlock()
		if es.rerr == nil {
			es.rerr = err
		}
		err = es.condfn(err) // Using this method to discriminate errors, get the error information from the upper level
	}
	return
}

Then we go on to the condfn(error) function of bodyEOFSignal:

func (es *bodyEOFSignal) condfn(err error) error {
	if es.fn == nil {
		return err //1
	}
	err = es.fn(err) // If the FN is not empty, it will continue to the bodyEOFSignal to get the upper error information; if the FN is empty, obviously the error has nothing to do with the upper layer, it will be returned at the top 1. In addition, because we are reading data from this body, the error here is obtained from the upper layer by fn.
	es.fn = nil
	return err
}

So let's go on to es.fn(err):

body := &bodyEOFSignal{
			body: resp.Body,
			earlyCloseFn: func() error {
				waitForBodyRead <- false
				<-eofc // will be closed by deferred call at the end of the function
				return nil

			},
			fn: func(err error) error {// Here, this code comes from the method readLoop in transport.go that encapsulates the internal class persistConn, as the name implies: Loop Read
			// Here's a simple skin test to determine if the error is io.EOF, and then proceed further
				isEOF := err == io.EOF
				waitForBodyRead <- isEOF
				if isEOF {
					<-eofc // see comment above eofc declaration
				} else if err != nil {
					if cerr := pc.canceled(); cerr != nil {// Continue debugging and we're here, apparently not an io.EOF error
						return cerr // Returns pc.canceled()
					}
				}
				return err
			},
		}

Continue to pc.canceled():

func (pc *persistConn) canceled() error {
	pc.mu.Lock()
	defer pc.mu.Unlock()
	return pc.canceledErr // The error returned, so the next step is to know what this canceledErr is and how it is assigned?
}

1.What is it?

canceledErr          error // set non-nil if conn is canceled 
//Is an error, and if not empty, the connection is cancelled, then this error is a flag of the connection status or the reason for the disconnection

2.How is it assigned?

Based on canceledErr, we find the following assigned functions:

func (pc *persistConn) cancelRequest(err error) {
	pc.mu.Lock()
	defer pc.mu.Unlock()
	pc.canceledErr = err // Assigned here
	pc.closeLocked(errRequestCanceled)
}

Error tracking comes here first. Next, let's look at it from a Context perspective.

Context

Don't talk about context s here. Interested partners will go Official Web Get it!!! Go back to the client code and pass in a WithCancel context to request to see what this function does:

func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {
	if parent == nil {
		panic("cannot create context from nil parent")
	}
	c := newCancelCtx(parent) // Wrap Parent Context
	propagateCancel(parent, &c)
	return &c, func() { 
		c.cancel(true, Canceled) // Returns a cancel function
	}
}

Entering c.cancel(), you will find Canceled as an error type, defined as follows:

// Canceled is the error returned by Context.Err when the context is canceled.
var Canceled = errors.New("context canceled")// Wasn't this printed by the client? Was it exciting to find the ancestor of the error message
...
//The cancel function is defined as follows:
// cancel closes c.done, cancels each of c's children, and, if
// removeFromParent is true, removes c from its parent's children.
func (c *cancelCtx) cancel(removeFromParent bool, err error) {
	...
	c.err = err //An assignment here is to pass this error to cancelCtx, an internal class of Context
	...
	// Do some sub-context ual notifications and erroneous deliveries, say cancel, don't do it
}

Context comes here, finds the source of the error message in the context, and then looks at how the error is passed to canceledErr, which we talked about earlier.
There seems to be one more entry left unattended, the http.client.Do method:
Let's break into the call entry for the RoundTrip method to see how context s are perceived to be cancelled:

resp, err = rt.RoundTrip(req) //This is called inside the send() method

...

// send issues an HTTP request.
// Caller should close resp.Body when done reading from it.
func send(ireq *Request, rt RoundTripper, deadline time.Time) (resp *Response, didTimeout func() bool, err error) {
	...
	resp, err = rt.RoundTrip(req) 
	...
}

Then follow RoundTrip(...) into:

func (t *Transport) roundTrip(req *Request) (*Response, error) {
	...
	var resp *Response
		if pconn.alt != nil {
			// HTTP/2 path.
			t.setReqCanceler(cancelKey, nil) // not cancelable with CancelRequest
			resp, err = pconn.alt.RoundTrip(req)
		} else {
			resp, err = pconn.roundTrip(treq) // Continue to get here, let's look at this pconn, just like persistConn mentioned earlier, which contains canceledErr, as if we were closer to the truth
		}
}

To get into persistConn's implementation method roundTrip(), let's look at this for loop:

var respHeaderTimer <-chan time.Time
cancelChan := req.Request.Cancel
ctxDoneChan := req.Context().Done() //This request is redefined in setRequestCancel(req *Request, rt RoundTripper, deadline time.Time), which implements the mechanism of timeout cancellation, where listening is timeout monitoring, not our canceled monitoring
pcClosed := pc.closech
canceled := false
for {
		testHookWaitResLoop()
		select { // select opens polling for channel s
		case err := <-writeErrCh:
			if debugRoundTrip {
				req.logf("writeErrCh resv: %T/%#v", err, err)
			}
			if err != nil {
				pc.close(fmt.Errorf("write error: %v", err))
				return nil, pc.mapRoundTripError(req, startBytesWritten, err)
			}
			if d := pc.t.ResponseHeaderTimeout; d > 0 {
				if debugRoundTrip {
					req.logf("starting timer for %v", d)
				}
				timer := time.NewTimer(d)
				defer timer.Stop() // prevent leaks
				respHeaderTimer = timer.C
			}
		case <-pcClosed:
			pcClosed = nil
			if canceled || pc.t.replaceReqCanceler(req.cancelKey, nil) {
				if debugRoundTrip {
					req.logf("closech recv: %T %#v", pc.closed, pc.closed)
				}
				return nil, pc.mapRoundTripError(req, startBytesWritten, pc.closed)
			}
		case <-respHeaderTimer:
			if debugRoundTrip {
				req.logf("timeout waiting for response headers.")
			}
			pc.close(errTimeout)
			return nil, errTimeout
		case re := <-resc:
			if (re.res == nil) == (re.err == nil) {
				panic(fmt.Sprintf("internal error: exactly one of res or err should be set; nil=%v", re.res == nil))
			}
			if debugRoundTrip {
				req.logf("resc recv: %p, %T/%#v", re.res, re.err, re.err)
			}
			if re.err != nil {
				return nil, pc.mapRoundTripError(req, startBytesWritten, re.err)
			}
			return re.res, nil
		case <-cancelChan:
			canceled = pc.t.cancelRequest(req.cancelKey, errRequestCanceled)
			cancelChan = nil
		case <-ctxDoneChan:
			canceled = pc.t.cancelRequest(req.cancelKey, req.Context().Err())
			cancelChan = nil
			ctxDoneChan = nil
		}
	}

So the listening here is not what we canceled. According to the output of the client, our request has been sent to the server, the request has not timed out, and the response has returned. So the function listening here should not be related to the data we read. At first, the editor thought it was here to listen for the return, but here the breakpoint can't be corrected.Pass, now that we've established a connection with the service, all that's left is to read data from the connection and go back to persistConn. First, look at how we've established the connection.

pconn, err := t.getConn(treq, cm)
...
func (t *Transport) getConn(treq *transportRequest, cm connectMethod) (pc *persistConn, err error) {
	req := treq.Request
	trace := treq.trace
	ctx := req.Context() //This goes to the context of request
	w := &wantConn{
			cm:         cm,
			key:        cm.key(),
			ctx:        ctx, //Pass to w
			ready:      make(chan struct{}, 1),
			beforeDial: testHookPrePendingDial,
			afterDial:  testHookPostPendingDial,
		}
	...
	
	select{
	case <-w.ready:
		if w.err != nil {
				// If the request has been canceled, that's probably
				// what caused w.err; if so, prefer to return the
				// cancellation error (see golang.org/issue/16049).
				//If the request is cancelled before the connection is established, the canceled err is monitored here
				select {
				case <-req.Cancel:
					return nil, errRequestCanceledConn
				case <-req.Context().Done():
					return nil, req.Context().Err()
				case err := <-cancelc:
					if err == errRequestCanceled {
						err = errRequestCanceledConn
					}
					return nil, err
				default:
					// return below
				}
			}
	return w.pc, w.err//This returns persistConn
		...	

Connect through this w to dialConn (ctx context.Context, cm connection method) (pconn *persistConn, err error). Open pconn.readLoop() here to read the data inside the connection, so we have errors in the process of reading, and then enter readLoop for analysis

(t *Transport) dialConn(ctx context.Context, cm connectMethod) (pconn *persistConn, err error) {
	...
	go pconn.readLoop()
}

We go directly to the For loop of readLoop():

for alive {
		...

		var resp *Response
		if err == nil {
			resp, err = pc.readResponse(rc, trace) // Read response
		} else {
			err = transportReadFromServerError{err}
			closeErr = err
		}
		...

		waitForBodyRead := make(chan bool, 2)
		body := &bodyEOFSignal{ //Here's the body we read, encapsulating the resp.Body we read above
			body: resp.Body,
			earlyCloseFn: func() error {
				waitForBodyRead <- false
				<-eofc // will be closed by deferred call at the end of the function
				return nil

			},
			fn: func(err error) error {// 
				isEOF := err == io.EOF
				waitForBodyRead <- isEOF
				if isEOF {
					<-eofc // see comment above eofc declaration
				} else if err != nil {
					if cerr := pc.canceled(); cerr != nil {
						return cerr
					}
				}
				return err
			},
		}

		resp.Body = body
		...

		// Before looping back to the top of this function and peeking on
		// the bufio.Reader, wait for the caller goroutine to finish
		// reading the response body. (or for cancellation or death)
		// There is listening turned on, apparently cancellations and timeouts that occur during listening
		select {
		case bodyEOF := <-waitForBodyRead:
			replaced := pc.t.replaceReqCanceler(rc.cancelKey, nil) // before pc might return to idle pool
			alive = alive &&
				bodyEOF &&
				!pc.sawEOF &&
				pc.wroteRequest() &&
				replaced && tryPutIdleConn(trace)
			if bodyEOF {
				eofc <- struct{}{}
			}
		case <-rc.req.Cancel:
			alive = false
			pc.t.CancelRequest(rc.req)
		case <-rc.req.Context().Done(): //Here we're listening for our cancellation
			alive = false //End cycle
			pc.t.cancelRequest(rc.cancelKey, rc.req.Context().Err())//Pass err
		case <-pc.closech:
			alive = false
		}

		testHookReadLoopBeforeNextRead()
	}

As you are familiar with contexts, when we call the cancel method of contexts, we have the following code in the cancel() method of the previous contexts:

	d, _ := c.done.Load().(chan struct{}) // Get the channel value returned by the Done method
	if d == nil {
		c.done.Store(closedchan)
	} else {
		close(d)// Close the channel and write a value to the channel when it is closed
	}

Back to:

ccase <-rc.req.Context().Done():// When contex cancels, it enters the code block
			alive = false
			pc.t.cancelRequest(rc.cancelKey, rc.req.Context().Err())

Enter rc.req.Context().Err() to cancelRequest(...)

func (c *cancelCtx) Err() error {
	c.mu.Lock()
	err := c.err//It seems familiar here that when we talk about context calling the cancel function, we assign c.err a value of cancelErr
	c.mu.Unlock()
	return err
}

So the err or passed into cancelRequest is cancelErr, and we enter cancelRequest:

func (t *Transport) cancelRequest(key cancelKey, err error) bool {
	// This function must not return until the cancel func has completed.
	// See: https://golang.org/issue/34658
	t.reqMu.Lock()
	defer t.reqMu.Unlock()
	cancel := t.reqCanceler[key]// The key here is the cancelkey of our incoming request, getting the func(error) from reqCanceler
	delete(t.reqCanceler, key)
	if cancel != nil {
		cancel(err) // Enter cancel
	}

	return cancel != nil
}

Enter cancel (err):

func (pc *persistConn) cancelRequest(err error) {//This function is not what we saw earlier with tracing errors, which also indicates that we are tracing correctly
	pc.mu.Lock()
	defer pc.mu.Unlock()
	pc.canceledErr = err 
	pc.closeLocked(errRequestCanceled)
}

Here our err is passed to the body body body EOFSignal, and the entire error transfer process is completed.
So the last question left is, what are the errors encountered by n, err = es.body.Read_in the read function of bodyEOFSignal?

n, err = es.body.Read(p)// Debugging found that there was a network connection shutdown error, which indicates that the root cause of err delivery was that the connection was closed
	if err != nil {
		es.mu.Lock()
		defer es.mu.Unlock()
		if es.rerr == nil {
			es.rerr = err
		}
		err = es.condfn(err)
	}
	return

So where is the connection closed?
Let's go back to cancelRequest(...)

pc.closeLocked(errRequestCanceled) //The connection is closed here

In this way, err's entire transfer logic and reason are in the same way!
Let's start with this today
If there are any errors, please correct them. If you have time to correct them later, it's already one o'clock. You will have to work tomorrow and go to bed.

Tags: Go http Context

Posted on Thu, 14 Oct 2021 12:19:04 -0400 by dmayo2