OkHttp's IO operation and progress monitoring are taught by Alibaba P8

  • What about file downloads?
  • How to monitor the progress of requests? Is there any general method?

With the above problems, this paper will explain how OkHttp handles the request and response body through the analysis and introduction of OkHttp (version No. 4.9.0) IO operation process, and then use the interceptor to realize a general upload and download progress monitoring.

1. How does OkHttp operate IO

1.1. Start with CallServerInterceptor

Friends who know a little about OkHttp should know that the last interceptor of OkHttp requests is CallServerInterceptor, which is responsible for writing the request data in HTTP format to the socket and reading the returned response data from the socket. Let's start here to analyze how OkHttp operates IO.

class CallServerInterceptor(private val forWebSocket: Boolean) : Interceptor {
  // ...
  override fun intercept(chain: Interceptor.Chain): Response {
    val exchange = realChain.exchange!!
    // ...
    exchange.writeRequestHeaders(request)
    // ...
  }
  // ...
} 

CallServerInterceptor first gets a thing called Exchange, and then calls its writeRequestHeaders, which looks like it is written to the requested header data.

class Exchange {
  fun writeRequestHeaders(request: Request) {
    try {
      eventListener.requestHeadersStart(call)
      codec.writeRequestHeaders(request)
      eventListener.requestHeadersEnd(call, request)
    } catch (e: IOException) {
      eventListener.requestFailed(call, e)
      trackFailure(e)
      throw e
    }
  }
}

/** Encodes HTTP requests and decodes HTTP responses. */
interface ExchangeCodec 

Let's take a look. The processing of request related events is encapsulated here. The real writeRequestHeaders is a codec of exchange code type. Look at the annotation, which is responsible for HTTP request encoding and response decoding, in short, writing request data and reading response data. ExchangeCodec has two implementations: Http1ExchangeCodec and Http2ExchangeCodec (here is a policy mode. Exchange only operates ExchangeCodec, regardless of which one is used). HTTP/2 is a binary protocol and has multiplexing. It is not easy to read like HTTP/1. We only analyze Http1ExchangeCodec (pick the soft persimmon pinch).

class Http1ExchangeCodec {
  // ...
  override fun writeRequestHeaders(request: Request) {
    val requestLine = RequestLine.get(request, connection.route().proxy.type())
    writeRequest(request.headers, requestLine)
  }

  fun writeRequest(headers: Headers, requestLine: String) {
    check(state == STATE_IDLE) { "state: $state" }
    sink.writeUtf8(requestLine).writeUtf8("\r\n")
    for (i in 0 until headers.size) {
      sink.writeUtf8(headers.name(i))
          .writeUtf8(": ")
          .writeUtf8(headers.value(i))
          .writeUtf8("\r\n")
    }
    sink.writeUtf8("\r\n")
    state = STATE_OPEN_REQUEST_BODY
  }
  // ...
} 

Http1ExchangeCodec.writeRequestHeaders first get a request line similar to "GET / HTTP/1.1" through RequestLine.get, and then write the request line and request header into a sink through writeRequest. This seems familiar, that is, the HTTP request message format.

1.2 where is the socket

It seems that the data is written to sink (friends who don't know okio can simply think that it is an OutputStream used to write data). It should perform socket IO operations. We continue to track who it is.

class RealConnection {
  // ...
  internal fun newCodec(client: OkHttpClient, chain: RealInterceptorChain): ExchangeCodec {
    val socket = this.socket!!
    val source = this.source!!
    val sink = this.sink!!
    val http2Connection = this.http2Connection

    return if (http2Connection != null) {
      Http2ExchangeCodec(client, this, chain, http2Connection)
    } else {
      // Set timeout
      socket.soTimeout = chain.readTimeoutMillis()
      source.timeout().timeout(chain.readTimeoutMillis.toLong(), MILLISECONDS)
      sink.timeout().timeout(chain.writeTimeoutMillis.toLong(), MILLISECONDS)
      Http1ExchangeCodec(client, this, source, sink)
    }
  }
} 

In RealConnection.newCodec, we can find out how to build Http1ExchangeCodec. Here, in addition to the sink we want, there will also be a source for reading data. These two will also set the timeout of writeTimeoutMillis and readTimeoutMillis respectively, which are the two timeout times we set when initializing OkHttpClient. Let's find out who the sink here is.

class RealConnection {
  private fun connectSocket(
    connectTimeout: Int,
    readTimeout: Int,
    call: Call,
    eventListener: EventListener
  ) {
    val proxy = route.proxy
    val address = route.address
    // Create socket
    val rawSocket = when (proxy.type()) {
      Proxy.Type.DIRECT, Proxy.Type.HTTP -> address.socketFactory.createSocket()!!
      else -> Socket(proxy)
    }
    this.rawSocket = rawSocket
    eventListener.connectStart(call, route.socketAddress, proxy)
    // Set next timeout
    rawSocket.soTimeout = readTimeout
    try {
      // Connect the socket, and then go inside is some internal operations of the socket, which are not in-depth
      Platform.get().connectSocket(rawSocket, route.socketAddress, connectTimeout)
    } catch (e: ConnectException) {
      // ...
    }
    // ...
    try {
      // This is where we assign the value of sink and source
      source = rawSocket.source().buffer()
      sink = rawSocket.sink().buffer()
    } catch (npe: NullPointerException) {
      // ...
    }
  }
} 

As the name suggests, connectSocket is to connect to the socket. Here, the corresponding sink and source will be obtained through the socket for writing and reading. Of course, there is another method. connectTls will create SSL socket and generate sink and source for HTTPS encrypted communication.

fun Socket.sink(): Sink {
  val timeout = SocketAsyncTimeout(this)
  val sink = OutputStreamSink(getOutputStream(), timeout)
  return timeout.sink(sink)
}

fun Socket.source(): Source {
  val timeout = SocketAsyncTimeout(this)
  val source = InputStreamSource(getInputStream(), timeout)
  return timeout.source(source)
} 

Okio uses the adapter mode to convert the socket's OutputStream and InputStream into Sink and Source, or okio is using the adapter to adapt the entire Java IO world to the ok IO world, providing more concise and efficient IO operations.

Sink = the buffer () after RawSocket. Sink(). buffer(). It is also briefly described here. It is encapsulated with RealBufferedSink, which is similar to the decorator mode (not very rigorous). The cache operation is added, so that the written data will be written to the cache first, and the original sink will be written at the right time. Buffer () has another very useful function, which we will talk about later.

/**
 * Returns a new sink that buffers writes to `sink`. The returned sink will batch writes to `sink`.
 * Use this wherever you write to a sink to get an ergonomic and efficient access to data.
 */
fun Sink.buffer(): BufferedSink = RealBufferedSink(this) 

1.3. Writing of request body

So far, we know where the request line, the request header and the socket come from. The next step is to write the request body. Let's go back to the CallServer interceptor and continue.

class CallServerInterceptor {
  // ...
  override fun intercept(chain: Interceptor.Chain): Response {
    // ...
    if (HttpMethod.permitsRequestBody(request.method) && requestBody != null) {
      // If there is a request body, create a Sink for writing and write the content of the request body
      val bufferedRequestBody = exchange.createRequestBody(request, false).buffer()
      // Write the request body into the created Sink, which is blocked
      requestBody.writeTo(bufferedRequestBody)
      // Close when you're finished
      bufferedRequestBody.close()
      // ...
    } else {
      // No request body
      exchange.noRequestBody()
    }
    // ...
  }
  // ...
}

class Exchange {
  fun createRequestBody(request: Request, duplex: Boolean): Sink {
    this.isDuplex = duplex
    val contentLength = request.body!!.contentLength()
    eventListener.requestBodyStart(call)
    // Get the sink of codec for writing the request body
    val rawRequestBody = codec.createRequestBody(request, contentLength)
    // The package adds length related inspection and error event processing, which is not deeply understood
    return RequestBodySink(rawRequestBody, contentLength)
  }
}

class Http1ExchangeCodec {
  override fun createRequestBody(request: Request, contentLength: Long): Sink {
    return when {
      // HTTP/1 is not supported
      request.body != null && request.body.isDuplex() -> throw ProtocolException(
          "Duplex connections are not supported for HTTP/1")
      // Request body of type chunked, length unknown
      request.isChunked -> newChunkedSink() // Stream a request body of unknown length.
      // Fixed length request body
      contentLength != -1L -> newKnownLengthSink() // Stream a request body of a known length.
      // Illegal situation
      else -> // Stream a request body of a known length.
        throw IllegalStateException(
            "Cannot stream a request body without chunked encoding or a known content length!")
    }
  }
} 

Simply put, Http1ExchangeCodec provides a Sink to write to the RequestBody. After encapsulation, the request body is written through RequestBody.writeTo. If you need to upload files, you may need to monitor the upload progress, so you need to write here.

What did RequestBody.writeTo do? Let's look at the implementation of the next File

abstract class RequestBody {
    /** Returns a new request body that transmits the content of this. */
    @JvmStatic
    @JvmName("create")
    fun File.asRequestBody(contentType: MediaType? = null): RequestBody {
      return object : RequestBody() {
        override fun contentType() = contentType

        override fun contentLength() = length()

        override fun writeTo(sink: BufferedSink) {
          source().use { source -> sink.writeAll(source) }
        }
      }
    }
} 

Here, convert the File to Source, and then write all? Write all? If the File is large, doesn't the memory explode? To tell you the truth, I was a little confused when I first saw here. I didn't understand until I studied it carefully.

Let's take a look at the place created by sink for writing. val bufferedRequestBody = exchange.createRequestBody(request, false).buffer(), and finally a buffer(). It is said that this thing is a RealBufferedSink. Let's take a look at its writeAll method.

internal actual class RealBufferedSink actual constructor(
  @JvmField actual val sink: Sink
) : BufferedSink {
  override fun writeAll(source: Source) = commonWriteAll(source)
  override fun emitCompleteSegments() = commonEmitCompleteSegments()
}

internal inline fun RealBufferedSink.commonWriteAll(source: Source): Long {
  var totalBytesRead = 0L
  while (true) {
    // Read Segment.SIZE (8192) bytes from source to cache buffer every time
    val readCount: Long = source.read(buffer, Segment.SIZE.toLong())
    // Return after reading
    if (readCount == -1L) break
    // Update bytes read
    totalBytesRead += readCount
    // This is where the sink is written
    emitCompleteSegments()
  }
  return totalBytesRead
}

internal inline fun RealBufferedSink.commonEmitCompleteSegments(): BufferedSink {
  check(!closed) { "closed" }
  // Get the number of bytes written (in fact, I haven't studied the logic)
  val byteCount = buffer.completeSegmentByteCount()
  // Write original sink
  if (byteCount > 0L) sink.write(buffer, byteCount)
  return this


## Write at the end

My sharing is coming to an end. Thank you for taking the afternoon out of your busy schedule to listen to me. I hope we can grow and make progress together in the next days!!!

Finally, put a general Android Learning direction and ideas (too many details)~),For you:

![](https://img-blog.csdnimg.cn/img_convert/d8414a16807d8f26c3f4f2fb2045d9f7.png)

For programmers, there are too many knowledge contents and technologies to learn. Here we will put some of them first. Other contents have the opportunity to be presented to you in the following articles. However, all my learning materials have been sorted into a document and have been learning continuously. I hope it can help you and save you the time to search for materials on the Internet You can share the news to your friends and learn together!

> **[CodeChina Open source projects:< Android Summary of study notes+Mobile architecture video+Real interview questions for large factories+Project practice source code](

)**

**Why some people will always be better than you is because they are excellent and continue to strive to become better, and are you still satisfied with the status quo and secretly happy! I hope you can give me a little praise and pay attention, and update the technical dry goods in the future. Thank you for your support!**

Android It's a long road for architects. Let's encourage each other!

If you think the article is well written, give it a compliment? If you think there is something worth improving, please leave me a message and will seriously query and correct the deficiencies. Thank you.

![](https://img-blog.csdnimg.cn/img_convert/a31a42fc7b75be02ac7b8dd558cdd4ae.png)

> **This article has been[tencent CODING Open source hosting project:< Android Summary of study notes+Mobile architecture video+Real interview questions for large factories+Project practice source code](https://ali1024.coding.net/public/P7/Android/git), self-study resources and series of articles are constantly updated**
Inner joy! I hope you can give me a little praise and pay attention, and update the technical dry goods in the future. Thank you for your support!**

Android It's a long road for architects. Let's encourage each other!

If you think the article is well written, give it a compliment? If you think there is something worth improving, please leave me a message and will seriously query and correct the deficiencies. Thank you.

[External chain picture transfer...(img-XaJTYu8O-1631260534105)]

> **This article has been[tencent CODING Open source hosting project:< Android Summary of study notes+Mobile architecture video+Real interview questions for large factories+Project practice source code](https://ali1024.coding.net/public/P7/Android/git), self-study resources and series of articles are constantly updated**

Tags: Java Design Pattern Programmer http

Posted on Sat, 20 Nov 2021 01:25:14 -0500 by Izzy1979