The source code of this article is okhttp: version 4.9.0.
implementation("com.squareup.okhttp3:okhttp:4.9.0")
Basic Usage
After creating the request Request, you need to create a RealCall object using the newCall() method of OkHttpClient, then call execute() to initiate synchronous requests or call enqueue() to initiate asynchronous requests.
//1. Create a request (including URL, method, headers and body) val request = Request .Builder() .url("https://developer.android.google.cn/") .build() //2. Create OkHttpClient (including scheduler, interceptor, DNS, etc.) val okHttpClient = OkHttpClient.Builder().build() //3. Create call (for calling request) val newCall = okHttpClient.newCall(request) //4. Request data asynchronously newCall.enqueue(object :Callback{ override fun onFailure(call: Call, e: IOException) {} override fun onResponse(call: Call, response: Response) {} }) //4. Request data through synchronization val response = newCall.execute()
Request scheduler
Network requests are mainly divided into synchronous requests and asynchronous requests. Dispatcher is mainly used to control concurrent requests. Both synchronous and asynchronous requests will be processed through dispatcher.
// Dispatcher.kt source code class Dispatcher constructor() { // Maximum number of requests executed concurrently @get:Synchronized var maxRequests = 64 //Maximum number of task requests per host @get:Synchronized var maxRequestsPerHost = 5 //Thread pool for executing asynchronous requests private var executorServiceOrNull: ExecutorService? = null @get:Synchronized @get:JvmName("executorService") val executorService: ExecutorService get() { if (executorServiceOrNull == null) { // Create thread pool by default executorServiceOrNull = ThreadPoolExecutor(0, Int.MAX_VALUE, 60, TimeUnit.SECONDS, SynchronousQueue(), threadFactory("$okHttpName Dispatcher", false)) } return executorServiceOrNull!! } //Asynchronous request queue ready to run private val readyAsyncCalls = ArrayDeque<AsyncCall>() //Running asynchronous request queue private val runningAsyncCalls = ArrayDeque<AsyncCall>() //Running synchronization request queue private val runningSyncCalls = ArrayDeque<RealCall>() constructor(executorService: ExecutorService) : this() { this.executorServiceOrNull = executorService } fun executorService(): ExecutorService = executorService }
Request distribution mechanism
Request operation Call
RealCall implements the Call interface and is the only implementation class of this interface. RealCall is a bridge between OkHttp application and network layer. RealCall can be understood as synchronous request operation, and the internal class AsyncCall of RealCall can understand asynchronous request operation.
- Initiate synchronization request execute()
// RealCall.kt source code override fun execute(): Response { check(executed.compareAndSet(false, true)) { "Already Executed" } // Close socket / stream when timeout occurs timeout.enter() callStart() try { // Add the current RealCall to the synchronization request operation queue client.dispatcher.executed(this) // Return Response return getResponseWithInterceptorChain() } finally { // Delete the current RealCall from the synchronization request operation queue client.dispatcher.finished(this) } } // Dispatcher.kt source code @Synchronized internal fun executed(call: RealCall) { runningSyncCalls.add(call) }
In the above code, the execute() method timeout.enter() calls the enter() method of AsyncTimeout to close the Socket or stream when the request times out. After the enter() method is called by the execute() method in RealCall, the executed() of Dispatcher is invoked, the request is added to the synchronous request queue, and then the getResponseWithInterceptorChain() method is used to get the response. After getting the response, Dispatcher will be asked to remove the request from the synchronous request operation queue.
- Initiate asynchronous request enqueue()
// RealCall.kt source code override fun enqueue(responseCallback: Callback) { check(executed.compareAndSet(false, true)) { "Already Executed" } callStart() client.dispatcher.enqueue(AsyncCall(responseCallback)) } // Dispatcher.kt source code internal fun enqueue(call: AsyncCall) { synchronized(this) { // Add AsyncCall to the asynchronous request queue to be executed readyAsyncCalls.add(call) //Not a WebSocket connection request if (!call.call.forWebSocket) { // Find the same AsyncCall as the host with the current request address val existingCall = findExistingCallWithHost(call.host) if (existingCall != null) call.reuseCallsPerHostFrom(existingCall) } } // Identify and execute executable requests promoteAndExecute() } private fun promoteAndExecute(): Boolean { this.assertThreadDoesntHoldLock() val executableCalls = mutableListOf<AsyncCall>() val isRunning: Boolean synchronized(this) { val i = readyAsyncCalls.iterator() while (i.hasNext()) { val asyncCall = i.next() // The number of requests executed concurrently exceeds the maximum number of requests 64 if (runningAsyncCalls.size >= this.maxRequests) break // Max capacity. // The number of concurrent requests of the host exceeds the maximum number of requests 5 if (asyncCall.callsPerHost.get() >= this.maxRequestsPerHost) continue // Host max capacity. i.remove() asyncCall.callsPerHost.incrementAndGet() executableCalls.add(asyncCall) // Add asyncCall to runningAsyncCalls runningAsyncCalls.add(asyncCall) } // isRunning = runningCallsCount() > 0 } for (i in 0 until executableCalls.size) { val asyncCall = executableCalls[i] // Submit asyncCall to thread pool executorService asyncCall.executeOn(executorService) } return isRunning }
In the above code, the enqueue() method of RealCall will create an asynchronous request operation asynccall and hand it over to the dispatcher for processing. AsyncCall implements the Runnable interface. After receiving AsyncCall, Dispatcher will add AsyncCall to the asynchronous request queue readyAsyncCalls to be executed, then call its promoteAndExecute() method. In this method, the readyasynccalls queue is traversed. The number of qualified concurrent requests is less than the maximum number of requests 64, and the number of concurrent requests of the host cannot exceed the maximum number of requests 5. After adding the asynccall to the runningAsyncCalls queue, the valid requests are filtered out and saved, and the traversal of the requests begins immediately, The ExecutorService in the scheduler dispatcher is used for runnable tasks, that is, after traversal, it is added to the thread pool to execute these effective network requests.
// RealCall.kt source code internal inner class AsyncCall( private val responseCallback: Callback ) : Runnable { fun executeOn(executorService: ExecutorService) { client.dispatcher.assertThreadDoesntHoldLock() var success = false try { //Put the current Runnable into the thread pool to run executorService.execute(this) success = true } catch (e: RejectedExecutionException) { val ioException = InterruptedIOException("executor rejected") ioException.initCause(e) noMoreExchanges(ioException) // Failed callback responseCallback.onFailure(this@RealCall, ioException) } finally { if (!success) { // If it fails, it is deleted from the execution asynchronous request queue client.dispatcher.finished(this) // This call is no longer running! } } } override fun run() { threadName("OkHttp ${redactedUrl()}") { var signalledCallback = false // Determine whether the timeout occurs timeout.enter() try { //Get the network response through the interceptor chain val response = getResponseWithInterceptorChain() signalledCallback = true // Successful callback responseCallback.onResponse(this@RealCall, response) } catch (e: IOException) { if (signalledCallback) { // Do not signal the callback twice! Platform.get().log("Callback failure for ${toLoggableString()}", Platform.INFO, e) } else { // Failed callback responseCallback.onFailure(this@RealCall, e) } } catch (t: Throwable) { cancel() if (!signalledCallback) { val canceledException = IOException("canceled due to $t") canceledException.addSuppressed(t) // Failed callback responseCallback.onFailure(this@RealCall, canceledException) } throw t } finally { // Remove the request from the running asynchronous request queue runningAsyncCalls client.dispatcher.finished(this) } } } }
The executeOn() method of AsyncCall class is mainly to put the running asynchronous request queue into the thread pool, and the thread pool starts the thread to execute tasks; The run() method is the request task executed in the thread pool. Get the network request result response through getResponseWithInterceptorChain().
Asynchronous request execution process:
- Interceptor chain
// RealCall.kt source code @Throws(IOException::class) internal fun getResponseWithInterceptorChain(): Response { //Create interceptor collection val interceptors = mutableListOf<Interceptor>() //Add application interceptor set by user interceptors += client.interceptors //Responsible for retry and redirection interceptors += RetryAndFollowUpInterceptor(client) //Request data for bridging application layer and network layer interceptors += BridgeInterceptor(client.cookieJar) //For processing cache interceptors += CacheInterceptor(client.cache) //Network connection interceptor, used to obtain a connection interceptors += ConnectInterceptor if (!forWebSocket) { //Add network interceptor set by user interceptors += client.networkInterceptors } //Used to request the network and obtain the network response interceptors += CallServerInterceptor(forWebSocket) //Create responsibility chain val chain = RealInterceptorChain( call = this, interceptors = interceptors, index = 0, exchange = null, request = originalRequest, connectTimeoutMillis = client.connectTimeoutMillis, readTimeoutMillis = client.readTimeoutMillis, writeTimeoutMillis = client.writeTimeoutMillis ) var calledNoMoreExchanges = false try { //Start responsibility chain val response = chain.proceed(originalRequest) if (isCanceled()) { response.closeQuietly() throw IOException("Canceled") } return response } catch (e: IOException) { calledNoMoreExchanges = true throw noMoreExchanges(e) as Throwable } finally { if (!calledNoMoreExchanges) { noMoreExchanges(null) } } }
The internal implementation of the getResponseWithInterceptorChain() method is accomplished through a responsibility chain mode, encapsulating all stages of the network request to each chain. First, a interceptors list is created, then the interceptor is added to the list, then interceptors is used to create an interceptor chain RealInterceptorChain, then the proceed() method of the interceptor chain is invoked.
// RealInterceptorChain.kt source code @Throws(IOException::class) override fun proceed(request: Request): Response { check(index < interceptors.size) calls++ if (exchange != null) { check(exchange.finder.sameHostAndPort(request.url)) { "network interceptor ${interceptors[index - 1]} must retain the same host and port" } check(calls == 1) { "network interceptor ${interceptors[index - 1]} must call proceed() exactly once" } } // Called the copy method of RealInterceptorChain, // A RealInterceptorChain will be created inside, // Loop the interceptors in interceptors with the parameter index+1 val next = copy(index = index + 1, request = request) // Gets the interceptor currently to be executed val interceptor = interceptors[index] // Run the current interceptor and set the next interceptor. // The internal logic is usually: after the current interceptor completes processing, it will then execute the proceed method of the next interceptor @Suppress("USELESS_ELVIS") val response = interceptor.intercept(next) ?: throw NullPointerException( "interceptor $interceptor returned null") if (exchange != null) { check(index + 1 >= interceptors.size || next.calls == 1) { "network interceptor $interceptor must call proceed() exactly once" } } check(response.body != null) { "interceptor $interceptor returned a response with no body" } return response }
The processed () method is the core of the responsibility chain pattern, handing over the request to the next interceptor.
Interceptor | effect |
---|---|
Application interceptor | After obtaining the original request, you can add some custom header s, general parameters, parameter encryption, gateway access, and so on |
RetryAndFollowUpInterceptor | Responsible for retrying and redirecting when requests fail |
BridgeInterceptor | The bridge interceptors at the application layer and network layer mainly work to add cookies and fixed header s for requests, such as Host, content length, content type, user agent, etc., and then save the cookies of the response results. If the response is compressed by gzip, it also needs to be decompressed |
CacheInterceptor (CACHE interceptor) | Responsible for reading and updating the cache, you can configure a custom cache interceptor |
ConnectInterceptor | The connection interceptor internally maintains a connection pool, which is responsible for connection reuse, creating connections (three handshakes, etc.), releasing connections, and creating socket streams on connections |
Network interceptors | User defined interceptors are usually used to monitor data transmission at the network layer |
CallServerInterceptor (request service interceptor) | The last interceptor in the interceptor chain, which is used to send data to the server and obtain a response |
Interceptor
RetryAndFollowUpInterceptor
Responsible for retrying and redirecting when requests fail.
The code in the intercept() method of RetryAndFollowUpInterceptor is executed in while. The request will be interrupted only when the retry condition is not tenable. Moreover, the interceptor does not set an upper limit for the number of retries, and the maximum number of redirects is 20 times written dead. If there are special requirements, a retry interceptor and redirect interceptor should be customized.
- Retry mechanism
// RetryAndFollowUpInterceptor.kt source code @Throws(IOException::class) override fun intercept(chain: Interceptor.Chain): Response { val realChain = chain as RealInterceptorChain var request = chain.request val call = realChain.call var followUpCount = 0 var priorResponse: Response? = null var newExchangeFinder = true var recoveredFailures = listOf<IOException>() while (true) { // Initialize the ExchangeFinder of RealCall, which is used to find reusable connections. call.enterNetworkInterceptorExchange(request, newExchangeFinder) var response: Response var closeActiveExchange = true try { if (call.isCanceled()) { throw IOException("Canceled") } try { response = realChain.proceed(request) newExchangeFinder = true } catch (e: RouteException) { //The attempt to connect via routing failed. The request will not be sent. if (!recover(e.lastConnectException, call, request, requestSendStarted = false)) { throw e.firstConnectException.withSuppressed(recoveredFailures) } else { recoveredFailures += e.firstConnectException } newExchangeFinder = false continue } catch (e: IOException) { // The attempt to communicate with the server failed. The request may have been sent. if (!recover(e, call, request, requestSendStarted = e !is ConnectionShutdownException)) { throw e.withSuppressed(recoveredFailures) } else { recoveredFailures += e } newExchangeFinder = false continue } // Last response if (priorResponse != null) { response = response.newBuilder() .priorResponse(priorResponse.newBuilder() .body(null) .build()) .build() } // Data exchange val exchange = call.interceptorScopedExchange // followUp redirect request; followUpRequest() determines whether a newRequest needs to be created according to different code s val followUp = followUpRequest(response, exchange) if (followUp == null) { if (exchange != null && exchange.isDuplex) { call.timeoutEarlyExit() } closeActiveExchange = false return response } val followUpBody = followUp.body if (followUpBody != null && followUpBody.isOneShot()) { closeActiveExchange = false return response } response.body?.closeQuietly() //If the number of retries is exceeded, an exception is thrown. MAX_FOLLOW_UPS=20 if (++followUpCount > MAX_FOLLOW_UPS) { throw ProtocolException("Too many follow-up requests: $followUpCount") } request = followUp priorResponse = response } finally { call.exitNetworkInterceptorExchange(closeActiveExchange) } } }
In the above source code, in the intercept() method of the retry and redirect interceptor, when the request is processed in the subsequent interceptor, the recover() method will be called to judge whether to retry or not when it encounters a route exception (RouteException) or IO exception (IOException).
// RetryAndFollowUpInterceptor.kt source code private fun recover(e: IOException,call: RealCall, userRequest: Request,requestSendStarted: Boolean ): Boolean { // The application layer prohibits retry; The value of retryOnConnectionFailure of OkHttpClient is false if (!client.retryOnConnectionFailure) return false // The request body cannot be sent again; // 1) IO exception encountered during request execution (excluding ConnectionShutdownException thrown by Http2Connection) // 2)requestIsOneShot() returns true. This method defaults to false unless we override this method ourselves if (requestSendStarted && requestIsOneShot(e, userRequest)) return false // Fatal abnormality; // 1) Protocol exception ProtocalException // 2)Socket timeout exception SocketTimeoutException // 3) Certificate validation exception certificateexception // 4)SSL peer validation exception SSLPeerUnverifiedException if (!isRecoverable(e, requestSendStarted)) return false // There are no more routes to try again // 1) Proxy is set for OkHttpClient // 2) The DNS server returned multiple IP addresses if (!call.retryAfterFailure()) return false return true }
- Redirection mechanism
// RetryAndFollowUpInterceptor.kt source code @Throws(IOException::class) private fun followUpRequest(userResponse: Response, exchange: Exchange?): Request? { val route = exchange?.connection?.route() val responseCode = userResponse.code val method = userResponse.request.method when (responseCode) { // 407 HTTP proxy authentication HTTP_PROXY_AUTH -> { val selectedProxy = route!!.proxy if (selectedProxy.type() != Proxy.Type.HTTP) { throw ProtocolException("Received HTTP_PROXY_AUTH (407) code while not using proxy") } return client.proxyAuthenticator.authenticate(route, userResponse) } // 401 unauthorized HTTP_UNAUTHORIZED -> return client.authenticator.authenticate(route, userResponse) //Temporary redirection 307 ~ 308300 ~ 303 HTTP_PERM_REDIRECT, HTTP_TEMP_REDIRECT, HTTP_MULT_CHOICE, HTTP_MOVED_PERM, HTTP_MOVED_TEMP, HTTP_SEE_OTHER -> { // Build redirect request return buildRedirectRequest(userResponse, method) } //Customer service timeout 408 HTTP_CLIENT_TIMEOUT -> { // 408's are rare in practice, but some servers like HAProxy use this response code. The // spec says that we may repeat the request without modifications. Modern browsers also // repeat the request (even non-idempotent ones.) if (!client.retryOnConnectionFailure) { // The application layer has directed us not to retry the request. return null } val requestBody = userResponse.request.body if (requestBody != null && requestBody.isOneShot()) { return null } val priorResponse = userResponse.priorResponse if (priorResponse != null && priorResponse.code == HTTP_CLIENT_TIMEOUT) { // We attempted to retry and got another timeout. Give up. return null } if (retryAfter(userResponse, 0) > 0) { return null } return userResponse.request } // Server unavailable 503 HTTP_UNAVAILABLE -> { val priorResponse = userResponse.priorResponse if (priorResponse != null && priorResponse.code == HTTP_UNAVAILABLE) { // We attempted to retry and got another timeout. Give up. return null } if (retryAfter(userResponse, Integer.MAX_VALUE) == 0) { // specifically received an instruction to retry without delay return userResponse.request } return null } // 421 HTTP_MISDIRECTED_REQUEST -> { // OkHttp can coalesce HTTP/2 connections even if the domain names are different. See // RealConnection.isEligible(). If we attempted this and the server returned HTTP 421, then // we can retry on a different connection. val requestBody = userResponse.request.body if (requestBody != null && requestBody.isOneShot()) { return null } if (exchange == null || !exchange.isCoalescedConnection) { return null } exchange.connection.noCoalescedConnections() return userResponse.request } else -> return null } }
From the above code, we can see that the followUpRequest() method will build different types of requests according to different response status codes. When the status code is 407 and the protocol is HTTP, a request containing an authentication challenge is returned; When the status code is 3XX, it will call buildRedirectRequest() to build a redirection request, tell the client to use an alternative location to access the resources of interest to the client, or provide an alternative response instead of the content of the resources.
BridgeInterceptor
As a bridge between application and server.
It is mainly responsible for adding necessary Request header information (including gzip compression, cookie addition, etc.) to the Request during the Request process, and parsing the Response information (including gzip decompression and cookie saving) during the Response process.
// BridgeInterceptor.kt source code @Throws(IOException::class) override fun intercept(chain: Interceptor.Chain): Response { // Create a new request val userRequest = chain.request() val requestBuilder = userRequest.newBuilder() val body = userRequest.body if (body != null) { val contentType = body.contentType() if (contentType != null) { // Add contenttype (media type of entity principal) requestBuilder.header("Content-Type", contentType.toString()) } val contentLength = body.contentLength() if (contentLength != -1L) { // Add content length requestBuilder.header("Content-Length", contentLength.toString()) requestBuilder.removeHeader("Transfer-Encoding") } else { // Add transfer encoding (specify the transmission mode of message body) requestBuilder.header("Transfer-Encoding", "chunked") requestBuilder.removeHeader("Content-Length") } } // Add Host (the server where the requested resource resides) if (userRequest.header("Host") == null) { requestBuilder.header("Host", userRequest.url.toHostHeader()) } // Default keep alive if (userRequest.header("Connection") == null) { requestBuilder.header("Connection", "Keep-Alive") } //The compression mode of transport stream is gzip by default var transparentGzip = false if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) { transparentGzip = true requestBuilder.header("Accept-Encoding", "gzip") } val cookies = cookieJar.loadForRequest(userRequest.url) //Add cookie (local cache) if (cookies.isNotEmpty()) { requestBuilder.header("Cookie", cookieHeader(cookies)) } // Add user agent (information of HTTP client program) if (userRequest.header("User-Agent") == null) { requestBuilder.header("User-Agent", userAgent) } // Return Response val networkResponse = chain.proceed(requestBuilder.build()) // Process the set cookie content in the header and save the cookie cookieJar.receiveHeaders(userRequest.url, networkResponse.headers) //Put the original request into the response val responseBuilder = networkResponse.newBuilder() .request(userRequest) if (transparentGzip && "gzip".equals(networkResponse.header("Content-Encoding"), ignoreCase = true) && networkResponse.promisesBody()) { val responseBody = networkResponse.body if (responseBody != null) { //Encapsulate it into GzipSource, rewrite the read method, and decompress gzip val gzipSource = GzipSource(responseBody.source()) //Thin header val strippedHeaders = networkResponse.headers.newBuilder() .removeAll("Content-Encoding") .removeAll("Content-Length") .build() responseBuilder.headers(strippedHeaders) val contentType = networkResponse.header("Content-Type") responseBuilder.body(RealResponseBody(contentType, -1L, gzipSource.buffer())) } } return responseBuilder.build() }
In fact, BridgeInterceptor is used to help users process network requests. It will help users fill in the configuration information required by server requests, such as user agent, Connection, Host, accept encoding, etc; At the same time, the result of the request will be processed accordingly.
The internal implementation of BridgeInterceptor is mainly divided into the following three steps:
- Set the content type, content length, Host, Connection, Cookie and other parameters for the user's network request, that is, convert the general request into a format suitable for server parsing to adapt to the server side;
- Hand over the converted request to the next interceptor CacheInterceptor through the chain. Processed (requestBuilder.build()) method, and receive the returned result Response;
- gzip and content type conversion are also performed on the result Response to adapt to the application side.
CacheInterceptor
Interceptor that handles the network request cache.
If the cache is used in OkHttp, the cache() method should be used to set the cache when creating OkHttpClient initialization
/** * Maximum value of network cache data (bytes) */ const val MAX_SIZE_NETWORK_CACHE = 50 * 1024 * 1024L private fun initOkHttpClient() { val networkCacheDirectory = File(cacheDir?.absolutePath + "networkCache") if (!networkCacheDirectory.exists()) { networkCacheDirectory.mkdir() } val cache = Cache(networkCacheDirectory, MAX_SIZE_NETWORK_CACHE) okHttpClient = OkHttpClient.Builder() // Use cache .cache(cache) .build() }
Note: CacheInterceptor only caches the requests of GET, HEAD and other methods to obtain resources, but does not cache the request and response data of POST, PUT and other methods to modify resources.
- Get / save cached response
// CacheInterceptor.kt source code @Throws(IOException::class) override fun intercept(chain: Interceptor.Chain): Response { val call = chain.call() //Get the cache. If we configure the cache, we will find out whether there is a cache // Note: okhttp does not configure the cache by default. It is set through the cache method of OkHttpClient.Builder // cacheCandidate is the Response of the last interactive cache with the server val cacheCandidate = cache?.get(chain.request()) val now = System.currentTimeMillis() //The policy here will automatically determine whether to use cache and whether there is cache val strategy = CacheStrategy.Factory(now, chain.request(), cacheCandidate).compute() // Network request val networkRequest = strategy.networkRequest // Network cache val cacheResponse = strategy.cacheResponse cache?.trackResponse(strategy) val listener = (call as? RealCall)?.eventListener ?: EventListener.NONE if (cacheCandidate != null && cacheResponse == null) { // Cache candidate not applicable. Close it. cacheCandidate.body?.closeQuietly() } // If the network request is not allowed and there is no cache in the current network, 504 (Gateway timeout) will be returned if (networkRequest == null && cacheResponse == null) { return Response.Builder() .request(chain.request()) .protocol(Protocol.HTTP_1_1) .code(HTTP_GATEWAY_TIMEOUT) .message("Unsatisfiable Request (only-if-cached)") .body(EMPTY_RESPONSE) .sentRequestAtMillis(-1L) .receivedResponseAtMillis(System.currentTimeMillis()) .build().also { listener.satisfactionFailure(call, it) } } // Do not allow the use of the network. Only use the cache directly if (networkRequest == null) { return cacheResponse!!.newBuilder() .cacheResponse(stripBody(cacheResponse)) .build().also { listener.cacheHit(call, it) } } if (cacheResponse != null) { // Called when the response is provided from the cache or network based on the freshness of the response in the validation cache listener.cacheConditionalHit(call, cacheResponse) } else if (cache != null) { listener.cacheMiss(call) } var networkResponse: Response? = null try { // Direct network request networkResponse = chain.proceed(networkRequest) } finally { // If we're crashing on I/O or otherwise, don't leak the cache body. if (networkResponse == null && cacheCandidate != null) { cacheCandidate.body?.closeQuietly() } } //Check whether the cache is available, if available. Then use the currently cached Response to close the network connection and release the connection. if (cacheResponse != null) { // If the cached data is not empty and the code is 304, it means that the data has not changed and the cached data continues to be used; if (networkResponse?.code == HTTP_NOT_MODIFIED) { val response = cacheResponse.newBuilder() .headers(combine(cacheResponse.headers, networkResponse.headers)) .sentRequestAtMillis(networkResponse.sentRequestAtMillis) .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis) .cacheResponse(stripBody(cacheResponse)) .networkResponse(stripBody(networkResponse)) .build() networkResponse.body!!.close() // Update the cache after combining headers but before stripping the // Content-Encoding header (as performed by initContentStream()). cache!!.trackConditionalCacheHit() // Update cache data cache.update(cacheResponse, response) return response.also { listener.cacheHit(call, it) } } else { cacheResponse.body?.closeQuietly() } } // Return network response val response = networkResponse!!.newBuilder() .cacheResponse(stripBody(cacheResponse)) .networkResponse(stripBody(networkResponse)) .build() if (cache != null) { if (response.promisesBody() && CacheStrategy.isCacheable(response, networkRequest)) { // Save network data to cache val cacheRequest = cache.put(response) return cacheWritingResponse(cacheRequest, response).also { if (cacheResponse != null) { // This will log a conditional cache miss only. listener.cacheMiss(call) } } } if (HttpMethod.invalidatesCache(networkRequest.method)) { try { // Not a get method, remove cache cache.remove(networkRequest) } catch (_: IOException) { // The cache cannot be written. } } } return response }
In the above code, call the compute() method of the CacheStrategy class to create CacheStrategy.
- If there is an onlyIfCached (No Reload response) instruction in CacheControl, the cacheResponse field of CacheStrategy is also empty.
- When the request is still fresh (the age of existence is less than the fresh time), the networkRequest field of CacheStrategy is empty, and the CacheInterceptor will return the response in the cache.
- When the request is no longer fresh, the CacheInterceptor will get the response through ConnectInterceptor and CallServerInterceptor
- CacheStrategy cache policy
// CacheStrategy.kt source code //Cache policy class class CacheStrategy{ //If we need to request the network, networkRequest is not null, otherwise it is null val networkRequest: Request? //The return of the request or the response of the request cannot be cached (generally null if it is expired or no cache) val cacheResponse: Response? }
Select whether to hit the cache according to the two variables of networkRequest and cacheResponse. Conclusion:
networkRequest\cacheResponse | cacheResponse is null | cacheResponse is not null |
---|---|---|
networkRequest is null | HTTP_GATEWAY_TIMEOUT 504 error | Use cache directly |
networkRequest is not null | Make a network request and cache a new response | Judge whether to re request according to code(304) |
// CacheStrategy.kt source code fun compute(): CacheStrategy { // Judge according to the cache control of request and the cache control of response val candidate = computeCandidate() // request uses cache only if (candidate.networkRequest != null && request.cacheControl.onlyIfCached) { return CacheStrategy(null, null) } return candidate } private fun computeCandidate(): CacheStrategy { // If it is null, there is no cache before if (cacheResponse == null) { return CacheStrategy(request, null) } // If the tls handshake is missing, request the network directly if (request.isHttps && cacheResponse.handshake == null) { return CacheStrategy(request, null) } // Judge whether cache is allowed according to the code of cacheResponse if (!isCacheable(cacheResponse, request)) { return CacheStrategy(request, null) } //cacheControl of request val requestCaching = request.cacheControl // noCache of cacheControl of request if (requestCaching.noCache || hasConditions(request)) { return CacheStrategy(request, null) } //cacheControl of response val responseCaching = cacheResponse.cacheControl val ageMillis = cacheResponseAge() var freshMillis = computeFreshnessLifetime() // It is fresh in freshMillis time, and there is no need to request resources from the server if (requestCaching.maxAgeSeconds != -1) { freshMillis = minOf(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds.toLong())) } // Minimum refresh time var minFreshMillis: Long = 0 if (requestCaching.minFreshSeconds != -1) { minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds.toLong()) } // Cache control field maxStale in the request: the client is willing to receive a resource that exceeds the cache time var maxStaleMillis: Long = 0 if (!responseCaching.mustRevalidate && requestCaching.maxStaleSeconds != -1) { maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds.toLong()) } if (!responseCaching.noCache && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) { val builder = cacheResponse.newBuilder() if (ageMillis + minFreshMillis >= freshMillis) { builder.addHeader("Warning", "110 HttpURLConnection \"Response is stale\"") } val oneDayMillis = 24 * 60 * 60 * 1000L if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) { builder.addHeader("Warning", "113 HttpURLConnection \"Heuristic expiration\"") } return CacheStrategy(null, builder.build()) } // Find a condition to add to the request. If the condition is satisfied, the response body // will not be transmitted. val conditionName: String val conditionValue: String? when { etag != null -> { // If none match: tell the server to return the status code 304 if the time is consistent conditionName = "If-None-Match" conditionValue = etag } // lastModified: the last change time of the resource lastModified != null -> { // If modified since: tell the server to return the status code 304 if the time is consistent conditionName = "If-Modified-Since" conditionValue = lastModifiedString } servedDate != null -> { conditionName = "If-Modified-Since" conditionValue = servedDateString } else -> return CacheStrategy(request, null) // No condition! Make a regular request. } val conditionalRequestHeaders = request.headers.newBuilder() conditionalRequestHeaders.addLenient(conditionName, conditionValue!!) val conditionalRequest = request.newBuilder() .headers(conditionalRequestHeaders.build()) .build() //Use cache directly return CacheStrategy(conditionalRequest, cacheResponse) } companion object { /** Returns true if [response] can be stored to later serve another request. */ fun isCacheable(response: Response, request: Request): Boolean { // Always go to network for uncacheable response codes (RFC 7231 section 6.1), This // implementation doesn't support caching partial content. when (response.code) { HTTP_OK, HTTP_NOT_AUTHORITATIVE, HTTP_NO_CONTENT, HTTP_MULT_CHOICE, HTTP_MOVED_PERM, HTTP_NOT_FOUND, HTTP_BAD_METHOD, HTTP_GONE, HTTP_REQ_TOO_LONG, HTTP_NOT_IMPLEMENTED, StatusLine.HTTP_PERM_REDIRECT -> { // These codes can be cached unless headers forbid it. } HTTP_MOVED_TEMP, StatusLine.HTTP_TEMP_REDIRECT -> { // These codes can only be cached with the right response headers. // http://tools.ietf.org/html/rfc7234#section-3 // s-maxage is not checked because OkHttp is a private cache that should ignore s-maxage. if (response.header("Expires") == null && response.cacheControl.maxAgeSeconds == -1 && !response.cacheControl.isPublic && !response.cacheControl.isPrivate) { return false } } else -> { // All other codes cannot be cached. return false } } // A 'no-store' directive on request or response prevents the response from being cached. return !response.cacheControl.noStore && !request.cacheControl.noStore } }
The computeCandidate() method calculates the cache policy according to the cacheControl of request and cacheResponse. When the status code of the response is 302 (HTTP_MOVED_TEMP) or 307 (HTTP_TEMP_REDIRECT), the isCacheable() method will judge whether to return false (no cache) according to the Expires header and cache control header of the response. The function of the Expires header is that the server can specify an absolute date. If this date has passed, it means that the document is not "fresh".
ConnectInterceptor
Responsible for establishing connection with the server. The process is to find connections from the connection pool; If it does not exist, create a connection and complete the TCP and TLS handshake.
Find connection
// ConnectInterceptor.kt source code @Throws(IOException::class) override fun intercept(chain: Interceptor.Chain): Response { val realChain = chain as RealInterceptorChain // exchange is an object encapsulation used to interact with the server val exchange = realChain.call.initExchange(chain) val connectedChain = realChain.copy(exchange = exchange) return connectedChain.proceed(realChain.request) } // RealCall.kt source code internal fun initExchange(chain: RealInterceptorChain): Exchange { ... val exchangeFinder = this.exchangeFinder!! // Codec: codec, which determines whether the request is made in the way of Http1 or Http2 val codec = exchangeFinder.find(client, chain) val result = Exchange(this, eventListener, exchangeFinder, codec) this.interceptorScopedExchange = result this.exchange = result synchronized(this) { this.requestBodyOpen = true this.responseBodyOpen = true } if (canceled) throw IOException("Canceled") return result }
From the above code, the initExchange() method of RealCall class initializes the exchange data exchange class, and then initExchange() - > exchangefinder #find() - > exchangefinder #findhealthyconnection() - > exchangefinder #findconnection() according to the following call chain.
Next, the following is the findConnection() method of exchange finder
// ExchangeFinder.kt source code @Throws(IOException::class) private fun findConnection( connectTimeout: Int, readTimeout: Int, writeTimeout: Int, pingIntervalMillis: Int, connectionRetryEnabled: Boolean ): RealConnection { if (call.isCanceled()) throw IOException("Canceled") // Attempt to reuse the connection in the call val callConnection = call.connection if (callConnection != null) { var toClose: Socket? = null synchronized(callConnection) { if (callConnection.noNewExchanges || !sameHostAndPort(callConnection.route().address.url)) { toClose = call.releaseConnectionNoEvents() } } // If the called connection is not released, it is reused. // We didn't call connectionAcquired() here because we already got it. if (call.connection != null) { check(toClose == null) return callConnection } // Release call connection toClose?.closeQuietly() eventListener.connectionReleased(call, callConnection) } // We need a new connection. Give it fresh stats. refusedStreamCount = 0 connectionShutdownCount = 0 otherFailureCount = 0 // First, find the connection from the connection pool if (connectionPool.callAcquirePooledConnection(address, call, null, false)) { val result = call.connection!! eventListener.connectionAcquired(call, result) return result } // The connection pool has no connection. Try to get a connection from the connection pool using another route val routes: List<Route>? val route: Route if (nextRouteToTry != null) { // Use routes from previous merge connections routes = null route = nextRouteToTry!! nextRouteToTry = null } else if (routeSelection != null && routeSelection!!.hasNext()) { // Use existing routes in routing routes = null route = routeSelection!!.next() } else { // Calculate a new routing (blocking operation) var localRouteSelector = routeSelector if (localRouteSelector == null) { localRouteSelector = RouteSelector(address, call.client.routeDatabase, call, eventListener) this.routeSelector = localRouteSelector } val localRouteSelection = localRouteSelector.next() routeSelection = localRouteSelection routes = localRouteSelection.routes if (call.isCanceled()) throw IOException("Canceled") // If there is a new route, continue to find it from the connection pool if (connectionPool.callAcquirePooledConnection(address, call, routes, false)) { val result = call.connection!! eventListener.connectionAcquired(call, result) return result } route = localRouteSelection.next() } // create a new connection val newConnection = RealConnection(connectionPool, route) call.connectionToCancel = newConnection try { // Make TCP and TLS connections newConnection.connect( connectTimeout, readTimeout, writeTimeout, pingIntervalMillis, connectionRetryEnabled, call, eventListener ) } finally { call.connectionToCancel = null } call.client.routeDatabase.connected(newConnection.route()) // If we raced another call connecting to this host, coalesce the connections. This makes for 3 // different lookups in the connection pool! if (connectionPool.callAcquirePooledConnection(address, call, routes, true)) { val result = call.connection!! nextRouteToTry = route newConnection.socket().closeQuietly() eventListener.connectionAcquired(call, result) return result } synchronized(newConnection) { // Put in connection pool connectionPool.put(newConnection) call.acquireConnectionNoEvents(newConnection) } eventListener.connectionAcquired(call, newConnection) return newConnection }
The indConnection() method does roughly three things. First, it will try to reuse the existing connection of RealCall. If there is no existing connection, it will try to obtain the connection reuse from the connection pool; If there is no reusable connection in the connection pool, a new connection is created and returned to the CallServer interceptor for use.
Next, you can look at the callAcquirePooledConnection() method of the RealConnectionPool class
// RealConnectionPool.kt source code fun callAcquirePooledConnection( address: Address, call: RealCall, routes: List<Route>?, requireMultiplexed: Boolean ): Boolean { for (connection in connections) { synchronized(connection) { // Determine whether the connection supports multiplexing if (requireMultiplexed && !connection.isMultiplexed) return@synchronized // Determine whether the host of the connection matches if (!connection.isEligible(address, routes)) return@synchronized call.acquireConnectionNoEvents(connection) return true } } return false }
The above method is whether the corresponding connection can be found from the connection pool.
Establish connection
// RealConnection.kt source code fun connect( connectTimeout: Int, readTimeout: Int, writeTimeout: Int, pingIntervalMillis: Int, connectionRetryEnabled: Boolean, call: Call, eventListener: EventListener ) { // Determine whether the protocol has been connected check(protocol == null) { "already connected" } var routeException: RouteException? = null val connectionSpecs = route.address.connectionSpecs val connectionSpecSelector = ConnectionSpecSelector(connectionSpecs) if (route.address.sslSocketFactory == null) { if (ConnectionSpec.CLEARTEXT !in connectionSpecs) { throw RouteException(UnknownServiceException( "CLEARTEXT communication not enabled for client")) } val host = route.address.url.host if (!Platform.get().isCleartextTrafficPermitted(host)) { throw RouteException(UnknownServiceException( "CLEARTEXT communication to $host not permitted by network security policy")) } } else { if (Protocol.H2_PRIOR_KNOWLEDGE in route.address.protocols) { throw RouteException(UnknownServiceException( "H2_PRIOR_KNOWLEDGE cannot be used with HTTPS")) } } while (true) { try { // Whether the request uses Proxy.Type.HTTP proxy and the target is an Https connection if (route.requiresTunnel()) { // Create a proxy tunnel connection; The purpose is to use Http to proxy requests for Https connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener) if (rawSocket == null) { // We were unable to connect the tunnel but properly closed down our resources. break } } else { // Connect socket(TCP connection) connectSocket(connectTimeout, readTimeout, call, eventListener) } // Establish request protocol establishProtocol(connectionSpecSelector, pingIntervalMillis, call, eventListener) eventListener.connectEnd(call, route.socketAddress, route.proxy, protocol) break } catch (e: IOException) { ... } } if (route.requiresTunnel() && rawSocket == null) { throw RouteException(ProtocolException( "Too many tunnel connections attempted: $MAX_TUNNEL_ATTEMPTS")) } idleAtNs = System.nanoTime() }
In the connect() method of RealConnection, you will first judge whether the current connection is connected, that is, whether the connect() method has been called. If it has been called, an illegal state exception will be thrown. If there is no connection, judge whether the HTTPS scheme is used for the request. If yes, connect the tunnel. If not, call the connectSocket() method to connect the Socket.
// RealConnection.kt source code @Throws(IOException::class) private fun establishProtocol( connectionSpecSelector: ConnectionSpecSelector, pingIntervalMillis: Int, call: Call, eventListener: EventListener ) { if (route.address.sslSocketFactory == null) { if (Protocol.H2_PRIOR_KNOWLEDGE in route.address.protocols) { socket = rawSocket protocol = Protocol.H2_PRIOR_KNOWLEDGE startHttp2(pingIntervalMillis) return } socket = rawSocket protocol = Protocol.HTTP_1_1 return } eventListener.secureConnectStart(call) // TLS connection connectTls(connectionSpecSelector) eventListener.secureConnectEnd(call, handshake) if (protocol === Protocol.HTTP_2) { startHttp2(pingIntervalMillis) } }
Judge whether the current address is HTTPS; If it is not HTTPS, judge whether the current protocol is plaintext http2. If so, call startHttp2 to start the handshake of http2. If it is Http/1.1, return directly; If it is HTTPS, start to establish TLS security protocol connection (connectTls); If it is HTTPS and http2, in addition to establishing a TLS connection, startHttp2 will be called to start the handshake of http2.
@Throws(IOException::class) private fun connectTls(connectionSpecSelector: ConnectionSpecSelector) { val address = route.address val sslSocketFactory = address.sslSocketFactory var success = false var sslSocket: SSLSocket? = null try { // Create an SSL socket using the request address host, port and TCP socket sslSocket = sslSocketFactory!!.createSocket( rawSocket, address.url.host, address.url.port, true /* autoClose */) as SSLSocket // Configure the encryption algorithm and TLS version for the Socket val connectionSpec = connectionSpecSelector.configureSecureSocket(sslSocket) if (connectionSpec.supportsTlsExtensions) { Platform.get().configureTlsExtensions(sslSocket, address.url.host, address.protocols) } // Call startHandshake() to force a handshake sslSocket.startHandshake() // block for session establishment val sslSocketSession = sslSocket.session val unverifiedHandshake = sslSocketSession.handshake() //Verify the validity of the server certificate if (!address.hostnameVerifier!!.verify(address.url.host, sslSocketSession)) { val peerCertificates = unverifiedHandshake.peerCertificates if (peerCertificates.isNotEmpty()) { val cert = peerCertificates[0] as X509Certificate throw SSLPeerUnverifiedException(""" |Hostname ${address.url.host} not verified: | certificate: ${CertificatePinner.pin(cert)} | DN: ${cert.subjectDN.name} | subjectAltNames: ${OkHostnameVerifier.allSubjectAltNames(cert)} """.trimMargin()) } else { throw SSLPeerUnverifiedException( "Hostname ${address.url.host} not verified (no certificates)") } } val certificatePinner = address.certificatePinner!! handshake = Handshake(unverifiedHandshake.tlsVersion, unverifiedHandshake.cipherSuite, unverifiedHandshake.localCertificates) { certificatePinner.certificateChainCleaner!!.clean(unverifiedHandshake.peerCertificates, address.url.host) } // Certificate locking verification using handshake records certificatePinner.check(address.url.host) { handshake!!.peerCertificates.map { it as X509Certificate } } // If the connection is successful, the handshake record and ALPN protocol are saved val maybeProtocol = if (connectionSpec.supportsTlsExtensions) { Platform.get().getSelectedProtocol(sslSocket) } else { null } socket = sslSocket source = sslSocket.source().buffer() sink = sslSocket.sink().buffer() protocol = if (maybeProtocol != null) Protocol.get(maybeProtocol) else Protocol.HTTP_1_1 success = true } finally { if (sslSocket != null) { Platform.get().afterHandshake(sslSocket) } if (!success) { sslSocket?.closeQuietly() } } }
CallServerInterceptor
Responsible for data interaction with the server; It is responsible for sending request data to the server and reading response data from the server.
@Throws(IOException::class) override fun intercept(chain: Interceptor.Chain): Response { val realChain = chain as RealInterceptorChain val exchange = realChain.exchange!! val request = realChain.request val requestBody = request.body val sentRequestMillis = System.currentTimeMillis() // The request header is written into the socket, and the bottom layer is through the ExchangeCodec protocol class //(corresponding to http1exchangecode and http2exchangecode), // Finally, it is implemented through Okio. The specific implementation is in the RealBufferedSink class exchange.writeRequestHeaders(request) var invokeStartEvent = true var responseBuilder: Response.Builder? = null // Judge whether there is a request body according to the request method if (HttpMethod.permitsRequestBody(request.method) && requestBody != null) { if ("100-continue".equals(request.header("Expect"), ignoreCase = true)) { exchange.flushRequest() responseBuilder = exchange.readResponseHeaders(expectContinue = true) exchange.responseHeadersStart() invokeStartEvent = false } if (responseBuilder == null) { // If multiplex transmission is supported, request body if (requestBody.isDuplex()) { // Prepare a duplex body so that the application can send a request body later. exchange.flushRequest() val bufferedRequestBody = exchange.createRequestBody(request, true).buffer() requestBody.writeTo(bufferedRequestBody) } else { // Write the request body if the "Expect: 100-continue" expectation was met. val bufferedRequestBody = exchange.createRequestBody(request, false).buffer() requestBody.writeTo(bufferedRequestBody) bufferedRequestBody.close() } } else { exchange.noRequestBody() if (!exchange.connection.isMultiplexed) { // If the "Expect: 100-continue" expectation wasn't met, prevent the HTTP/1 connection // from being reused. Otherwise we're still obligated to transmit the request body to // leave the connection in a consistent state. exchange.noNewExchangesOnConnection() } } } else { exchange.noRequestBody() } if (requestBody == null || !requestBody.isDuplex()) { //End of Request exchange.finishRequest() } if (responseBuilder == null) { responseBuilder = exchange.readResponseHeaders(expectContinue = false)!! if (invokeStartEvent) { exchange.responseHeadersStart() invokeStartEvent = false } } // Return response var response = responseBuilder .request(request) .handshake(exchange.connection.handshake()) .sentRequestAtMillis(sentRequestMillis) .receivedResponseAtMillis(System.currentTimeMillis()) .build() var code = response.code if (code == 100) { // 100 means continue responseBuilder = exchange.readResponseHeaders(expectContinue = false)!! if (invokeStartEvent) { exchange.responseHeadersStart() } // response = responseBuilder .request(request) .handshake(exchange.connection.handshake()) .sentRequestAtMillis(sentRequestMillis) .receivedResponseAtMillis(System.currentTimeMillis()) .build() code = response.code } exchange.responseHeadersEnd(response) response = if (forWebSocket && code == 101) { // Connection is upgrading, but we need to ensure interceptors see a non-null response body. response.newBuilder() .body(EMPTY_RESPONSE) .build() } else { response.newBuilder() .body(exchange.openResponseBody(response)) .build() } if ("close".equals(response.request.header("Connection"), ignoreCase = true) || "close".equals(response.header("Connection"), ignoreCase = true)) { exchange.noNewExchangesOnConnection() } if ((code == 204 || code == 205) && response.body?.contentLength() ?: -1L > 0L) { throw ProtocolException( "HTTP $code had non-zero Content-Length: ${response.body?.contentLength()}") } return response }