Solution

   When the author was in Ctrip group, a large number of middleware had been incubated internally. Among them, the distributed log component had been applied to different applications under major business departments. According to statistics, tens of thousands of applications in the whole group were connected to this log component, and a design diagram was drawn according to the impression

 

  text

The theme of this blog is MDC   Mapped Diagnostic Context can be roughly understood as a thread safe container for storing diagnostic logs). Its specific process is to string the whole track through some identifiers, such as A-B-C-remote interface-D. the link related log information can be quickly found in the log file through a certain identifier. The following describes the log scheme in the project I am currently responsible for

logback.xml

Configuring traceId in logback.xml is a bit like a placeholder

 

 MDC

      Write the corresponding traceId variable through MDC

 

Source code analysis

1. What is MDC?

The following figure shows that MDC is a class of slf4j API, which provides put,get,remove and other methods. After reading the source code, we can see that it is actually a ThreadLocal. Each put element is put into it. When logger.info is called, take out the ThreadLocal variable and assign it to the output log

 

　　　　

 

 

 

It can be seen from the above

1 MDCAdapter is an adaptation interface, which is stored under the spi package. Therefore, it is known that MDCAdapter is used to adapt other log components

2. The put method provided by MDC can put a K-V Key value pair into the container, and ensure that the Key is unique in the same thread, and the MDC values of different threads do not affect each other

3 in logback.xml, you can output the req in MDC by declaring% X{REQ_ID} in layout_ ID information

4. The remove method provided by MDC can clear the key value pair information corresponding to the specified key in MDC

 

LogbackMDCAdapters source code

 

 

The above is the use method and source code analysis of MDC. The following describes how to extract and package the logs before and after the call and print them uniformly when calling an external system locally. Because the author did not extract the previous code, each different calling method has to write log.info manually, Although this method has no big problem, it is obviously redundant and can be extracted

 

External interface log trace output

The external interface is involved in the calling process. Because the external interface is in a third-party system, we cannot pass the traceId down. We need to modify our remote calling code. Because the author's project uses restTemplate, we need to add an interceptor to restTemplate to print out relevant logs before and after sending the request, The following is the log interceptor corresponding to my restTemplate

class MyRequestInterceptor implements ClientHttpRequestInterceptor {

        @Override
        public ClientHttpResponse intercept(HttpRequest request, byte[] bytes, ClientHttpRequestExecution execution) throws IOException {
            traceRequest(request, bytes);

            ClientHttpResponse response = execution.execute(request, bytes);
            ClientHttpResponse responseCopy = new BufferingClientHttpResponseWrapper(response);
            traceResponse(responseCopy);
            return responseCopy;
        }

        /**
         * Print request data
         *
         * @param request request
         * @param bytes   Request body
         */
        private void traceRequest(HttpRequest request, byte[] bytes) {
            String body = new String(bytes, StandardCharsets.UTF_8);
            log.info("Request Body = {}", body);
        }

        /**
         * Print response results
         *
         * @param response Response results
         * @throws IOException io
         */
        private void traceResponse(ClientHttpResponse response) throws IOException {
            StringBuilder inputStringBuilder = new StringBuilder();
            try (BufferedReader bufferedReader =
                         new BufferedReader(new InputStreamReader(response.getBody(), StandardCharsets.UTF_8))) {
                String line = bufferedReader.readLine();
                while (line != null) {
                    inputStringBuilder.append(line);
                    // inputStringBuilder.append('\n');
                    line = bufferedReader.readLine();
                }
            }
            log.info("Response Body: {}", inputStringBuilder.toString());
        }

        final class BufferingClientHttpResponseWrapper implements ClientHttpResponse {
            private final ClientHttpResponse response;
            private byte[] body;

            BufferingClientHttpResponseWrapper(ClientHttpResponse response) {
                this.response = response;
            }


            @Override
            public HttpStatus getStatusCode() throws IOException {
                return this.response.getStatusCode();
            }

            @Override
            public int getRawStatusCode() throws IOException {
                return this.response.getRawStatusCode();
            }

            @Override
            public String getStatusText() throws IOException {
                return this.response.getStatusText();
            }

            @Override
            public HttpHeaders getHeaders() {
                return this.response.getHeaders();
            }

            @Override
            public InputStream getBody() throws IOException {
                if (this.body == null) {
                    this.body = StreamUtils.copyToByteArray(this.response.getBody());
                }
                return new ByteArrayInputStream(this.body);
            }

            @Override
            public void close() {
                this.response.close();
            }

        }
    }

　　

last

  The above is about the common usage scenarios of MDC, including the log component in Ctrip. In fact, it is also implemented through MDC internally, but it is adjusted according to the business. In general, in a distributed environment, it is best to output the log to Redis or ES, and then provide an interface to query the log. At present, many similar open-source frameworks integrate distributed link log printing + kanban