Implementation of a lightweight RPC framework based on Netty and SpringBoot Client side request response synchronization

premise

Pre article:

The previous article briefly introduced the function of transforming Client-side contract interface call to send RPC protocol request through dynamic proxy. This paper mainly solves a legacy technical problem: request response synchronization.

The required dependencies are as follows:

  • JDK1.8+
  • Netty:4.1.44.Final
  • SpringBoot:2.2.2.RELEASE

Simple analysis of Netty request response processing flow

[the external link image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-lnplodyj-1579330696725) (https://throwable-blog-1256189093.cos.ap-guangdong.myqcloud.com/202001/n-s-b-r-c-r-1. PNG))

In the figure, the codec and other inbound and outbound processors have been ignored. Threads of different colors represent totally different threads. The processing logic between different threads is completely asynchronous. That is to say, when the Netty IO thread (n-l-g-1) receives the message from the Server and the parsing is completed, the user calling thread (u-t-1) cannot perceive the parsed message package. Then All we need to do is let the user call thread (u-t-1) get the message package received and parsed by Netty IO thread (n-l-g-1).

Here we can use a simple example to illustrate the process of simulating the Client calling thread waiting for the processing result of Netty IO thread to be returned synchronously.

@Slf4j
public class NettyThreadSyncTest {

    @ToString
    private static class ResponseFuture {

        private final long beginTimestamp = System.currentTimeMillis();
        @Getter
        private final long timeoutMilliseconds;
        @Getter
        private final String requestId;
        @Setter
        @Getter
        private volatile boolean sendRequestSucceed = false;
        @Setter
        @Getter
        private volatile Throwable cause;
        @Getter
        private volatile Object response;

        private final CountDownLatch latch = new CountDownLatch(1);

        public ResponseFuture(String requestId, long timeoutMilliseconds) {
            this.requestId = requestId;
            this.timeoutMilliseconds = timeoutMilliseconds;
        }

        public boolean timeout() {
            return System.currentTimeMillis() - beginTimestamp > timeoutMilliseconds;
        }

        public Object waitResponse(final long timeoutMilliseconds) throws InterruptedException {
            latch.await(timeoutMilliseconds, TimeUnit.MILLISECONDS);
            return response;
        }

        public void putResponse(Object response) throws InterruptedException {
            this.response = response;
            latch.countDown();
        }
    }

    static ExecutorService REQUEST_THREAD;
    static ExecutorService NETTY_IO_THREAD;
    static Callable<Object> REQUEST_TASK;
    static Runnable RESPONSE_TASK;

    static String processBusiness(String name) {
        return String.format("%s say hello!", name);
    }

    private static final Map<String /* request id */, ResponseFuture> RESPONSE_FUTURE_TABLE = Maps.newConcurrentMap();

    @BeforeClass
    public static void beforeClass() throws Exception {
        String requestId = UUID.randomUUID().toString();
        String requestContent = "throwable";
        REQUEST_TASK = () -> {
            try {
                // 3 seconds no response considered timeout
                ResponseFuture responseFuture = new ResponseFuture(requestId, 3000);
                RESPONSE_FUTURE_TABLE.put(requestId, responseFuture);
                // The operation of sending the request is ignored here, only printing the log and simulation takes 1 second
                Thread.sleep(1000);
                log.info("Request sent successfully,request ID:{},Request content:{}", requestId, requestContent);
                // Update tag properties
                responseFuture.setSendRequestSucceed(true);
                // Two seconds to wait - this is a rough calculation
                return responseFuture.waitResponse(3000 - 1000);
            } catch (Exception e) {
                log.info("Send request failed,request ID:{},Request content:{}", requestId, requestContent);
                throw new RuntimeException(e);
            }
        };
        RESPONSE_TASK = () -> {
            String responseContent = processBusiness(requestContent);
            try {
                ResponseFuture responseFuture = RESPONSE_FUTURE_TABLE.get(requestId);
                if (null != responseFuture) {
                    log.warn("Response processed successfully,request ID:{},Response content:{}", requestId, responseContent);
                    responseFuture.putResponse(responseContent);
                } else {
                    log.warn("request ID[{}]Corresponding ResponseFuture Non-existent,Ignore processing", requestId);
                }
            } catch (Exception e) {
                log.info("Failed to process response,request ID:{},Response content:{}", requestId, responseContent);
                throw new RuntimeException(e);
            }
        };
        REQUEST_THREAD = Executors.newSingleThreadExecutor(runnable -> {
            Thread thread = new Thread(runnable, "REQUEST_THREAD");
            thread.setDaemon(true);
            return thread;
        });
        NETTY_IO_THREAD = Executors.newSingleThreadExecutor(runnable -> {
            Thread thread = new Thread(runnable, "NETTY_IO_THREAD");
            thread.setDaemon(true);
            return thread;
        });
    }

    @Test
    public void testProcessSync() throws Exception {
        log.info("Asynchronous submit request processing task......");
        Future<Object> future = REQUEST_THREAD.submit(REQUEST_TASK);
        // Simulation request time consuming
        Thread.sleep(1500);
        log.info("Asynchronous submit response processing task......");
        NETTY_IO_THREAD.execute(RESPONSE_TASK);
        // Timeout can be set here
        log.info("Get request results synchronously:{}", future.get());
        Thread.sleep(Long.MAX_VALUE);
    }
}

Execute testProcessSync() method, and the console output is as follows:

2020-01-18 13:17:07 [main] info c.t.client.nettythreadsynctest - asynchronous submit request processing task
 2020-01-18 13:17:08 [request [thread] info c.t.client.nettythreadsynctest - request ID:71f47e27-c17c-458d-b271-4e74fad33a7b, request content: throwable
 2020-01-18 13:17:09 [main] info c.t.client.nettythreadsynctest - asynchronous submit response processing task
 2020-01-18 13:17:09 [netty_io_thread] warn c.t.client.nettythreadsynctest - processing response succeeded, request ID:71f47e27-c17c-458d-b271-4e74fad33a7b, response content: throwable say hello!
2020-01-18 13:17:09 [main] info c.t.client.nettythreadsynctest - synchronous get request result: throwable say hello!

The thread synchronization in the above example mainly refers to the implementation logic of the mainstream client part of the Netty framework: RocketMQ (specifically NettyRemotingClient class) and Redisson (specifically RedisExecutor class), which are used to convert asynchronous thread processing into synchronous processing.

Client side request response synchronization processing

According to the previous example, first add a ResponseFuture to host the sent but unresponsive requests:

@ToString
public class ResponseFuture {

    private final long beginTimestamp = System.currentTimeMillis();
    @Getter
    private final long timeoutMilliseconds;
    @Getter
    private final String requestId;
    @Setter
    @Getter
    private volatile boolean sendRequestSucceed = false;
    @Setter
    @Getter
    private volatile Throwable cause;
    @Getter
    private volatile ResponseMessagePacket response;

    private final CountDownLatch latch = new CountDownLatch(1);

    public ResponseFuture(String requestId, long timeoutMilliseconds) {
        this.requestId = requestId;
        this.timeoutMilliseconds = timeoutMilliseconds;
    }

    public boolean timeout() {
        return System.currentTimeMillis() - beginTimestamp > timeoutMilliseconds;
    }

    public ResponseMessagePacket waitResponse(final long timeoutMilliseconds) throws InterruptedException {
        latch.await(timeoutMilliseconds, TimeUnit.MILLISECONDS);
        return response;
    }

    public void putResponse(ResponseMessagePacket response) throws InterruptedException {
        this.response = response;
        latch.countDown();
    }
}

Then you need to add a new HashMap to cache the ResponseFuture that has been successfully returned but has not received response processing:

Map<String /* request id */, ResponseFuture> RESPONSE_FUTURE_TABLE = Maps.newConcurrentMap();

Here, the KEY selects the requestId, which has been defined as UUID before, to ensure that each request will not be repeated. For simplicity, all logic is currently written in the contract proxy factory ContractProxyFactory, adding the following functions:

  • Add a synchronous sending method sendRequestSync() to handle sending and synchronous response of message package, and the logic of converting RequestMessagePacket to return value type of calling agent target method is also written in this method temporarily.
  • Add a thread pool with the number of core threads as the number of logical cores * 2 for processing requests.
  • Add a single thread scheduling thread pool to regularly clean up those expired ResponseFuture. The cleaning method is scanResponseFutureTable().

The modified ContractProxyFactory is as follows:

@Slf4j
public class ContractProxyFactory {

    private static final RequestArgumentExtractor EXTRACTOR = new DefaultRequestArgumentExtractor();
    private static final ConcurrentMap<Class<?>, Object> CACHE = Maps.newConcurrentMap();
    static final ConcurrentMap<String /* request id */, ResponseFuture> RESPONSE_FUTURE_TABLE = Maps.newConcurrentMap();
    // Define a maximum timeout of 3 seconds for requests
    private static final long REQUEST_TIMEOUT_MS = 3000;
    private static final ExecutorService EXECUTOR;
    private static final ScheduledExecutorService CLIENT_HOUSE_KEEPER;
    private static final Serializer SERIALIZER = FastJsonSerializer.X;


    @SuppressWarnings("unchecked")
    public static <T> T ofProxy(Class<T> interfaceKlass) {
        // Caching proxy class instances of contract interfaces
        return (T) CACHE.computeIfAbsent(interfaceKlass, x ->
                Proxy.newProxyInstance(interfaceKlass.getClassLoader(), new Class[]{interfaceKlass}, (target, method, args) -> {
                    RequestArgumentExtractInput input = new RequestArgumentExtractInput();
                    input.setInterfaceKlass(interfaceKlass);
                    input.setMethod(method);
                    RequestArgumentExtractOutput output = EXTRACTOR.extract(input);
                    // Encapsulate request parameters
                    RequestMessagePacket packet = new RequestMessagePacket();
                    packet.setMagicNumber(ProtocolConstant.MAGIC_NUMBER);
                    packet.setVersion(ProtocolConstant.VERSION);
                    packet.setSerialNumber(SerialNumberUtils.X.generateSerialNumber());
                    packet.setMessageType(MessageType.REQUEST);
                    packet.setInterfaceName(output.getInterfaceName());
                    packet.setMethodName(output.getMethodName());
                    packet.setMethodArgumentSignatures(output.getMethodArgumentSignatures().toArray(new String[0]));
                    packet.setMethodArguments(args);
                    Channel channel = ClientChannelHolder.CHANNEL_REFERENCE.get();
                    return sendRequestSync(channel, packet, method.getReturnType());
                }));
    }

    /**
     * Send request synchronously
     *
     * @param channel channel
     * @param packet  packet
     * @return Object
     */
    static Object sendRequestSync(Channel channel, RequestMessagePacket packet, Class<?> returnType) {
        long beginTimestamp = System.currentTimeMillis();
        ResponseFuture responseFuture = new ResponseFuture(packet.getSerialNumber(), REQUEST_TIMEOUT_MS);
        RESPONSE_FUTURE_TABLE.put(packet.getSerialNumber(), responseFuture);
        try {
            // Get the Future of the hosted response Packet
            Future<ResponseMessagePacket> packetFuture = EXECUTOR.submit(() -> {
                channel.writeAndFlush(packet).addListener((ChannelFutureListener)
                        future -> responseFuture.setSendRequestSucceed(true));
                return responseFuture.waitResponse(REQUEST_TIMEOUT_MS - (System.currentTimeMillis() - beginTimestamp));
            });
            ResponseMessagePacket responsePacket = packetFuture.get(
                    REQUEST_TIMEOUT_MS - (System.currentTimeMillis() - beginTimestamp), TimeUnit.MILLISECONDS);
            if (null == responsePacket) {
                // Response package acquisition failed due to timeout
                throw new SendRequestException(String.format("ResponseMessagePacket Get timeout,request ID:%s", packet.getSerialNumber()));
            } else {
                ByteBuf payload = (ByteBuf) responsePacket.getPayload();
                byte[] bytes = ByteBufferUtils.X.readBytes(payload);
                return SERIALIZER.decode(bytes, returnType);
            }
        } catch (Exception e) {
            log.error("Synchronous send request exception,Request packet:{}", JSON.toJSONString(packet), e);
            if (e instanceof RuntimeException) {
                throw (RuntimeException) e;
            } else {
                throw new SendRequestException(e);
            }
        }
    }

    static void scanResponseFutureTable() {
        log.info("Start execution ResponseFutureTable Cleaning task......");
        Iterator<Map.Entry<String, ResponseFuture>> iterator = RESPONSE_FUTURE_TABLE.entrySet().iterator();
        while (iterator.hasNext()) {
            Map.Entry<String, ResponseFuture> entry = iterator.next();
            ResponseFuture responseFuture = entry.getValue();
            if (responseFuture.timeout()) {
                iterator.remove();
                log.warn("Remove expired requests ResponseFuture,request ID:{}", entry.getKey());
            }
        }
        log.info("implement ResponseFutureTable End of cleanup task......");
    }

    static {
        int n = Runtime.getRuntime().availableProcessors();
        EXECUTOR = new ThreadPoolExecutor(n * 2, n * 2, 0, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(50), runnable -> {
            Thread thread = new Thread(runnable);
            thread.setDaemon(true);
            thread.setName("CLIENT_REQUEST_EXECUTOR");
            return thread;
        });
        CLIENT_HOUSE_KEEPER = new ScheduledThreadPoolExecutor(1, runnable -> {
            Thread thread = new Thread(runnable);
            thread.setDaemon(true);
            thread.setName("CLIENT_HOUSE_KEEPER");
            return thread;
        });
        CLIENT_HOUSE_KEEPER.scheduleWithFixedDelay(ContractProxyFactory::scanResponseFutureTable, 5, 5, TimeUnit.SECONDS);
    }
}

Then add a client inbound processor to match the target ResponseFuture instance through the reuqestId, set the response property in the ResponseFuture instance as the response package, and release the lock at the same time:

@Slf4j
public class ClientHandler extends SimpleChannelInboundHandler<ResponseMessagePacket> {

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, ResponseMessagePacket packet) throws Exception {
        log.info("Response packet received,content:{}", JSON.toJSONString(packet));
        ResponseFuture responseFuture = ContractProxyFactory.RESPONSE_FUTURE_TABLE.get(packet.getSerialNumber());
        if (null != responseFuture) {
            responseFuture.putResponse(packet);
        } else {
            log.warn("Receive response package query ResponseFuture Non-existent,request ID:{}", packet.getSerialNumber());
        }
    }
}

Finally, the client starts the ClientApplication class and adds clientandler to the processor pipeline of Netty

bootstrap.handler(new ChannelInitializer<SocketChannel>() {

    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024, 0, 4, 0, 4));
        ch.pipeline().addLast(new LengthFieldPrepender(4));
        ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
        ch.pipeline().addLast(new RequestMessagePacketEncoder(FastJsonSerializer.X));
        ch.pipeline().addLast(new ResponseMessagePacketDecoder());
        ch.pipeline().addLast(new ClientHandler());
    }
});

Before running- Implementing a lightweight RPC framework Server based on Netty and SpringBoot For the ServerApplication written in, start ClientApplication, and the log output is as follows:

// Server side
2020-01-18 14:32:59 [nioEventLoopGroup-3-2] INFO  club.throwable.server.ServerHandler - Received by the server:RequestMessagePacket(interfaceName=club.throwable.contract.HelloService, methodName=sayHello, methodArgumentSignatures=[java.lang.String], methodArguments=[PooledUnsafeDirectByteBuf(ridx: 0, widx: 11, cap: 11/144)])
2020-01-18 14:32:59 [nioEventLoopGroup-3-2] INFO  club.throwable.server.ServerHandler - Success in finding the target realization method,Target class:club.throwable.server.contract.DefaultHelloService,Host class:club.throwable.server.contract.DefaultHelloService,Host method:sayHello
2020-01-18 14:32:59 [nioEventLoopGroup-3-2] INFO  club.throwable.server.ServerHandler - Server output:{"attachments":{},"errorCode":200,"magicNumber":10086,"message":"Success","messageType":"RESPONSE","payload":"\"throwable say hello!\"","serialNumber":"21d131d26fc74f91b4691e0207826b90","version":1}

// Client
2020-01-18 14:32:59 [nioEventLoopGroup-2-1] INFO  club.throwable.client.ClientHandler - Response packet received,content:{"attachments":{},"errorCode":200,"magicNumber":10086,"message":"Success","messageType":"RESPONSE","payload":{"contiguous":true,"direct":true,"readOnly":false,"readable":true,"writable":false},"serialNumber":"21d131d26fc74f91b4691e0207826b90","version":1}
2020-01-18 14:32:59 [main] INFO  c.throwable.client.ClientApplication - HelloService[throwable]Call result:"throwable say hello!"
2020-01-18 14:33:04 [CLIENT_HOUSE_KEEPER] INFO  c.t.client.ContractProxyFactory - Start execution ResponseFutureTable Cleaning task......
2020-01-18 14:33:04 [CLIENT_HOUSE_KEEPER] WARN  c.t.client.ContractProxyFactory - Remove expired requests ResponseFuture,request ID:21d131d26fc74f91b4691e0207826b90

It can be seen that the asynchronous thread model has been transformed into synchronization, and now the server can be called synchronously through RPC through the contract interface.

Summary

The basic transformation of Client-side request response synchronization has been completed. So far, an RPC framework has been basically completed. Next, some modifications will be made to the Client-side and Server-side to let the contract related components be hosted in the IOC container and realize the automatic contract interface injection and other functions.

Demo project address:

(end of this paper e-a-20200118 c-2-d)

Published 7 original articles, won praise 6, visited 10000+
Private letter follow

Tags: Netty SpringBoot JSON codec

Posted on Sat, 18 Jan 2020 02:32:15 -0500 by scotch33