Protocol resolution is essential in daily network development. Although length-based, delimiter-based codecs are built into Netty, in most scenarios we use custom protocols, so Netty provides MessageToByteEncoder<I> and ByteToMessageDecoder Two abstract classes that encode and decode private protocols by inheriting overrides of their encode and decode methods.In this article, we will practice and analyze the custom codec in Netty.
I. Use of codecs
Below is a simple example of the use of MessageToByteEncoder and ByteToMessageDecoder, with no specific protocol coding involved.
Create a sever-side service
EventLoopGroup bossGroup = new NioEventLoopGroup(1); EventLoopGroup workerGroup = new NioEventLoopGroup(); final CodecHandler codecHandler = new CodecHandler(); try { ServerBootstrap b = new ServerBootstrap(); b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class).option(ChannelOption.SO_BACKLOG, 100) .handler(new LoggingHandler(LogLevel.INFO)).childHandler(new ChannelInitializer<SocketChannel>() { @Override public void initChannel(SocketChannel ch) throws Exception { ChannelPipeline p = ch.pipeline(); if (sslCtx != null) { p.addLast(sslCtx.newHandler(ch.alloc())); } //Add Codec handler p.addLast(new MessagePacketDecoder(),new MessagePacketEncoder()); //Add Customization handler p.addLast(codecHandler); } }); // Start the server. ChannelFuture f = b.bind(PORT).sync();
Inherit MessageToByteEncoder and override the encode method for encoding
public class MessagePacketEncoder extends MessageToByteEncoder<byte[]> { @Override protected void encode(ChannelHandlerContext ctx, byte[] bytes, ByteBuf out) throws Exception { //Processing specific encoding Print the byte array here System.out.println("Encoder receives data:"+BytesUtils.toHexString(bytes)); //Write and transfer data out.writeBytes(bytes); } }
Inherit ByteToMessageDecoder and override decode method for decoding
public class MessagePacketDecoder extends ByteToMessageDecoder { @Override protected void decode(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out){ try { if (buffer.readableBytes() > 0) { // Message Packets to Process byte[] bytesReady = new byte[buffer.readableBytes()]; buffer.readBytes(bytesReady); //Conduct specific decoding System.out.println("Decoder receives data:"+ByteUtils.toHexString(bytesReady)); //Don't overdo it here, just put the received message in the list and pass it back out.add(bytesReady); } }catch(Exception ex) { } } }
Implement a custom message processing handler, where you actually get the coded data
public class CodecHandler extends ChannelInboundHandlerAdapter{ @Override public void channelRead(ChannelHandlerContext ctx, Object msg) { System.out.println("CodecHandler Data received:"+ByteUtils.toHexString((byte[])msg)); byte[] sendBytes = new byte[] {0x7E,0x01,0x02,0x7e}; ctx.write(sendBytes); } @Override public void channelReadComplete(ChannelHandlerContext ctx) { ctx.flush(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { // Close the connection when an exception is raised. cause.printStackTrace(); ctx.close(); } }
Run a client simulation to send bytes 0x01,0x02 to see the output execution results
Decoder receives data: 0102 CodecHandler received data: 0102 Encoder receives data: 7E01027E
Based on the output, you can see that the inbound and outbound messages are delivered in a sequence customized in the pipeline, and the specific protocol codec operations we need are implemented by rewriting the encode and decode methods.
2. Source Code Analysis
From the example above, you can see that MessageToByteEncoder <I> and ByteToMessageDecoder inherit ChannelInboundHandlerAdapter and ChannelOutboundHandlerAdapter respectively, so they are also implementations of channelHandler.And is added to the pipeline when sever is created.At the same time, for our convenience, netty has built-in and encapsulated some of its operations in these two abstract classes; outbound and inbound messages trigger both the write and channelRead event methods, respectively, so the encode and decode methods we override in the above example are also called in the write and channelRead methods of the parent class. Let's start with these two methods separately and edit the whole programThe decoding process is sorted out and analyzed.
1,MessageToByteEncoder
Encoding requires outbound data, so our overridden encode implementation is invoked in the write method of MesageToByteEncoder to encode our internally defined message entity as the final byte stream data to be sent.
@Override public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception { ByteBuf buf = null; try { if (acceptOutboundMessage(msg)) {//Determine incoming msg Is it consistent with the type you defined @SuppressWarnings("unchecked") I cast = (I) msg;//Convert to message type defined by you buf = allocateBuffer(ctx, cast, preferDirect);//Package as one ByteBuf try { encode(ctx, cast, buf);//Incoming declarative ByteBuf，Perform specific encoding operations } finally { /** * If the type you define is ByteBuf, this can help you free resources without having to do it yourself * If you define a message type that contains ByteBuf, this is not useful and you need to release it yourself */ ReferenceCountUtil.release(cast);//Release your incoming resources } //Send out buf if (buf.isReadable()) { ctx.write(buf, promise); } else { buf.release(); ctx.write(Unpooled.EMPTY_BUFFER, promise); } buf = null; } else { //Send directly without execution if type is inconsistent encode Method, so here's a note that if you pass a message that doesn't match the generic type, it won't actually execute ctx.write(msg, promise); } } catch (EncoderException e) { throw e; } catch (Throwable e) { throw new EncoderException(e); } finally { if (buf != null) { buf.release();//Release Resources } } }
The MessageToByteEncoder's write method achieves the simple function of converting and sending your incoming data types; here are two points to note:
- Typically, you need to override the encode method to convert the generic type you defined to the ByteBuf type, and the write method automatically performs a pass or send operation for you.
- Although passed in code ReferenceCountUtil.release(cast) Release the type resource you defined, but if the defined message class contains a ByteBuf object, you still need to actively release the object resource;
2,ByteToMessageDecoder
As you can see from the naming, the ByteToMessageDecoder is used to convert the byte stream data encoding to the data format we need
As an inbound event, the entry to the decoding operation is naturally the channelRead method
@Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { if (msg instanceof ByteBuf) {//If the message is bytebuff CodecOutputList out = CodecOutputList.newInstance();//Instantiate a list of chains try { ByteBuf data = (ByteBuf) msg; first = cumulation == null; if (first) { cumulation = data; } else { cumulation = cumulator.cumulate(ctx.alloc(), cumulation, data); } callDecode(ctx, cumulation, out);//Start decoding } catch (DecoderException e) { throw e; } catch (Exception e) { throw new DecoderException(e); } finally { if (cumulation != null && !cumulation.isReadable()) {//Not empty and no readable data, freeing up resources numReads = 0; cumulation.release(); cumulation = null; } else if (++ numReads >= discardAfterReads) { // We did enough reads already try to discard some bytes so we not risk to see a OOME. // See https://github.com/netty/netty/issues/4275 numReads = 0; discardSomeReadBytes(); } int size = out.size(); decodeWasNull = !out.insertSinceRecycled(); fireChannelRead(ctx, out, size);//Send message down out.recycle(); } } else { ctx.fireChannelRead(msg); } }
The callDecode method decodes ByteBuf data internally through a while loop until there is no readable data in it
protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) { try { while (in.isReadable()) {//judge ByteBuf Yes, there is still readable data int outSize = out.size();//Get record list size if (outSize > 0) {//Determine if there is already data in the list fireChannelRead(ctx, out, outSize);//If any data continues to be passed down out.clear();//Empty Chain List // Check if this handler was removed before continuing with decoding. // If it was removed, it is not safe to continue to operate on the buffer. // // See: // - https://github.com/netty/netty/issues/4635 if (ctx.isRemoved()) { break; } outSize = 0; } int oldInputLength = in.readableBytes(); decodeRemovalReentryProtection(ctx, in, out);//Start Call decode Method // Check if this handler was removed before continuing the loop. // If it was removed, it is not safe to continue to operate on the buffer. // // See https://github.com/netty/netty/issues/1664 if (ctx.isRemoved()) { break; } //Here if the list is empty and bytebuf Jump out of the loop without readable data if (outSize == out.size()) { if (oldInputLength == in.readableBytes()) { break; } else {//Readable data to continue reading continue; } } if (oldInputLength == in.readableBytes()) {//beytebuf Not read, but decoded throw new DecoderException( StringUtil.simpleClassName(getClass()) + ".decode() did not read anything but decoded a message."); } if (isSingleDecode()) {//Whether to set each inbound data to be decoded only once, default false break; } } } catch (DecoderException e) { throw e; } catch (Exception cause) { throw new DecoderException(cause); } }
The decodeRemovalReentryProtection method calls our overridden decode implementation internally
final void decodeRemovalReentryProtection(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception { decodeState = STATE_CALLING_CHILD_DECODE;//Tag Status try { decode(ctx, in, out);//Call us to override decode Decoding implementation } finally { boolean removePending = decodeState == STATE_HANDLER_REMOVED_PENDING; decodeState = STATE_INIT; if (removePending) {//Judge markers here to prevent handlerRemoved Events conflict with decoding operations handlerRemoved(ctx); } } }
The channelRead method accepts data through a series of logical processing and eventually invokes our overridden decode method to implement specific decoding functions; in the decode method, we only need the data of type ByteBuf to be parsed into the data format we need to be placed directly in the List <Object> out chain table, and ByteToMessageDecoder will automatically help you pass messages down.
3. Summary
From the explanations above, we can learn a little about the built-in custom codecs MessageToByteEncoder and ByteToMessageDecoder in Netty, which are essentially a set of channelHandler implementation classes encapsulated by Netty specifically for custom coding.In the actual development, the implementation based on these two abstract classes is very practical, so we will make a little analysis here, in which some deficiencies and inaccuracies are expected to point out and culvert.
Watch the WeChat Public Number for more technical articles.
Reproduction instructions: Unauthorized not to reproduce, after authorization must indicate the source (Note: from public number: architecture space, author: generic)