[Challenge Learning 100 Days Sprint Interview] 21: BIO and NIO for Java Network Programming

Occasionally do an internship, then year appraisal and other miscellaneous things (also shake, king...)

Some basic concepts

Blocking and non-blocking

Refers to the state of the caller before the result of the call is returned

Blocking: No other action can be taken while making a request waiting for the result of the request

Non-blocking: Additional actions can be performed while making a request waiting for the result of the request

Synchronization and Asynchronization

Refers to the difference in communication mechanism, waiting for the result of the call

Synchronization: You must wait for the result after the call to return

Asynchronous: You can return without knowing the result until you receive it later

Four permutation combinations

Based on blocking and non-blocking, synchronous and asynchronous, IO operations can be divided into four categories:

  • Synchronization blockage (waiting for a reply while telling the story, nothing else)
  • Sync is non-blocking (waiting for a reply when you tell, but doing something else when you wait)
  • Asynchronous blocking (message to go home after confession, etc., but always thinking about results)
  • Asynchronous, non-blocking (message home on complaint, etc., and lots of things to do at the same time)

The Evolution History of Network Programming

Beginning with BIO, falling into NIO, finally AIO

Synchronous Block IO ->Synchronous Block IO and Non-Block Mode (NIO No/New Block IO jdk1.4) ->Asynchronous Non-Block I/O Model (AIO Asynchronous IO jdk1.7)

Network Layer Resolution and Protocol

Resolution of URL s

Resolution of domain names

The domain name is resolved from right to left. Take www.google.com.root as an example to see the hierarchy of the domain name. Root is omitted by default.

Two ways of domain name DNS query

DNS has a distributed mapping database. DNS queries are divided into recursive queries and iterative queries. The domain name server that passes through the query will be cached whenever the IP address is queried for the next query.

Recursive query: DNS client queries the root domain name, returns directly if the root domain name knows the IP address, queries the top-level domain name if it doesn't know, and...

Iterative query: If the root domain name knows the I address, it returns directly. If it does not know, it returns the top-level domain name that can be queried. DNS client goes to the top-level domain name to query. If it knows, it returns directly. If it does not know, it returns the second-level domain name.

Network Protocol Fast Literacy

We are actually converting the web page information into 0 and 1 in the electrical signal, passing it through the cable to the other side, which then parses and loads the electrical signal layer into the browser.

After layering the network transport, each layer only needs to depend on (adapted to) the next layer, so there is much less to worry about. When you make changes, you don't have to worry about adapting to other layers, you just need to worry about one layer.

LayeredintroduceCommon ProtocolsUpward Transfer Dependency
application layerUser-accessible applicationsHTTP FTP SMTPapplication program
transport layerConnection between portsTCP UDPport
network layerConnection between host and hostIPIP Address
link layerTransport of network cards and network cardsEthernetmac address
Entity LayerTransmission of electrical signals through physical connectionselectrical signalelectrical signal

Network Layer Packet Formats

LayeredPacket Format
application layerPartitioning TCP/UDP data: application layer data, etc.
transport layerPartition IP data: TCP/UDP header+TCP/UDP data
network layerPartition Ethernet data: IP header + IP data
link layerFrame: 1500(Ethernet data) +18(Ethernet header saves mac address, etc.) bytes

Basic knowledge of network programming

The nature of network programming

Network programming is essentially interprocess communication, which is actually the input and output of data. So we need to understand the input and output model. Data input from data source to application process is input stream, and vice versa. Data source can be file, string, object, Socket, etc.

Streams from java.io

Byte streams operate in bytes, or 8bit, which humans often read character by character. To reduce the extra effort to convert, Java helps us do this.

Character Flow Introduction

Let's first look at the common character streams:

Reader and Writer, which provide additional functionality, need to be based on the basic Reader and Writer. InputStreamReader is a bridge connecting byte streams to character streams

Byte Stream Introduction

Kangkang's most basic byte stream object:

Add some functional byte stream objects to the base byte stream, DataInputStream, or help us convert directly to Java data types:

Decorator modes present in streams

Whether it's a character stream or a byte stream, we have some advanced objects, but these objects are based on some basic objects. We have the basic object functionality and provide better functionality. BufferedInputStream, for example, provides a cache, which is decoration.

Overview of Socket

Sockets are also a kind of data source. Sockets can transmit data streams when they are bound. How can sockets send data? In fact, network cards are needed to transmit data. First, the application creates a socket, then the ip address, port and socket are bound to the driver of the network card. Then the information is sent to the socket first, then the socket is sent to the driver of the network card.Network Card sends information from hardware level

Similarly, when receiving data, network cards collect data from remote network cards and send it to socket s, which send information to application processes

Thread pool for network communication

It would be wasteful to create a thread for each request and recycle it, and we can't create endless threads because of operating system constraints. Java provides many tools to implement thread pools. By implementing the ExecutorService interface, two different types of tasks are provided. Runnable is the value that is not returned and Callable is needed

Ultimately, what we get from the thread pool is a Future object, which represents the state in which the task is ultimately completed, such as isDone() indicating whether the task is now complete or not, and the result can be obtained using get() method.

[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-D8EhElGX-163146540) (C:UsersforevAppDataRoamingTyporatypora-user-imagesimage-2021.png)]

Executors provides many static methods to help us create thread pools

BIO Blocking Model

socket and serversocket

Sockets are client-side, server socket s are server-side, and the connection process is usually as follows

BIO Blocking Model

One thread, Acceptor, accepts requests from other threads, creates a new thread to connect to the client, and the same applies to another client

Pseudo-Asynchronous IO Programming Model (Thread Pool)

Optimize using a thread pool, otherwise too many threads are wasted, use threads if they are available, and wait until they are available, which improves the reliability and scalability of the system

NIO Non-Blocking Model

Overview of NIO

ServerSocket's appcet is blocked, InputStream's read and OutStream's write are blocked, and multiple Stream IO s cannot be processed in the same thread, that is, a user blocks the entire thread, in BIO we can use multithreading and then use thread pools to optimize. We can use a non-blocking form to process data input and output, that is, NIO.

NIO can be interpreted as new or NoblockingIO. Instead of using streams, we use Channels instead of Stream. Streams are directional and one-way, but Channels are bidirectional; read and write of streams are blocked, and Channels can be blocked or non-blocked.

NIO also provides a Selector, a Selector that monitors multiple Channels, such as when we want to read data from a non-blocking Channel, but when we want to use the read data we don't know if the Channel is finished, we can use the Selector.

At the same time, NIO can handle multiple Channel I/O s in one thread. Multithreading does not necessarily improve efficiency. Too many threads can cause context exchange pressure, and creating or destroying a thread can also waste system resources.

A Brief Analysis of Buffer

We use Channel to read and write data, which is actually a Buffer. For a Channel operation, we cannot do without its Buffer operation. Channel supports two-way operation, so Buffer can also operate in both directions, and buffers are big and small.

  • Change buffer from write mode to read mode using flip() method
  • clear() method changes buffer from read mode to write mode (actually returning position, limit pointer to its original position)
  • The compact() method puts the data left over from the buffer read and modified to the write mode on top, the position immediately follows below it, and the write starts from the position, without overwriting the data that was not read before. The next flip() read is the first one

For dynamic processes, you can download this original video null

Channel Analysis

Channel operates through Buffer and different Channels can also transfer data directly. A few Channel classes are briefly introduced

Copy files in multiple ways

  • Write byte by byte without using any buffers
  • Read buffers one at a time using buffers
  • Channel uses its buffer operation
  • Two Channel s directly transfer data

Test small file (179 KB) time: 1783, 2, 13, 1

Testing large files (36.2 MB) takes time:

interface FileCopyRunner {
    void copyFile(File source,File target);
}
public class FileCopyDemo {

    private static final int ROUNDS=5;

    private static void bencgmark(FileCopyRunner fileCopyRunner,File source,File target) {
        long elapsed=0L;
        for (int i=0;i<ROUNDS;i++) {
            long startTime = System.currentTimeMillis();
            fileCopyRunner.copyFile(source, target);
            elapsed += System.currentTimeMillis()-startTime;
            target.delete();
        }
        System.out.println(fileCopyRunner+":"+elapsed/ROUNDS);
    }

    private static void close(Closeable closeable) {
        if (closeable != null) {
            try {
                closeable.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    public static void main(String[] args) {
        //Write byte by byte without using any buffers
        FileCopyRunner noBufferStreamCopy = new FileCopyRunner() {
            @Override
            public void copyFile(File source, File target) {
                InputStream fin = null;
                OutputStream fout = null;
                try {
                    fin = new FileInputStream(source);
                    fout = new FileOutputStream(target);
                    int read;
                    while ((read = fin.read())!=-1) {
                        fout.write(read);
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    close(fin);
                    close(fout);
                }
            }
        };
        //Read buffers one at a time using buffers
        FileCopyRunner bufferedStreamCopy = new FileCopyRunner() {
            @Override
            public void copyFile(File source, File target) {
                BufferedInputStream fin = null;
                BufferedOutputStream fout = null;
                try {
                    fin = new BufferedInputStream(
                            new FileInputStream(source)
                    );
                    fout = new BufferedOutputStream(
                            new FileOutputStream(target)
                    );
                    byte[] buffer = new byte[1024];
                    int result;
                    while ((result = fin.read(buffer))!=-1) {
                        fout.write(buffer,0,result);
                        fout.flush();
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    close(fin);
                    close(fout);
                }
            }
        };
        //Channel uses its buffer operation
        FileCopyRunner nioBufferCopy = new FileCopyRunner() {
            @Override
            public void copyFile(File source, File target) {
                FileChannel fin = null;
                FileChannel fout = null;
                try {
                    fin = new FileInputStream(source).getChannel();
                    fout = new FileOutputStream(target).getChannel();
                    ByteBuffer buffer = ByteBuffer.allocate(1024);
                    while ((fin.read(buffer)!=-1)) {
                        buffer.flip();
                        //Read All
                        while (buffer.hasRemaining()) {
                            fout.write(buffer);
                        }
                        buffer.clear();
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    close(fin);
                    close(fout);
                }
            }
        };
        //Two Channel s directly transfer data
        FileCopyRunner nioTransferCopy = new FileCopyRunner() {
            @Override
            public void copyFile(File source, File target) {
                FileChannel fin = null;
                FileChannel fout = null;
                try {
                    fin = new FileInputStream(source).getChannel();
                    fout = new FileOutputStream(target).getChannel();
                    long transferred=0L;//Record how many bytes were copied in total
                    long size = fin.size();
                    while (transferred != size) {
                        //Where to start, how much to transfer, and where to transfer
                        transferred += fin.transferTo(0, size, fout);
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    close(fin);
                    close(fout);
                }
            }
        };
        File smallFile = new File("E:\\IdeaProjects\\EasyIO\\resources\\smallFile.jpg");
        File smallFileCopy = new File("E:\\IdeaProjects\\EasyIO\\resources\\smallFileCopy.jpg");
        bencgmark(noBufferStreamCopy,smallFile,smallFileCopy);
        bencgmark(bufferedStreamCopy,smallFile,smallFileCopy);
        bencgmark(nioBufferCopy,smallFile,smallFileCopy);
        bencgmark(nioTransferCopy,smallFile,smallFileCopy);
    }
}

Selector Analysis

Monitoring the status of multiple channels requires that we register the Channel with the Selector, and for different types, we can have a set of logic

Common methods used in Selector are as follows:

  • innterestOps(): Status of registration
  • readyOps(): What are the States Channel is ready for and operational
  • channel(): Returns a Channel object
  • selector(): Returns a Seletor object
  • attachment: Each Channel object adds an Object object, which can be any secondary object

NIO Optimized Chat Room

Tags: Java Interview udp

Posted on Sun, 12 Sep 2021 12:09:51 -0400 by markduce