Basic | network programming

1, Related concepts of computer network

1. Computer network

Through transmission medium, network protocol and communication facilities, computers scattered in different locations are interconnected to realize resource sharing and data transmission system.

Functions of computer network:
1. Resource sharing
2. Information transmission and centralized processing
3. Balanced load and distribution treatment
4. Comprehensive information service

2. Network programming

Socket programming, also known as socket programming, refers to the use of computer programming language to realize resource sharing and information transmission between computers under the management and coordination of operating system, network management software and network communication protocol.

2, Three elements of computer network:

1.IP address

IP address is a uniform address format provided by IP protocol. It assigns a logical address to every network and every host on the Internet. It refers to the unique identification of the computer in the network. The length of the IP address is 32 bits (4 bytes), which is generally represented by "dotted decimal".

classification First byte start bit First byte value range Network format address Maximum number of networks Maximum hosts per network
Class A 0 0-127 Network. Host. Host. Host 127 16777214
Class B 10 128-191 Network, network, host, host 16384 65534
Class C 110 192-223 Network. Network. Network. Host 2097152 254
Class D 1110 224-239 For multicast in IP network, no longer assigned
Class E 1111 240-255 Reserved for research, no longer allocated

2. Port number:

The port number identifies the logical address of the process; its valid ports range from 0 to 65535, where 0-1024 systems use or reserve ports. Note: different software communication, port number can not be the same, there can be no conflict.

Port number classification Explain
Public port 0-1023. These ports are reserved for the operating system. They are tightly bound to some services and are not generally used
Registration port 1024-49151, these ports can be used, but system occupation is not excluded
Private port 49152-65535, port number for user

3. agreement

The collection of communication rules among all devices on the network, which specifies the format that information must adopt when communicating, the meaning of these formats, and the transmission mode of data, etc.

Common protocols and port numbers are:

Protocol name Protocol specification Port number
FTP File transfer protocol 21
HTTP Hyper Text Transfer Protocol 80
HTTPS Encrypted Hypertext Transfer Protocol 443
POP3 Receive mail protocol 110
SMAP Send mail protocol 25
IMAP Internet Mail Access Protocol 143
DNS Domain name resolution 53

4. Resource search method

Through the IP address to find the computer, through the port number to find the software, through the protocol to agree on the format of data transmission.

3, Network model

For example, chestnut: A sends A "helloworld" message to B. according to the different use protocol (TCP/UDP), it needs to go through n-layer model, step-by-step transformation until the message "helloworld" is received by the other party.

What kind of rules are used to communicate between computer networks is the problem studied by the network model. The actual meaning is to build a certain model to realize the transmission of network data. In the middle, the data will be encapsulated and unpacked many times to achieve the lowest transmission requirements.

Network model generally refers to OSI seven layer reference model and TCP/IP four layer reference model.

1. The OSI model divides the network into seven layers. The meaning of each layer is as follows:

  • physical layer
    It mainly defines physical equipment standards, such as interface type of network cable, interface type of optical fiber, transmission rate of various transmission media, etc.
    The main function is to encode the data into the bit stream identified by 0 and 1, and transmit it through the physical medium. The data in this layer is called bits.
  • data link layer
    It mainly provides media access and link management. This layer encapsulates and unpacks the received data with MAC address (network card address). The data in this layer is often called frame. The device that works in this layer is the switch.
  • Network layer:
    This layer encapsulates and de encapsulates the received data. This layer of data is often called a packet. This layer of equipment is router.
  • Transport layer:
    Some data transmission protocols and port numbers are defined. It mainly establishes, manages and maintains the end-to-end connection, which segments and transmits the received data, and reorganizes the data after reaching the destination address. The data in this layer is often called segment.
  • Session level:
    Establish the data transmission path through the transmission layer, and then establish, manage and maintain the dialogue. It mainly initiates or receives session requests between systems.
  • Presentation layer:
    Mainly for the interpretation of the received data, format conversion, encryption and decryption, compression and decompression. Ensure that the data sent by the application layer of one system can be identified by the application layer of another system.
  • Application layer:
    It mainly provides services for some terminal applications. Facing users directly.

2. The TCP / IP protocol cluster divides the network into four layers. The meaning of each layer is as follows:

  • Network interface layer: establish, maintain and disconnect physical connection
  • Network layer: handling the activities of packets in the network
  • Transport layer: provides end-to-end communication for applications on both hosts
  • Application layer: responsible for handling specific application details

4, TCP protocol

1. Meaning of TCP message segment header and each field

  • Source port and destination port: 16 bits each, each TCP message segment contains source port and destination port, which are used to find the application process of sender and receiver. Generally speaking, the port number and IP address can uniquely determine a TCP connection.
  • Serial number: 32 bits, used to identify the data byte stream sent from the sender to the receiver.
  • Confirmation serial number: 32 bits, including the next data byte of the source segment expected to be received by the target end. The confirm sequence number field is only valid when ACK flag bit 1.
  • Data offset: 4 bits, used to indicate the length of the TCP header. If this value does not exist, the default length of the header is 20 bytes, the maximum value of the data offset is 60 bytes, and the maximum can be expanded by 40 bytes
  • Flag bit: 1 bit each
    ① URG: "emergency pointer" is valid
    ② ACK: confirm serial number is valid
    ③ PSH: the receiver directly handles the current package as an emergency: pass it to the application layer
    ④ RST: discard the packet and reconnect
    ⑤ SYN: used to synchronize the serial number when establishing a connection
    ⑥ FIN: release connection
  • Window: 16 bits, used for flow control and congestion control, indicating the size of current receiving buffer. In the computer network, the receiving capacity of the receiver is usually used to control the amount of data sent. So the maximum value represented by the current window is 65535.
  • Check and: 16 bits, covering the whole TCP message segment (header and data). This is a mandatory field, which must be calculated and stored by the sender and verified by the receiver.

2.TCP connection establishment: three handshakes process


Where: seq is the serial number of the packet itself; ack is the serial number of the packet that the other party is expected to continue sending.

Process Description:

  1. The client sends the connection request message segment to the server and enters the SYN-SENT state, waiting for the server to confirm
  2. The server receives the connection request message. If the server agrees to establish the connection, it will send the confirmation message segment to the client, and allocate the TCP cache and variables for the TCP connection
  3. After receiving the confirmation message segment from the server, the client sends the confirmation message segment to the server, and also allocates cache and variable to the connection. After this packet is sent, the client and the server enter the ESTABLISHED state and complete three handshakes, so far the TCP connection is successful.

3. Release of connection: four wave process


Process Description:

  1. TCP client sends a FIN to close data transmission from client to server
  2. The server receives the FIN and sends back an ACK to confirm that the SN is the received Sn + 1. Like SYN, a FIN will occupy a Sn
  3. The server closes the client connection and sends a FIN to the client
  4. The client sends back ACK message for confirmation, and sets the confirmation serial number to the received serial number + 1

Why does A need to enter the TIME-WAIT state first and wait for 2MSL before entering the CLOSED state? In order to ensure that B can receive the confirmation response from A.
If A directly enters the CLOSED state after sending the confirmation reply, then if the reply is lost, B will resend the connection release request after the timeout of waiting, but at this time A has been CLOSED and will not make any response, so B will never be able to shut down normally.

Why do TCP connections use three or four handshakes?

The difference between TCP and UDP:
TCP is connection oriented (three handshakes will be performed before connection establishment), UDP is connectionless
TCP reliable, UDP unreliable
TCP is oriented to byte stream, and UDP is based on datagram (the messages in application layer are packed separately and transmitted to the receiving end in message mode)
TCP guarantees data correctness, UDP may lose packets
The information provided by the header is different. UDP has no connection service and retransmission function

5, IO model

Before introducing the IO model, we need to understand the concepts of [synchronous / asynchronous] and [blocking / non blocking]
(1) Synchronous / asynchronous: for application and kernel space interactions.
Synchronization refers to the user process triggering IO operation and waiting or polling to see whether the IO operation is ready. Asynchronous refers to the user process can do its own thing after triggering IO operation. When the IO operation has been completed, the user will be notified of the completion of IO operation.
(2) Blocking / non blocking: when a process accesses data, it adopts different ways according to the ready state of IO operation, which is an implementation way of read / write operation function. >In blocking mode, the read or write function will wait all the time. In non blocking mode, the read or write function will immediately return a status value.

1. Synchronous blocking model

Take a chestnut: Xiaoming goes to the restaurant, orders a scrambled egg with tomato, and then waits for the dish to be prepared in the position all the time. He brings it to the table for dinner. This is the synchronous blocking. During the cook's cooking time, Xiao Ming needs to wait all the time.

In this way, after an IO operation is initiated, the user process must wait for the completion of the IO operation. Only after the IO operation is completed can the user process run. JAVA traditional IO model belongs to this way.

2. synchronous non blocking model

Xiao Ming goes to the restaurant and orders a scrambled egg with tomato for dinner. It's estimated that the time is almost over. I'll ask the boss if the dish is ready. If it's ready, I'll serve it. If it's not, I'll wait for a while and ask again. This cycle lasts until the dish is ready for Xiaoming to eat. This is synchronous non blocking. Xiao Ming can do his own thing in the interval of asking the boss about the meal.

In this way, the user process can return to do other things after initiating an IO operation, but the user process needs to ask if the IO operation is ready from time to time, which requires the user process to keep asking, so as to introduce unnecessary CPU resource waste. Currently, NIO of JAVA belongs to synchronous non blocking io.

3.IO reuse model

Or Xiaoming, ordered a few dishes, and then said to the waiter, "if any dishes are ready, you tell me, I'll take them." This is the IO reuse model. Before the waiter calls Xiaoming, Xiaoming can do his own thing.

In this way, the user process needs to process multiple IOS. The process of IO multiplexing is divided into two steps: the first system call is used to request data until the data is copied to the memory buffer; the second system call needs to be initiated to synchronize data, which is still blocked. Although IO multiplexing still causes blocking, when there are many IO operation objects, a large part of blocking time can be avoided.

Why is it blocked?
Because this is done through the select system call, and the implementation of the select function itself is blocked,
The advantage of using select function is that it can listen to multiple file handles at the same time, so as to improve the concurrency of the system

4. Signal driven model

It's Xiaoming again. He goes to the restaurant and orders some dishes. Then he says to the boss, "if any dishes are ready, please tell me. I'll take them." This is the signal driven IO model. Before the boss calls Xiaoming, Xiaoming can do his own thing.

In this way, the user can let the kernel send SIGIO signal to inform us when the descriptor is ready. Using this model, we need to first turn on the socket's signal driven IO function and install a signal processing function through the sigaction system call.

5. Asynchronous blocking model

This time, Xiao Ming didn't want to take the dishes himself. He told the boss that if the dishes are ready, you can bring them directly, or Xiao Ming can take them to his home.

In this way, the user process only needs to initiate an IO operation and return immediately. After the IO operation is completed, the application program will be notified that the IO operation is completed. At this time, the user process only needs to process the data, and no actual IO read / write operation is required, because the real IO read / write operation has been completed by the kernel. At present, this IO model is not supported in Java.

Compared with signal driven IO, the main differences of asynchronous IO are as follows:
The signal driver tells us when we can start an IO operation (the data is in the kernel buffer), while the asynchronous IO tells us when the IO operation has completed (the data is in the user space).

6, Network programming model

Based on the above five IO models, in Java, with the introduction of NIO and AIO, there are generally the following network programming models: BIO, NIO, AIO

1.BIO synchronous blocking model

Before JDK 1.4, we used the BIO model to process the data in the way of stream (low efficiency) when establishing the network connection.

The general process of realizing C/S by BIO:
① Start a ServerSocket instance on the server side
② Start the Socket instance at the client to communicate with the server
By default, the server needs to set up a heap of thread waiting requests for each request. After the client sends the request, first ask whether there is a thread corresponding to the request. If there is no thread, it will wait all the time or be rejected. If there is one, the client will wait for the end of the request before continuing.
It can be seen that the server implementation mode is one connection one thread, that is, when the client has a connection request, the server needs to start a thread for processing. If the link does not do anything, it will cause unnecessary thread overhead.

  • Server programming
import java.io.*;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;

/**
 * BIO Server programming
 */
public class BIOEchoServerDemo {
    public static void main(String[] args) {
        try {
            //Create ServerSocket instance
            ServerSocket serverSocket = new ServerSocket();

            //Binding port
            serverSocket.bind(new InetSocketAddress(8889));
            System.out.println("Server started");

          
            //Listen for and receive connections from this client
            Socket socket = serverSocket.accept();
            System.out.println("Client"+socket.getRemoteSocketAddress()+"Connect to server");
            //Read write business logic
            InputStream inputStream = socket.getInputStream();
            //Read the message from the client and store it in the reader
            BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
            //Reply to client message
            OutputStream outputStream = socket.getOutputStream();
            String msg;
            while ((msg = reader.readLine()) != null) {
                  System.out.println(socket.getRemoteSocketAddress() + ":" + msg);
                  outputStream.write((msg + "\n").getBytes());
                  outputStream.flush();
            }
            //close resource
            reader.close();
            socket.close();
            serverSocket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
  • Client programming
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.util.Scanner;
/**
 * BIO Client programming
 */
public class BIOEchoClientDemo {
    public static void main(String[] args) {
        try {
            //Create socket instance
            Socket socket = new Socket();
            //Connect to the server
            socket.connect(new InetSocketAddress("127.0.0.1",8889));//Consistent with the server
            //Read and write operations
            OutputStream outputStream = socket.getOutputStream();
            //Receive the message returned by the server
            BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
            Scanner scanner = new Scanner(System.in);
            String sendMsg;
            while(scanner.hasNext()){
                sendMsg=scanner.next();
                if(sendMsg!=null) {
                    //Send message to server
                    outputStream.write((sendMsg+"\n").getBytes());
                    //Content waiting for the server to return
                    String recvMsg = reader.readLine();
                    System.out.println("[echo];" + recvMsg);
                }
            }
            //close resource
            socket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

2.NIO synchronous non blocking model

NIO is an improvement of BIO, a synchronous non blocking IO model, which processes data in block mode (high efficiency). NIO operates on data based on the channel and buffer, and the data is always read from the channel to the buffer or written to the channel from the buffer. The Selector is used to listen to the time of multiple channels (such as connection request, data arrival, etc.), so a single thread can be used to listen to multiple client channels.

The general process of NIO to realize C/S:
Client:
① Connect to remote server through SocketChannel
② Create a read / write data buffer object to read or send data to the server
③ Close SocketChannel
Server:
① Binding ip address and port number through ServerSocketChannel
② Create a SocketChannel object through the accept() method of ServerSocketChannelImpl. The user reads / writes data from the client
③ Create a read-write data buffer object to read or send data to the client
④ Close SocketChannel and ServerSocketChannel

  • Server programming
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.*;
import java.util.Iterator;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * NIO Server programming
 */
public class NIOServerDemo {
    public static void main(String[] args) {
        try {
            //Create ServerSocketChannel channel channel instance
            ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
            //Binding port
            serverSocketChannel.bind(new InetSocketAddress(8889));
            System.out.println("Server started");
           
            //Set serverSocketChannel to non blocking
            serverSocketChannel.configureBlocking(false);//true: blocking; false: non blocking
            //Instantiate io multiplexer
            Selector selector = Selector.open();
            //Register the serverSocketChannel to the multiplexer and listen for the accept event
            serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

            //TODO

            int num;
            //If there is an event in the multiplexer, select() will be blocked until the event is completed
            while ((num = selector.select()) > 0) {
                  //Gets the result set of listening events, including the set of completed events
                  Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
                  while (iterator.hasNext()) {
                       SelectionKey selectionKey = iterator.next();
                       iterator.remove();

                       //If there is an acceptable event completed
                       if (selectionKey.isAcceptable()) {
                           //Get ServerSocketChannel instance
                           ServerSocketChannel channel = (ServerSocketChannel) selectionKey.channel();

                           //Receive the user's connection, the result of this acquisition will not block
                           SocketChannel socketChannel = channel.accept();
                           System.out.println("Client" + socketChannel.getRemoteAddress() + "Connect to the server");
                           //Read and write operations -- > take read operations as an example
                           //Set socketChannel channel channel to non blocking
                           socketChannel.configureBlocking(false);
                           //Register the socketChannel channel channel to the multiplexer and listen for the read event
                           socketChannel.register(selector, SelectionKey.OP_READ);
                       }
                       //If there is a read event completed
                       if (selectionKey.isReadable()) {
                           //Get the channel instance and read the data through the channel
                           SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
                           //Read
                           ByteBuffer buffer = ByteBuffer.allocate(1024);
                           //Write data to buffer
                           int read = socketChannel.read(buffer);
                           if (read != -1) {
                               //Switch between read and write mode switch between pos and lim pointer
                               buffer.flip();

                               byte[] bytes = new byte[buffer.remaining()];
                               buffer.get(bytes);//Read data
                               String msg = new String(bytes);
                               if (!"".equals(msg) && "bye".equals(msg)) {
                                    //Close socketChannel instance
                                    socketChannel.close();
                               }
                               System.out.println("[recv]" + msg);
                           }
                        }
                    }
                }
                //close resource
                selector.close();
                serverSocketChannel.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
  • Client programming
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;

/**
 * NIO Client programming
 */
public class NIOClientDemo {
    public static void main(String[] args) {
        try {
            //Get SocketChannel instance
            SocketChannel socketChannel = SocketChannel.open();
            //Set socketChannel to non blocking
            socketChannel.configureBlocking(false);
            //Instantiate multiplexer instance
            Selector selector = Selector.open();
            
            boolean isConnect = socketChannel.connect(new InetSocketAddress("127.0.0.1", 8889));
            if(!isConnect){
                //The connection operation is not completed, indicating that the connect event is not completed. Register the socketChannel with the multiplexer, and pay attention to the connect event
                socketChannel.register(selector,SelectionKey.OP_CONNECT);
                //Wait for events in multiplexer to complete
                selector.select();
                Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
                while (iterator.hasNext()){
                    SelectionKey selectionKey = iterator.next();
                    iterator.remove();
                    if(selectionKey.isConnectable()){
                        SocketChannel channel = (SocketChannel)selectionKey.channel();
                        //Connectable event completion
                        channel.finishConnect();
                    }
                }
            }
            //Reading events is equivalent to writing data to buffer
            ByteBuffer buffer = ByteBuffer.allocate(1024);
            buffer.put("helloworld".getBytes());
            //Read / write mode switching
            buffer.flip();
            //Write data to the socketChannel channel, that is, read the data in the buffer
            socketChannel.write(buffer);

            //close resource
            selector.close();
            socketChannel.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

3. Application scenario:

  • BIO
    It is suitable for the architecture with small number of connections and fixed number of connections. This method requires high server resources and is the only choice before JDK 1.4, but the program is simple, intuitive and easy to understand.
  • NIO
    It is suitable for architectures with large number of connections and short connections (light operation), such as chat recommendation server. The concurrency is limited to the application, the programming is complex, and JDK 1.4 starts to support it.

Reference blog:
https://blog.csdn.net/qq_41652863/article/details/90451133
https://www.cnblogs.com/javalyy/p/8882066.html
https://blog.csdn.net/dhrtyuhrthfgdhtgfh/article/details/90271123

Published 24 original articles, won praise 0, visited 1650
Private letter follow

Tags: network Java socket Programming

Posted on Thu, 06 Feb 2020 00:51:28 -0500 by darknuke