Sticky packet problem involved in network sending and receiving data and its solution

1, What is a sticky bag:

Sticky packet means that there is no clear boundary between data and data, resulting in incorrect reading of data
The application cannot directly operate the hardware. If the application wants to send data, it must hand over the data to the operating system, and the operating system needs to provide data transmission services for all applications at the same time, which means that the operating system can't send the data of the application immediately, so it needs to provide data for the application
The sequence provides a buffer for temporarily storing data. The specific process is as follows:

This means that UDP will not stick packets at all, but it will lose data and is unreliable.
It means that TCP data transmission is reliable, but packets will stick.

2, Explain the sticking problem with code (the sticking problem of the sender):

Server side:

from socket import *

# todo 1. Create server-side socket and SOCK_STREAM: TCP based
server_socket = socket(AF_INET, SOCK_STREAM)

# todo 2. Create a target server and bind an IP and port server. The empty string in the server represents the server_socket is bound to any IP address under this machine
host_port = ('', 8080)
server_socket.bind(host_port)

# todo 3. listen to the socket of the server. listen makes the socket passive. At this time, you can receive the connection request of the client
server_socket.listen(5)

# todo 4. Wait for the connection request from the client. The current function is a thread blocking function. accept returns two values. The first is a new socket and the second is the client address.
#  The newly created socket is server_ The sub socket in the socket only sends and receives data with the current client (a client)

new_socket, client_addr = server_socket.accept()

# todo 5. The server receives the data sent by the client. recv is generally used for the end data of TCP protocol, and recvfrom is used for UDP
data1 = new_socket.recv(1024)  # Receive 1kb of data
data2 = new_socket.recv(1024)  # Receive 1kb of data

print('Article 1 data', data1)
print('Article 2 data', data2)

# todo 6. Close the service of the current client
new_socket.close()
# todo 7. Shut down the whole server
server_socket.close()

client:

from socket import *

# todo 1. Create the socket and sock of the client_ Stream: TCP protocol
client_socket = socket(AF_INET, SOCK_STREAM)

# todo 2. The client sends a connection request (not a request to transmit data, but a request to create a connection)
client_socket.connect(('192.168.1.112', 8080))

# todo 3. Send data
client_socket.send('hello'.encode('utf-8'))

client_socket.send('zhil'.encode('utf-8'))

# todo 4. Close the client socket
client_socket.close()

After starting the server and client
The running result is

Article 1 data b'hellozhil'
Article 2 data b''

The above operation results obviously have the problem of sticking package
The client sends two data packets, but when the server receives data1, it accepts all the data of the two packets. This appearance is sticky packets. In fact, if the server point code is changed to recv(2), it will also cause sticky (sticky) packets. The client sends a piece of data, the server receives only a - small part, and sticky packets are also generated.

3, Explain the sticking problem with code (the sticking problem of the receiver):

Server side:

from socket import *
import time

# Packet sticking problem: the packet sticking problem of the receiver
# todo 1. Create server-side socket and SOCK_STREAM: indicates the TCP protocol
server_socket = socket(AF_INET, SOCK_STREAM)

# todo 2. Bind the ip and protocol of the server. An empty string represents the server_socket can bind any ip address under the current machine
server_socket.bind(('', 9999))

# todo 3. listen to the socket on the server. listen makes the socket passive. At this time, you can receive the connection request from the client
server_socket.listen(5)

# todo 4. The current function is a blocking function. The client sends a connection request and accept returns two values. The first is a new socket and the second is the client address
new_socket, client_addr = server_socket.accept()
print('Connection succeeded', client_addr)

# todo 5. Receive the data sent by the client
data1 = new_socket.recv(3)  # First time did not receive complete
print('First packet', data1.decode('utf-8'))

time.sleep(6)

data2 = new_socket.recv(10)  # The old data will be received the second time, and then new data will be received if there is room. The first time the data is not received completely, and the remaining data is received completely,
print('Second packet', data2.decode('utf-8'))

# todo 6. Close sub socket
new_socket.close()
# todo 7. Close the entire server socket
server_socket.close()

client:

from socket import *
import time  # The time module ensures a long interval when the client sends multiple packets

# todo 1. Create client socket and SOCK_STREAM: indicates the TCP protocol
client_socket = socket(AF_INET, SOCK_STREAM)
# todo 2. Connect to the target server
client_socket.connect(('192.168.1.112', 9999))
# todo 3. Send data
client_socket.send('mashibing'.encode('utf-8'))

time.sleep(5)  # Let the current thread sleep for 5 seconds
# todo sends data for the second time
client_socket.send('laoxiao'.encode('utf-8'))
# todo 4. Close the client socket
client_socket.close()

After starting the server and client
The running result is

Connection succeeded ('192.168.1.112', 52352)
First packet mas
 Second packet hibinglaox

4, Cause of sticking:

The so-called sticking problem is mainly due to:
1. The receiver does not know the boundaries between messages and how many bytes of data a message needs to extract. (sticky packets appear on the server side)
2. When tcp sends data with little data and short interval, it will send several and together. (sticky packets appear on the client)

5, Solution of sticking package

At present, the more reasonable processing method is to add a header to the byte stream, tell the total size of the transmitted byte stream, and then the receiving end receives all data in an endless loop. The serialized data length is packed into 4 bytes with struck (4 bytes are enough).
The struct module can be used to convert Python values into C language structure (byte type) according to format symbols, which is convenient for data stream transmission.
Case: the client sends a file to the server (based on TCP protocol), and the packet sticking problem should be solved at the same time.
Server side

from socket import *
import struct  # pack
import os

server = socket(AF_INET, SOCK_STREAM)
server.bind(('', 8088))
server.listen(5)

new_socket, addr = server.accept()

f = open(r'D:\The server.txt', 'wb')

#todo receives the header sent by the client
header_data = new_socket.recv(4)
# todo size indicates the length of the packet
size = struct.unpack('!i', header_data)[0]  # unpack returns a tuple, and the first value of the tuple is the length

recv_size = 0  # How long data has been received
while recv_size < size:
    data = new_socket.recv(1024)
    recv_size += len(data)  # The length of bytes received shall be accumulated
    f.write(data)
print('Server side reception completed')

f.close()
new_socket.close()
server.close()

client:

from socket import *
import struct  # pack
import os

client_socket = socket(AF_INET, SOCK_STREAM)
client_socket.connect(('192.168.1.112', 8088))

# Client transfers files to server new.mp4
file_path = 'new.txt'
f = open(file_path, 'rb')

# todo prepares a header before sending real file data
size = os.path.getsize(file_path)  # Byte length of the file
# todo creates a header with i as a 4-byte int.
header = struct.pack('!i', size)  # The receiver will unpack with struct to get a number of type int
# todo send packet header
client_socket.send(header)

# todo send file content
while True:
    data = f.read(1024)  # 1024 bytes read at a time
    if not data:
        break
    client_socket.send(data)  # File content sent to server

print('Client upload file completed')
f.close()
client_socket.close()

Execution results:


Summary: the client encapsulates the data length into a fixed size data. At this time, the server can specify to read the fixed size content without reading the data content. The server just needs to receive the data content according to the data length. Therefore, the client sends data (files) twice in a row without sticking the package, Because the server only receives the data received this time every time.

Tags: Python udp TCP/IP

Posted on Sat, 16 Oct 2021 18:21:06 -0400 by su1d