Python Advanced Software Technology

1. Select Sort

1.1 algorithm steps

1. First find the smallest (large) element in the unsorted sequence and store it at the beginning of the sort sequence
2. Continue looking for the smallest (large) element from the remaining unsorted elements and place it at the end of the sorted sequence
3. Repeat the second step until all elements are sorted

1.2 Code implementation

def selectionSort(arr): for i in range(len(arr) - 1): # Index of record minimum minIndex = i for j in range(i + 1, len(arr)): if arr[j] < arr[minIndex]: minIndex = j # If i is not the minimum, swap i with the minimum if i != minIndex: arr[i], arr[minIndex] = arr[minIndex], arr[i] return arr

2. Quick Sort

2.1 Algorithmic steps

1. Pick out an element from a number of columns, called a pivot;
2. Reorder the number columns, all elements smaller than the base value are placed in front of the base, and all elements larger than the base value are placed behind the base (the same number can be on either side). After the partition exits, the benchmark is in the middle of the column. This is called a partition operation;
3. Recursively sorts subordinate columns of elements less than the base value and those greater than the base value

2.2 Code implementation

def quickSort(arr, left=None, right=None): left = 0 if not isinstance(left,(int, float)) else left right = len(arr)-1 if not isinstance(right,(int, float)) else right if left < right: partitionIndex = partition(arr, left, right) quickSort(arr, left, partitionIndex-1) quickSort(arr, partitionIndex+1, right) return arr def partition(arr, left, right): pivot = left index = pivot+1 i = index while i <= right: if arr[i] < arr[pivot]: swap(arr, i, index) index+=1 i+=1 swap(arr,pivot,index-1) return index-1 def swap(arr, i, j): arr[i], arr[j] = arr[j], arr[i]

# Complete a round of exchange def sub_sort(list_, low, high): temp = list_[low] # Selected Baseline while low < high: while list_[high] >= temp and high > low: high -= 1 list_[low] = list_[high] while list_[low] < temp and low < high: low += 1 list_[high] = list_[low] list_[low] = temp return low def quicksort(list_, low, high): """ Quick Sort :param list_: List to Sort :param low: Index of the first element of the list :param high:Last element index :return:A sorted list """ if low < high: key = sub_sort(list_, low, high) quicksort(list_, low, key - 1) quicksort(list_, key + 1, high) return list_

3. Insert Sort

3.1 Algorithmic steps

1. Consider the first element of the first sequence to be sorted as an ordered sequence, and the second to the last element as an unsorted sequence.
2. Unordered sequences are scanned from beginning to end, and each element scanned is inserted into the appropriate position in the ordered sequence. (If the element to be inserted is equal to an element in an ordered sequence, the element to be inserted is inserted after the equal element)

3.2 Code implementation

def insertionSort(arr): for i in range(len(arr)): preIndex = i-1 current = arr[i] while preIndex >= 0 and arr[preIndex] > current: arr[preIndex+1] = arr[preIndex] preIndex-=1 arr[preIndex+1] = current return arr

1. Byte Strings and Strings

1. Common ASCII encoding strings can be converted to byte strings by preceding b, for example, b'hello'
2. String to byte string method: str.encode()
3. Byte string to string method: bytes.decode()

2. File operations

2.1 Read Files

• read([size])
• readline([size])
• readlines([sizeint])

If the file object is read-fetched, it is an iterative object that iterates over each line of the file in a for loop

f = open('test', 'r') while True: # Returns an empty string when reading to the end of the file data = f.read(100) # Read up to 100 characters at a time if not data: break print(data)

2.2 with Operation

The with statement in python is used when accessing resources. It guarantees that no matter whether errors or exceptions occur during processing, a specified "clean up" operation will be performed to release the accessed resources, such as automatic closure of files after reading and writing, automatic acquisition and release of locks in threads, etc.

The syntax format of the with statement is as follows

with context_expression [as target(s)]: with-body

_With the with method, close() is not required, because the object generated by with is automatically processed after the statement block ends, so close is not necessary, but this file object can only be used within the with statement block

with open('file', 'r+') as f: r.read()

3. File offset

3.1 Definition

_When a file is opened for operation, a record is automatically generated that describes a series of our operations on the file. This includes the location of the file to which each operation takes place. Read and write operations start from this location.

3.2 Basic Operations

Function: Get file offset size

_Function: Move file offset location
_parameter: offset represents the number of bytes moved relative to a location. Negative numbers mean moving forward, positive numbers mean moving backward. When is the base location, the default value is 0, which means counting from the beginning of the file, 1 from the current location, and 2 from the end of the file.
_Note: The base position can only be 1 or 2 when the file must be opened binary

4. File descriptors

4.1 Definition

Each IO operation in the system is assigned an integer as the number, which is the file descriptor of the IO operation.

4.2 Get the file descriptor

fileno():
_Get the corresponding file descriptor from the IO object

4.3 File Management Functions

1. Get file size
   os.path.getsize(file)
2. View File List
   os.listdir(dir)
3. See if the file exists
   os.path.exists(file)
4. Determine File Type
   os.path.isfile(file)
5. Delete Files
   os.remove(file)

5. Network Model

5.1 OSI Seven Layer Model

Formulate organization
_ISO (International Organization for Standardization)
Effect
_Standardize network communication workflow

Application layer: Provide user services, specific functions implemented by applications
Representation layer: data compression optimizes encryption
Session Layer: Establish user-level connections and select appropriate transport services
Transport Layer: Provide transport services
Network layer: routing, network connectivity
Link Layer: Conduct data exchange to control the sending of specific data
Physical Layer: Provides hardware assurance for data transmission, network card interface, transmission media

Advantage:

1. A unified workflow has been established
2. Clear division, clear division of duties, clear division of work in each step
3. Reduce coupling between modules for easy development

5.2 TCP/IP Four-Layer Model

• application layer
• transport layer
• Internet Layer
• network interface

Network Protocol

In the network data transmission, rules are followed, including what kind of data structure to establish, what kind of special marks, and so on.

5.3 Basic Network Concepts

IP Address

• Function: Determine a host's network routing location
• View local network address command: ifconfig

domain name

• Definition: Name the address of the network server
• Function: Easy to remember, express certain meaning
• ping[ip]: test whether a host is connected

Port number

• Role: Ports are part of a network address used to distinguish between different network applications on a host
• Features: Application listening ports in a system cannot be duplicated
• Value range: 1 - 65535
• 1-1023 System Application or Popular Program Listening Port
• 1024-65535 self-use port

5.4 Transport Layer Service

Transmission characteristics:

• Provides reliable data transmission, reliability index data transmission process without loss, disorder, error, duplication

Means of implementation:

• Data connection needs to be established before communication ends and disconnects normally

Three handshakes (make connection)

1. Client sends message message message to server requesting connection
2. After the server receives the request, the reply confirms that it can connect
3. Client receives reply, sends final message connection establishment

Four wave (disconnect)

1. The active party sends a message requesting disconnection
2. When the passive party receives the message, it immediately replies, indicating that it is ready to disconnect
3. The passive party is ready to send the message again to indicate that it can be disconnected
4. The active party receives the confirmation, sends the final message and finishes disconnecting

Transmission characteristics:

• The reliability of transmission is not guaranteed, the transmission process is not connected or disconnected, and the data is sent and received freely.

Where applicable:

• The network is poor and the reliability of transmission is not high. Examples: web video, group chat, radio

5.5 socket socket programming

Socket:

• A Technical Mean to Implement Network Programming for Data Transfer

Python implements socket programming:

• import socket
• from socket import socket

Socket Classification

1. Streaming socket (SOCK_STREAM): Transmit data in byte stream for tcp network transmission
2. Datagram socket: Transmit data as datagram to implement udp network transmission scheme.

Service-side process:
socket–>bind–>listen–>accept–>send/recv–>close
1. Create sockets

sockfd = socket.socket(socket_family = AF_INET, socket_type = SOCK_STREAM, proto=0) # Function: Create socket # Parameter socket_family Network Address Type AF_INET stands for ipv4 # Parameter socket_type socket type SOCK_ STREAM (Streaming) # proto Usually 0 Select Subprotocol

2. Binding Address

• Local address: localhost 127.0.0.1
• Network address: 172.40.91.185
• Automatic Address Acquisition: 0.0.0.0

sockfd.bind(addr) # Function: Bind local network address # Parameter: Binary tuple (ip, port) ('0.0.0.0', 8888)

3. Set up monitoring

sockfd.listen(n) # Function: Set the socket as a listening socket, determine the size of the listening queue # Parameter: listening queue size

4. Waiting to process client connection requests

connfd, addr = sockfd.accept() # Function: Block waiting to process client requests # Return value: connfd Client Connection Socket # addr Connected Client Address

5. Message Receiving and Sending

data = connfd.recv(buffersize) #Function: Receive client messages #Parameter: Size of the maximum message received at a time #Return value: Received content n = connfd.send(data) #Function: Send message #Parameter: What to send bytes format #Return value: number of bytes sent

6. Close the socket

sockfd.close() # Function: close socket

Code Samples

import socket # Create Socket Object import unicodedata sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Binding Address sockfd.bind(('127.0.0.1', 8888)) # Set Listening Socket sockfd.listen(5) while True: # Block waiting for client connection print("Waiting for connect") try: connfd, addr = sockfd.accept() print("Connect from", addr) except KeyboardInterrupt: print("Server Exit") break except Exception as e: print(e) continue # Send and receive messages while True: data = connfd.recv(1024) if not data: # Empty data indicates client exit break print("Received:", data.decode()) n = connfd.send(b'Thanks') print("Send out%d byte" % n) # Close connfd.close() sockfd.close()

Client Process
_socket ->bind < optional > (generally not written) - connect ->send/recv ->close

1. Create sockets

• Note: Only sockets of the same type can communicate

2. Connect to the server

sockfd.connect((server_addr,port)) # Function: Connect to server # Parameter: Tuple server address

3. Send and receive messages

• Note: to prevent blocking at both ends, recv send should work together

4. Close the socket

Code Samples

""" tcp_client.py tcp Client Process """ from socket import * # Create tcp socket sockfd = socket() # Use default parameter-->tcp socket # Connect to Server server_addr = ('127.0.0.1', 8888) sockfd.connect(server_addr) # Send and receive messages while True: data = input("Msg>>") if not data: break sockfd.send(data.encode()) data = sockfd.recv(1024) print("Server:", data.decode()) # Print Received Content # Close sockfd.close()

_tcp connection when one end exits and the other is blocked in recv, recv immediately returns an empty string
BrokenPipeError in tcp connection if one end no longer exists and still attempts to send via send
A listening socket can connect multiple clients at the same time, or it can be connected repeatedly

1. Network buffers effectively coordinate the sending and receiving speed of messages
2. Send and recv actually send receive messages to the buffer, and recv will not block if the buffer is not empty

Reason:

• tcp is transmitted as a byte stream without message boundaries. Messages sent multiple times are received once, and a sticky packet is formed

Influence:

• If each sent content has a separate meaning, the glue will have an impact when the receiver needs to resolve it independently

Processing method:

1. Artificial Add Message Boundary
2. Control Send Speed

Service-side process
socket–>bind–>recvfrom–>sendto–>close

1. Create a datagram socket

sockfd = sock(AF_INET, SOCK_DGRAM)

2. Binding Address

sockfd.bind(addr)

3. Message Receiving and Receiving

data, addr = sockfd.recvfrom(buffersize) # Function: Receive UDP messages # Parameter: Maximum number of bytes received at a time # Return value:data Received content; addr Message sender address n = sockfd.sendto(data, addr) #Function: Send UDP message #Parameters:data Sent content (bytes format); addr destination address #Return value: number of bytes sent

4. Close the socket

sockfd.close()

Service-side code example

""" udp_server.py udp Socket server process """ import socket # Create UDP socket sockfd = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Binding Address server_addr = ('127.0.0.1', 8888) sockfd.bind(server_addr) # Receive and receive messages in a loop while True: data, addr = sockfd.recvfrom(1024) print("Received message:", data.decode()) sockfd.sendto(b'Thanks', addr) # Close socket.close()

udp client code example

from socket import * # server address ADDR = ('127.0.0.1', 8888) # Create Socket sockfd = socket(AF_INET, SOCK_DGRAM) # Receive and receive messages in a loop while True: data = input("Msg>>") if not data: break # Send a message to the server sockfd.sendto(data.encode(), ADDR) # receive messages msg, addr = sockfd.recvfrom(1024) # Print received messages print("From server:", msg.decode()) sockfd.close()

1. Streaming sockets transfer data as byte streams, and datagram sockets transfer data as datagrams
2. tcp sockets will stick, udp sockets will not stick if they have message boundaries
3. tcp sockets guarantee message integrity, udp sockets do not
4. tcp sockets rely on listen accept to establish a connection to send and receive messages, udp sockets do not need
5. tcp socket uses send, recv to send and receive messages, udp socket uses sendto, recvfrom

1. sockfd.type socket type
2. sockfd.family socket address type
3. sockfd.getsockname() Gets the socket binding address
4. sockfd.fileno() Gets the file descriptor of the socket
5. sockfd.getpeername() Gets the connection socket client address
6. sockfd.setsockopt(level,option,value)
   _Function: Setting socket options
   _Parameter: level Option Category SOL_SOCKET
   Selection Specific Options
   Value option value
7. sockfd.getsockopt(level, option)
   _Function: Get socket option value

Code Samples

""" Introduction to socket properties """ from socket import * # Create Socket s = socket() # Set the port for immediate reuse s.setsockopt(SOL_SOCKET, SO_REUSEADDR, True) s.listen(3) c, addr = s.accept() print("Connection End Address:",c.getpeername()) print("Address type:",s.family) print("Socket type:",s.type) print("Binding Address:",s.getsockname()) print("File Descriptor:",s.fileno())

1. Broadcast Definition: One End Sends Multipoint Receive
2. Broadcast Address: The maximum address of each network is the address where the broadcast is sent, and when sent to that address, all hosts in the network segment can receive it.

Code Samples

""" Broadcast reception broadcast_recv.py 1.Establish udp socket 2.Set up a socket to send and receive broadcasts ( setsockopt) 3.Select the port to receive 4.Receive radio """ from socket import * s = socket(AF_INET, SOCK_DGRAM) # Set up socket to receive broadcasts s.setsockopt(SOL_SOCKET, SO_BROADCAST, True) # Set up a socket to receive broadcasts s.bind(('0.0.0.0', 9999)) msg, addr = s.recvfrom(1024) print(msg.decode())

""" Broadcast Send broadcast_send.py 1.Establish udp socket 2.Set up to send broadcasts 3.Loop to broadcast address """ from socket import * from time import sleep # Address of broadcast address ifconfig dest = ('192.168.0.161', 9999) # Create Datagram Socket s = socket(AF_INET, SOCK_DGRAM) # Set up to send broadcasts s.setsockopt(SOL_SOCKET, SO_BROADCAST, 1) data = """ ****************** Beijing 11.17 Early winter Temperature: 7 Status: No four five chicks ****************** """ sleep(2) # Send every two seconds s.sendto(data.encode(), dest) # The destination address is the broadcast address

1. Purpose: Web page acquisition, data transfer
2. Characteristic:
   _Application Layer Protocol, Transport Layer uses tcp transport
   _Simple, flexible, many languages have HTTP-specific interfaces
   _Stateless, protocol does not record transmission content
   _http1.1 supports persistent connections, enriching request types

• Request line: specific request category and content

GET / HTTP/1.1
Request Category_Request Content_Protocol Version

• Request Header: A further explanation and description of the request in the form of one key-value pair per line

Accept-Encoding: gzip

• Blank Line
• Requestor: Request parameters or submissions

• Response line: Feedback basic response

HTTP/1.1 200 OK
Version Information_Response Code_Additional Information

• Response Header: Description of response content

Content-Type: text/html

• Blank Line
• Response body: The body content information of the response

Code examples for http requests and responses

""" httpserver v1.0 Basic requirements: Get requests from browsers Determine if the content of the request is/，take index.html Response to Client Response 404 if something else is returned """ from socket import * # Client (Browser) Processing def request(connfd): # Get the request and extract the content of the request data = connfd.recv(4096) # Receive requests if not data: # Prevent browser from exiting abnormally return # print(data.decode(), type(data.decode())) request_line = data.decode().split('\n')[0] # Get Request Line # print(request_line, type(request_line)) # Print Request Line info = request_line.split(" ")[1] # Get Request Content # print(info) # Print Request Content # Judgment Yes/Returns index.html, not/Returns 404 if info == "/": # Yes/then return index.html with open('index.html') as f: # Open index.html file with with response = "HTTP/1.1 200 OK\r\n" # Response line response += "Content-Type:text/html\r\n" # Response Header (Writable) response += "\r\n" # Blank Line response += f.read() # Response Body else: # No/Return 404 response = "HTTP/1.1 404 Not Found\r\n" # Response line response += "Content-Type:text/html\r\n" # Response Header (Writable) response += "\r\n" # Blank Line response += "<h1>Sorry...</h1>" # Response Body # Send response content to browser connfd.send(response.encode()) # Set up tcp network sockfd = socket() # Set the port for immediate use sockfd.setsockopt(SOL_SOCKET, SO_REUSEADDR, True) # Binding Address sockfd.bind(('0.0.0.0', 8000)) # Set socket to listen on socket sockfd.listen(3) while True: connfd, addr = sockfd.accept() request(connfd) # Processing client requests

1. Principle: Package a set of simple data and send it in bytes format. Or parse a set of bytes formatted data
2. Interface use

import struct st = struct.Struct(fmt) # Function: Generate structured objects # Parameter: fmt Customized data structure st.pack(v1, v2, v3...) # Function: Converts a set of data packaged in a specified format to bytes # Parameter: Data to be packaged # Return value: bytes byte string st.unpack(bytes_data) # Function: parse bytes string in specified format # Parameter: Byte string to parse # Return value: parsed content struct.pack(fmt, v1, v2, v3,...) struct.unpack(fmt, bytes_data) # Note: You can use the struct module to call pack unpack directly. In this case, the first parameter of the two functions is passed into fmt. Other usage functions are the same # Note: What format (fmt) is packaged when packaging and what format is parsed when parsing

Demonstrate in terminal:

In [1]: import struct In [2]: st = struct.Struct('i4sf') In [3]: data = st.pack(1, b'Lily', 1.65) In [4]: data Out[4]: b'\x01\x00\x00\x00Lily33\xd3?' In [5]: st.unpack(data) Out[5]: (1, b'Lily', 1.649999976158142)

Exercises for struct module

Complete with udp

• Enroll student information on the client side and send it to the server
• Write student information into a file on the server side, one line for each student
• Information format: id(int) name(str) age(int) score(float)

""" struct_send.py """ from socket import * import struct # Define the data format first st = struct.Struct('i32sif') # udp socket s = socket(AF_INET, SOCK_DGRAM) ADDR = ('127.0.0.1', 8888) while True: print("=====================") id = int(input("ID:")) name = input("Name:").encode() age = int(input("Age:")) score = float(input("Score:")) # Data Packaging Send data = st.pack(id, name, age, score) s.sendto(data, ADDR)

""" struct_recv.py """ from socket import * import struct # Define the same data format on both sides st = struct.Struct('i32sf') # Create udp socket s = socket(AF_INET, SOCK_DGRAM) # binding s.bind(('127.0.0.1', 8888)) # Open File f = open('student.txt', 'a') while True: data, addr = s.recvfrom(1024) data = st.unpack(data) # write file info = "%d %-10s %d %.1f\n" % data f.write(info) f.flush()

6. Network Concurrency

6.1 Multitask Programming

1. Significance: Make full use of computer multi-core resources to improve program efficiency
2. Implementation scenario: multi-process, multi-threaded
3. Parallel and Concurrent
   _Concurrent: Processing multiple tasks at the same time, the kernel continuously switches between tasks to achieve the effect of as if multiple tasks were executed at the same time, in fact, only one task occupies the kernel at a time
   _Parallel: Multiple tasks are executed simultaneously using computer multi-core resources, where the relationship between multiple tasks is parallel

6.2 process

1. Definition: A program is run once on a computer
_Program is an executable file, it is a static occupying disk
_Process is a dynamic description of the process, occupies computer running resources, has a certain life cycle

2. How a process is generated in the system
[1] User space initiates requests by calling program interfaces or commands
[2] The operating system accepts user requests and starts creating processes
[3] The operating system allocates computer resources, determines process status, etc.
[4] The operating system provides the processes created for the user to use

3. Basic concepts of processes

_cpu time slice: If a process occupies the CPU core, it is said that the process is on the CPU time slice

_PCB (Process Control Block): A space in memory that stores basic information about a process and is also used for system lookups to identify processes

_Process ID(PID): The system assigns an integer greater than 0 to each process as the process ID. Do not duplicate each process ID_Linux view process ID:ps-aux

Parent-Child Processes: Each process in the system (except the system initialization process) has a unique parent process and can have zero or more child processes. Parent-child process relationships are easy to manage. View the process tree: pstree

_Process state (three states):
_Ready state: Process is ready to execute, waiting to allocate CPU resources
Runtime: Processes holding CPU time slices are running
Waiting State: Process is temporarily stopped and CPU is relinquished

_five states (new and terminated on the basis of three states):
New: Create a process, get resources
Termination: The process that ends, the process that releases resources

Status View Command: ps-aux -->STAT Column
Waiting S tate
_R_Runtime
Waiting State
Waiting State
Z ombies

<has higher priority
Low priority
Foreground process
Session Group Leader
Multithreaded

2. Running characteristics of processes
[1] Processes can use computer multi-core resources
[2] Process is the smallest unit of computer resources allocated
[3] The processes run independently of each other.
[4] Each process has its own space and uses its own space resources

6.3 fork-based multi-process programming

"""fork Create Process Demo""" import os pid = os.fork() # Function: Create a new process # Return value: Integer, returns a negative number if the creation process fails, returns the PID of the new process in the original process if successful, and returns 0 in the new process if pid < 0: print("Create process failed") #Part of subprocess execution elif pid == 0: print("The new progress") # Part of parent process execution else: print("The old process") # Both parent and child processes execute print("Fork test over")

Be careful:

• The child process copies all the memory space of the parent process, starting with the next sentence of fork
• Parent-child processes run independently, in varying order
• Using the difference of parent-child process fork return value, it is almost fixed to match if structure to allow parent-child process to perform different content
• Parent-child processes have their own unique features such as PID, PCB, command set, etc.
• Spatial and child processes opened up before the parent process fork also have the same ownership, and parent-child processes do not influence each other's operations in their own space

6.4 Process-related functions

os.getpid() # Function: Get the PID value of a process # Return value: Returns the PID of the current process os.getppid() # Function: Get the PID of the parent process # Return value: Returns the PID of the parent process os._exit(status) # Function: End a process # Parameter: Termination state of the process sys.exit([status]) # Function: Exit process # Parameter: Integer denotes exit status # If the parameter passes a string: indicating what to print on exit

Code Samples

import os, sys pid = os.fork() if pid < 0: print("Error") elif pid == 0: print("Child PID:", os.getpid()) # SubPID print("Parent PID:", os.getppid()) # Parent PID else: print("Get Child PID:", pid) # SubPID print("Parent PID:", os.getpid()) # Parent PID os._exit(0) # Exit the process, the following statements will not be executed sys.exit("Sign out") # Exit the process and print the passed string print("Exit") # Will not execute

6.5 Orphans and Zombies

1. Orphan process: The parent process exits before the child process, when the child process becomes an orphan process
Features: The orphan process will be adopted by the system process, at which time the system process will become the new parent of the orphan process, the orphan process will be automatically processed to exit the process

2. Zombie process: When a child process exits before the parent process, and the parent process does not handle the exit state of the child process, the child process is called a zombie process.
Features: Although the zombie process ends, it will retain part of the PCB in memory, a large number of zombie processes will waste the system's memory resources

3. How to Avoid Zombie Processes

1. Use wait function to handle child process exit

pid, status = os.wait() # Function: Block in parent process waiting for processing child process to exit # Return value: pid of the child process that pid exits; Status Subprocess Exit Status

Code Samples

import os, sys pid = os.fork() if pid < 0: print("Error") elif pid == 0: print("Child PID:", os.getpid()) sys.exit("Subprocess Exit") else: """ os.wait() Dealing with zombie processes """ pid, status = os.wait() print("pid:", pid) print("status:", status) # The printed status is the exit status of the child process multiplied by 256 while True: # Let the parent process not exit pass

1. Create a secondary subprocess to handle the zombies
   _Parent process creates child process, waiting to recycle child process
   _Subprocess Creates a secondary subprocess and exits
   _Second-level child processes are called orphans and execute events with the parent of first-level child processes

2. Exit by signal processing subprocess
   Principle: When a child process exits, it sends a signal to the parent process. If the parent process ignores the child process signal, the system automatically processes the child process exit
   _Method: Use the signal module to write the following statement before a parent process creates a child process

import signal
signal.signal(signal.SIGCHLD,signal.SIG_IGN)

Sample code

"""Signal Processing Zombies""" import os, sys import signal # When a child process exits, the parent process ignores the exit behavior and the child process is handled by the system signal.signal((signal.SIGCHLD, signal.SIG_IGN)) pid = os.fork() if pid < 0: print("Error") elif pid == 0: print("Child PID:", os.getpid()) sys.exit(2) else: while True: # Parent process does not exit pass

6.6 Group Chat Room

Requirements:

1. When someone enters a chat room, they need to enter their name. The name cannot be repeated.
2. When someone enters the chat room, others will be notified that XXX enters the chat room
3. One person sends a message, others receive: XXX:xxxxxxxxxxx
4. When someone exits the chat room, others will be notified that XXX exits the chat room
5. Extension: Server can send announcements to all users: Administrator Message: XXXX

code implementation

Client Code

"""chat_client.py""" import os import sys from socket import * # server address ADDR = ('127.0.0.1', 8888) # send message def send_msg(s, name): while True: try: text = input("Speeches:") except KeyboardInterrupt: text = 'quit' if text.strip() == 'quit': msg = 'Q' + ' ' + name s.sendto(msg.encode(), ADDR) sys.exit("Exit Chat Room") msg = 'C %s %s' % (name, text) s.sendto(msg.encode(), ADDR) # receive messages def recv_msg(s): while True: try: data, addr = s.recvfrom(4096) except KeyboardInterrupt: sys.exit() if data.decode() == 'EXIT': sys.exit() print(data.decode() + '\n Speeches:', end='') # Client startup function def main(): s = socket(AF_INET, SOCK_DGRAM) # Enter the chat room while True: name = input("Please enter your name:") msg = 'L' + ' ' + name s.sendto(msg.encode(), ADDR) # Receive feedback from the server data, addr = s.recvfrom(128) if data.decode() == 'OK': print("You have entered the chat room") break else: print(data.decode()) # Exit the loop to indicate that you have entered the chat room pid = os.fork() if pid < 0: sys.exit("Error!") elif pid == 0: send_msg(s, name) # Subprocess sends message else: recv_msg(s) # Parent process receives messages main()

Service-side code

"""chat_server""" from socket import * import os, sys # server address ADDR = ('0.0.0.0', 8888) # Store user information user = {} # Sign in def do_login(s, name, addr): if name in user: s.sendto("The user already exists".encode(), addr) return s.sendto(b'OK', addr) # Notify Others msg = "\n Welcome%s Enter the chat room" % name for i in user: s.sendto(msg.encode(), user[i]) user[name] = addr # chat def do_chat(s, name, text): msg = "\n%s: %s" % (name, text) for i in user: if i != name: s.sendto(msg.encode(), user[i]) # Sign out def do_quit(s, name): msg = "\n%s Exit Chat Room" % name for i in user: if i != name: s.sendto(msg.encode(), user[i]) else: s.sendto(b'EXIT', user[i]) del user[name] # Processing Request def do_request(s): while True: data, addr = s.recvfrom(1024) temp = data.decode().split(' ') if temp[0] == 'L': do_login(s, temp[1], addr) elif temp[0] == 'C': text = ' '.join(temp[2:]) do_chat(s, temp[1], text) elif temp[0] == 'Q': do_quit(s, temp[1]) # Set up a network def main(): # udp server s = socket(AF_INET, SOCK_DGRAM) s.setsockopt(SOL_SOCKET, SO_REUSEADDR, True) s.bind(ADDR) pid = os.fork() if pid == 0: while True: msg = input("Administrator message:") msg = "C Administrators " + msg s.sendto(msg.encode(), ADDR) # Request Handling Function do_request(s) main()

6.7 multiprocessing module creation process

1. Encapsulate events that require subprocesses to execute as functions
2. Create process objects, associated functions, from the module's Process class
3. Process information and properties can be set through process objects
4. start a process by calling it from a process object
5. Call join recycling process through process object

Be careful:

• Start the process and the target binding function starts executing, which executes as a child process, when the process is actually created
  
  Be careful:

1. Creating a process with multiprocessing is also a child process that copies the parent process space code snippet, and parent-child processes run independently of each other
2. The child process only runs the function part of the target binding, the rest of which is the parent process execution
3. A parent process in multiprocessing is often used only to create a child process to recycle a child process, and the specific event is completed by the child process
4. Standard input cannot be used in a subprocess created by multiprocessing (input cannot be used in a subprocess)

""" multiprocessing Module Creation Process 1.Writing process functions 2.Generate process objects 3.Start process 4.Recycling process """ import multiprocessing as mp from time import sleep # Create process functions def fun(): print("Start a process") sleep(5) print("End of child process") # Create Process Object p = mp.Process(target=fun) # p This object can represent a process p.start() # Start process # Parent Process Event sleep(3) print("Part of parent process execution is written in start and join between") p.join() # Recycling process

""" multiprocessing Create multiple processes """ from multiprocessing import Process from time import sleep import os def th1(): sleep(3) print("Having dinner") print(os.getppid(), '--', os.getpid()) def th2(): sleep(2) print("Sleep") print(os.getppid(), '--', os.getpid()) def th3(): sleep(4) print("Bean Bean") print(os.getppid(), '--', os.getpid()) things = [th1, th2, th3] jobs = [] for th in things: p = Process(target=th) jobs.append(p) # Save process objects from a list p.start() # Recycle all subprocesses together for i in jobs: i.join()

Result of running the code above

""" Process Pass-through to process function """ from multiprocessing import Process from time import sleep # Process functions with parameters def worker(sec, name): for i in range(3): sleep(sec) print("I'm %s" % name) print("I'm working...") # p = Process(target=worker, args=(2, 'Baron')) # Pass by location: 2 to sec, Baron to name # p = Process(target=worker, kwargs={'sec': 2, 'name': 'Baron'}) # Pass by keyword p = Process(target=worker, args=(2,), kwargs={'name': 'Baron'}) p.start() p.join()

• p.name process name
• The PID number of the p.pid subprocess
• p.is_alive() to see if a child process is in the lifecycle
• p.daemon sets parent-child process exit relationships
  _If set to True, the child process ends with the parent process exiting
  _Requires that must be set before start
  _If daemon is set to True, join() will not normally be used

""" Process Object Properties """ import time from multiprocessing import Process def tm(): for i in range(3): time.sleep(2) print(time.ctime()) p = Process(target=tm, name="pname") p.daemon = True # Child process exits when parent process exits p.start() print("Process name:", p.name) # Default name Process-1, assigned pname print("process ID: ", p.pid) # Get the PID of the corresponding subprocess print("is alive: ", p.is_alive()) # Check to see if the process is in the lifecycle, True is not in the lifecycle False

6.8 Process Pool

_The process of creation and destruction consumes more resources, and when there are many tasks and each task is completed in a very short time, frequent creation and destruction processes are required. At this time, the pressure on the computer is high, and the process pool technology solves the above problems well.

_Create a number of processes to handle events. When the process is finished, it does not exit but continues to process other events until all events have been processed and destroyed uniformly. Increase process reuse and reduce resource consumption

1. Create process pool objects and place appropriate processes

from multiprocessing import Pool Pool(processes) # Function: Create process pool object # Parameters: Specify the number of processes, automatically determined by default based on the system

1. Queue events to process pool for execution

pool.apply_async(func,args,kwargs) # Function: Use process pool to execute func events # Parameter: func event function # Parameter: args tuple passes positional arguments to func # Parameters: The kwargs dictionary passes keywords to func # Return value: Return function event object

1. Close process pool

pool.close() # Function: Close process pool

1. Recycle processes in process pool

pool.join() # Function: Recycle processes in the process pool

""" Process pool usage example """ from multiprocessing import Pool from time import ctime, sleep # Process pool events def worker(msg): sleep(2) print(ctime(), '--', msg) # Create process pool pool = Pool(4) # If no parameters are written, the number of processes in the process pool is determined by the system by default # Add event to process pool queue for i in range(10): msg = "Tedu %d" % i pool.apply_async(func=worker, args=(msg,)) # Close process pool pool.close() # Recycle process pool pool.join()

Run Results

6.9 Interprocess Communication (IPC)

1. Necessity:
_Inter-process space is independent, resources are not shared, at this time, when data transfer between processes is required, specific means of data communication are needed.
2. Common methods of interprocess communication:
Pipeline_Message Queue_Shared Memory_Signal_Semaphore_Socket

1. Communication principle
   _Open up pipeline space in memory, generate pipeline operation objects, multiple processes can use the same pipeline object to read and write to achieve communication
2. Implementation Method

from multiprocessing import Pipe fd1,fd2 = Pipe(duplex=True) # Function: Create Pipeline # Parameter: Default for two-way pipe, if False for one-way pipe # Return value: A read-write object representing both ends of the pipe #Readable and Writable in Bidirectional Representation #If it is a one-way pipe fd1 read-only fd2 write-only fd.recv() # Function: Get content from pipeline fd.send(data) # Function: Write content to pipe # Parameter: Data to be written

1. Code Samples

""" pipe.py pipeline communication Be careful: 1.multiprocessing Medium pipeline communication can only be used in relational processes 2.Pipeline objects are created in the parent process, and child processes are acquired through the parent process """ from multiprocessing import Process, Pipe # Create Pipeline fd1, fd2 = Pipe() def app1(): print("start-up app1,Please login") print("request app2 To grant authorization") fd1.send("app1 Request login") # Write Pipeline data = fd1.recv() if data: print("Logon success:", data) def app2(): data = fd2.recv() # Block waiting to read pipe contents print(data) fd2.send(('Dave', '123')) # You can send any type of Python type data, where tuples are sent p1 = Process(target=app1) p2 = Process(target=app2) p1.start() p2.start() p1.join() p2.join()

Run Screenshot

1. Communication principle
   _Build a queue model in memory where processes either store messages in or take them out to complete interprocess communication
2. Implementation Method

from multiprocessing import Queue q = Queue(maxsize=0) # Function: Create Queue Objects # Parameter: Maximum number of messages to store # Return value: Queue object q.put(data,[block,timeout]) # Function: Save messages to queue # Parameter: whether the content block settings of the data to be stored are blocked (False is non-blocked) timeout detection q.get([block,timeout]) # Function: Remove messages from queue # Parameter: block setting whether blocking (False is non-blocking) timeout detection # Return value: Return what you get q.full() #Determine if the queue is full q.empty() #Determine if the queue is empty q.qsize() #Determine the number of messages in the queue q.close() #Close Queue

1. Code Samples

""" queue_0.py Message Queue Demo Be careful: Message queue complies with FIFO principle """ from multiprocessing import Queue, Process from time import sleep from random import randint # Create Message Queue q = Queue(5) # Store up to five messages def handle(): for i in range(6): x = randint(1, 33) q.put(x) # Message Entry q.put(randint(1, 16)) def request(): list_ = [] for i in range(6): list_.append(q.get()) list_.sort() list_.append(q.get()) print(list_) p1 = Process(target=handle) p2 = Process(target=request) p1.start() p2.start() p1.join() p2.join()

1. Communication principle
   _Opens up a space in memory where processes can write and read content to complete communication, but each write will overwrite the previous content
2. Implementation Method
   

from multiprocessing import Value, Array obj = Value(ctype, data) # Function: Open up shared memory # Parameter: ctype denotes shared memory space type'i''f''c' # Data shared memory space initial data # Return value: Shared memory object obj.value # Viewing modifications to this property reads and writes to shared memory obj.Array(ctype, data) # Function: Open up shared memory space # Parameter: ctype represents shared memory data type # The data integer represents the size of the open space # Return value: Shared memory object

""" value.py Open up a single shared memory space Note: Shared memory can only have one value """ from multiprocessing import Process, Value import time import random # Create Shared Memory money = Value('i', 5000) # Operational shared memory def man(): for i in range(30): time.sleep(0.2) money.value += random.randint(1, 1000) def girl(): for i in range(30): time.sleep(0.15) money.value -= random.randint(100, 800) p1 = Process(target=man) p2 = Process(target=girl) p1.start() p2.start() p1.join() p2.join() # Get Shared Memory print("One month balance", money.value)

""" array.py Shared memory holds a set of data """ from multiprocessing import Process, Array # Create Shared Memory # shm = Array('i', [1, 2, 3, 4]) # shm = Array('i', 5) # Initially open up five reshaping spaces shm = Array('c', b'hello') def func(): # Shared memory objects created by array can be iterated for i in shm: print(i) p = Process(target=func) p.start() p.join()

1. Communication principle
   Given a number, it is visible to multiple processes. Multiple processes can operate on the increase or decrease of this number and determine their own behavior based on the value of the number.
2. Implementation Method

from multiprocessing import Semaphore sem = Semaphore(num) # Function: Create semaphore object # Parameter: Initial value of semaphore # Return value: semaphore object sem.acquire() # Reduce semaphore by 1 Blocking when semaphore is 0 sem.release() # Add semaphore to 1 sem.get_value() # Get the number of semaphores

1. Code Samples

""" sem.py Semaphore demonstration Idea: The number of semaphores is equivalent to resources, so resources must be consumed to perform tasks """ from multiprocessing import Semaphore, Process from time import sleep import os # Create semaphores (up to 3 tasks allowed to execute simultaneously) sem = Semaphore(3) # Task Functions def handle(): sem.acquire() # A semaphore must be consumed to execute print("%s Execute Tasks" % os.getpid()) sleep(2) print("%s Task Execution Completed" % os.getpid()) sem.release() # Return semaphore # 10 tasks to perform for i in range(10): p = Process(target=handle) p.start()

7. Thread Programming

7.1 Thread Basic Concepts

1. Threads are called lightweight processes
2. Threads can also use computer multicore resources, which are multitask programming
3. Threads are the smallest unit of system allocation kernel
4. Threads can be understood as branch tasks of a process

1. A process can contain multiple threads
2. Threads are also a running behavior, consuming computer resources
3. All threads in a process share the resources of that process
4. Running between threads does not affect each other's operation
5. Thread creation and destruction consume much less resources than process
6. Each thread also has its own ID and other features

7.2 threading module create thread

1. Create Thread Object

from threading import Thread t = Thread() # Function: Create Thread Object # Parameter: target bound thread function # args tuple passes parameter to thread function location # The kwargs dictionary passes keys to thread functions

1. Start Thread

t.start()

1. Recycle Threads

t.join([timeout])

Code Samples

""" thread.py Thread Basic Usage Steps: 1.Encapsulate Thread Function 2.Create Thread Object 3.Start Thread 4.Recycle Threads """ import threading from time import sleep import os # Thread Functions def music(): for i in range(3): sleep(2) print(os.getpid(), "Play: Yellow River Chorus") # Create Thread Object t = threading.Thread(target=music) # Start Thread t.start() for i in range(4): sleep(1) print(os.getpid(), "Play: Cucurbita") # Recycle Threads t.join()

""" thread Thread Function Parameter Demo """ from threading import Thread from time import sleep # Thread function with parameters def func(sec, name): print("Thread Function Parameters") sleep(sec) print("%s completion of enforcement" % name) # Create plated multiple threads jobs = [] for i in range(5): t = Thread(target=func, args=(2,), kwargs={'name': 'T%d' % i}) jobs.append(t) t.start() for i in jobs: i.join()

7.3 Thread Object Properties

• t.name thread name
• t.setName() Sets the thread name
• t.getName() Gets the thread name
• t.is_alive() to see if a thread is in its lifecycle
• t.daemon sets the exit relationship between main and branch threads
• t.setDaemon() Sets the daemon property value
• t.isDaemon() View daemon attribute values
  The main thread exits the branch thread when daemon is True. To set it before start, it is usually not used with join
  Code Samples

""" Thread Properties """ from threading import Thread from time import sleep def func(): sleep(3) print("Thread Property Test") t = Thread(target=func, name='よう') t.setDaemon(True) # Main thread exits branch thread also exits t.start() print("name:", t.getName()) t.setName('Hello') print("name:", t.getName()) print("is alive:", t.is_alive()) print("daemon:", t.isDaemon()) # Print the value of daemon

7.4 Custom Thread Class

1. Inherit Thread Class
2. Rewrite_u init_u Method adds its own property, using super to load parent property
3. Override run method

1. Instantiate Object
2. Call start to automatically execute run method
3. Call join recycle thread

Code Samples

""" Custom Thread Class Example """ from threading import Thread # Custom Thread Class class ThreadClass(Thread): # Override parent init def __init__(self, *args, **kwargs): self.attr = args[0] super().__init__() # Load parent init # Assume there are many steps to complete def f1(self): print("step1") def f2(self): print("step2") # Override run logical call def run(self): self.f1() self.f2() t = ThreadClass('abc') t.start() t.join()

from threading import Thread from time import sleep, ctime class MyThread(Thread): def __init__(self, target=None, args=(), kwargs=None): super().__init__() # Passwords are not allowed on this line self.target = target self.args = args self.kwargs = kwargs def run(self): self.target(*self.args, **self.kwargs) def player(sec, song): for i in range(3): print("Playing %s : %s" % (song, ctime())) sleep(sec) t = MyThread(target=player, args=(3,), kwargs={'song': 'Cool'}) t.start() t.join()

7.5 Interthread Communication

Communication between threads using global variables

Shared resources: Resources that can be manipulated by multiple processes or threads are called shared resources. Operational code snippets for shared resources are called critical zones
Impact: Unordered operations on shared resources can lead to data confusion or operational errors. It is often necessary to synchronize mutexes to coordinate the order of operations.

Synchronization: Synchronization is a collaborative relationship in which multiple processes or threads form a coordination and perform operations in an orderly manner according to the necessary steps to complete an operation.

Mutual exclusion: Mutual exclusion is a restriction. When a process or thread occupies a resource, it is locked, and other processes or threads cannot operate the resource until it is unlocked.

1. Thread Event

from threading import Event e = Event() # Create Thread Evet Object e.wait([timeout]) # Block waiting for e to be set e.set() # Set e to end the wait blocking e.clear() # Return e to an unset state e.is_set() # Check if the current e is set

1. thread lock

from threading import Lock lock = Lock() # Create Lock Object lock.acquire() # Lock Up Blocking if lock is already locked and then called lock.release() # Unlock with lock: # Uplock ... ... # Automatically unlock when the with block ends

Code Samples

""" thread_lock.py thread lock """ from threading import Thread, Lock a = b = 0 lock = Lock() # Create Lock def value(): while True: lock.acquire() # Uplock if a != b: print("a = %d,b=%d" % (a, b)) lock.release() # Unlock t = Thread(target=value) t.start() while True: with lock: a += 1 b += 1 # Unlock automatically after the with statement ends

Add new comment

0 comments