Operating System History

Reference Blog: https://www.cnblogs.com/Dominic-Ji/articles/10929381.html

Functions of the operating system:

1. Hide the ugly hardware call interface, providing a better, simpler and clearer model (system call interface) for the application programmer to call hardware resources. With these interfaces, an application developer can concentrate on developing his or her own application without considering the details of operating hardware.

2. Orderize application race requests for hardware resources

Computer history

• First-generation computers: vacuum tubes and perforated cards. People convert programs into perforated cards beforehand and execute them on computers
• Second-generation computers: transistors and batch processing systems: all programs are processed together into perforated cards (batches), run on the computer, executed sequentially, and distributed to others after processing results
• Third Generation Computers: Integrated Circuit Chips and Multichannel Technologies: Basically today's computers

Multichannel Technology

The purpose of multichannel technology is to achieve the concurrent effect of a single core. Computers are generally multicore now. In fact, each core implements multichannel technology.

Multichannel refers to multiple programs. Multichannel technology is implemented to solve the problem of orderly scheduling of multiple programs or sharing the same resource (such as cpu). The solution is multiplex, which is divided into time multiplex and space multiplex.

• Spatial reuse: divide the memory into parts, each part into a program, so that there are multiple programs in the memory at the same time

• Time reuse: When one program is waiting for I/O, another program can use cpu. If enough jobs can be stored in memory at the same time, the utilization of CPU can reach nearly 100%, which is similar to the overall method we learned in elementary school mathematics. (When the operating system uses multichannel technology, it can control the switching of processes, or compete for the execution rights of the CPU between processes. This switching will not only happen when a process encounters io, but also when a process takes up too much CPU time, it will switch, or the operating system will take away the execution rights of the cpu)

• Switch CPU in both cases

  1. When a program encounters an IO operation, the operating system deprives the program of CPU execution privileges
     Role: Improves CPU utilization without affecting program execution efficiency

  2. When a program consumes CPU for a long time, the operating system also deprives the program of CPU execution privileges
     Explain that if a program takes up CPU for a long time, the operating system will execute the program that takes up CPU for a short time before it returns to execute the original program, but this actually reduces the efficiency of the CPU.
     Disadvantages: Reduced program execution efficiency (original time + switching time)

socketserver for concurrency

tcp-based sockets, the key is two loops, one link loop, one communication loop

There are two main categories in the socketserver module: the server class (resolving link problems) and the request class (resolving communication problems)

Take the following code as an example to analyze the socketserver source code:

ftpserver=socketserver.ThreadingTCPServer(('127.0.0.1',8080),FtpServer)
ftpserver.serve_forever()

Order of finding attributes: ThreadingTCPServer->ThreadingMixIn->TCPServer->BaseServer

1. Instantiate to get ftpserver, first look for u of class ThreadingTCPServer init_u, Find in TCPServer and execute server_bind,server_active
2. Find serve_under ftpserver Forever, found in BaseServer, to execute self._handle_request_noblock(), which is also in BaseServer
3. Execute self._handle_request_noblock() to execute request, client_address = self.get_request() (that is, self.socket.accept()) in TCP Server) and execute self.process_request(request, client_address)
4. Find process_in ThreadingMixIn Request, turn on Multithreading for concurrency, and execute process_request_thread, execute self.finish_request(request, client_address)
5. The four sections above complete the link loop, which begins with the processing of communications and finds finish_in BaseServer Request, trigger the instantiation of our own defined class, to find u init_u Method, and our own class does not have this method, then go to its parent, BaseRequestHandler...

Source analysis summary:

tcp-based socketserver s in our own defined classes

1. self.server is the socket object
2. self.request is a link
3. self.client_address is the client address

udp-based socketserver s in our own defined classes

1. self.request is a tuple (the first element is data from the client and the second part is udp socket object from the server), such as (b'adsf', <socket.socket fd=200, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 8080)>)
2. self.client_address is the client address

# Server
import socketserver
import struct
import json
import os
class FtpServer(socketserver.BaseRequestHandler):
    coding='utf-8'
    server_dir='file_upload'
    max_packet_size=1024
    BASE_DIR=os.path.dirname(os.path.abspath(__file__))
    def handle(self):
        print(self.request)
        while True:
            data=self.request.recv(4)
            data_len=struct.unpack('i',data)[0]
            head_json=self.request.recv(data_len).decode(self.coding)
            head_dic=json.loads(head_json)
            # print(head_dic)
            cmd=head_dic['cmd']
            if hasattr(self,cmd):
                func=getattr(self,cmd)
                func(head_dic)
    def put(self,args):
        file_path = os.path.normpath(os.path.join(
            self.BASE_DIR,
            self.server_dir,
            args['filename']
        ))

        filesize = args['filesize']
        recv_size = 0
        print('----->', file_path)
        with open(file_path, 'wb') as f:
            while recv_size < filesize:
                recv_data = self.request.recv(self.max_packet_size)
                f.write(recv_data)
                recv_size += len(recv_data)
                print('recvsize:%s filesize:%s' % (recv_size, filesize))

ftpserver=socketserver.ThreadingTCPServer(('127.0.0.1',8080),FtpServer)
ftpserver.serve_forever()

# Client
import socket
import struct
import json
import os

class MYTCPClient:
    address_family = socket.AF_INET
    socket_type = socket.SOCK_STREAM
    allow_reuse_address = False
    max_packet_size = 8192
    coding='utf-8'
    request_queue_size = 5

    def __init__(self, server_address, connect=True):
        self.server_address=server_address
        self.socket = socket.socket(self.address_family,
                                    self.socket_type)
        if connect:
            try:
                self.client_connect()
            except:
                self.client_close()
                raise

    def client_connect(self):
        self.socket.connect(self.server_address)

    def client_close(self):
        self.socket.close()

    def run(self):
        while True:
            inp=input(">>: ").strip()
            if not inp:continue
            l=inp.split()
            cmd=l[0]
            if hasattr(self,cmd):
                func=getattr(self,cmd)
                func(l)

    def put(self,args):
        cmd=args[0]
        filename=args[1]
        if not os.path.isfile(filename):
            print('file:%s is not exists' %filename)
            return
        else:
            filesize=os.path.getsize(filename)

        head_dic={'cmd':cmd,'filename':os.path.basename(filename),'filesize':filesize}
        print(head_dic)
        head_json=json.dumps(head_dic)
        head_json_bytes=bytes(head_json,encoding=self.coding)

        head_struct=struct.pack('i',len(head_json_bytes))
        self.socket.send(head_struct)
        self.socket.send(head_json_bytes)
        send_size=0
        with open(filename,'rb') as f:
            for line in f:
                self.socket.send(line)
                send_size+=len(line)
                print(send_size)
            else:
                print('upload successful')

client=MYTCPClient(('127.0.0.1',8080))
client.run()