Asynchronous processing of time-consuming tasks using task process in swoole

As we know, there are two processes in swoole, the master process and the manager management process.

There will be one main reactor thread and multiple reactor threads in the master main process. The main function is to maintain the TCP connection, process network IO, and send and receive data.

The role of manager is to fork and manage worker and task processes.

The function of the worker process is to receive the data passed by the reactor thread, process the data, and return the processing results to the reactor thread.

The role of task process is to process some relatively time-consuming tasks. Task and worker process are independent, which will not affect the worker process to process client requests.

 

I. application scenario of task process:

1. Relatively time-consuming mass mailing, such as a certain activity, needs to send activity mails to 100W users.

2. Push the dynamics of some big V, such as a new message sent by a big V, which fans need to get in time.

 

2. The relationship between worker and task:

1. In the worker process, task() can be called to post tasks. In the task process, the onTask event is used to respond to the tasks posted.

2. In the task process, you can directly return or call finish() to tell the worker process that the task has been processed. In the worker process, you can respond to the task through the onFinish event.

 

3. Before using task:

1. Configure the number of task worker num in the Server.

2. Set the Server's onTask and onFinish event callback functions.

 

IV. simple calculation example of using task to accumulate sum

<?php
 
$server = new swoole_server('0.0.0.0', 6666);
 
$server->set([
    'worker_num' => 2,
    'task_worker_num' => 16,
]);
 
$server->on('WorkerStart', function ($server, $worker_id) {
    //Notice here, we pass taskworker To judge is task Process or worker process
    //You need to call task() in the worker process, otherwise you will report a warning.
    //It will be executed twice because we set the number of worker num to 2
    if (!$server->taskworker) {
        echo 'Delivery task start...', PHP_EOL;
        //Deliver 32 cumulative calculation tasks to 16 task process
        for ($ix = 0; $ix < 32; $ix++) {
            //Note that the delivery here is asynchronous
            $server->task([mt_rand(1, 100), mt_rand(1000, 9999)]);
        }
        echo 'End of delivery task...', PHP_EOL;
    }
});
 
//server Service must have onReceive Callback
$server->on('Receive', function ($server, $fd, $reactor_id, $data) {
 
});
 
//Be careful, task The process is completely in synchronous blocking mode
$server->on('Task', function ($server, $task_id, $src_worker_id, $data) {
    echo "task {$task_id} Process is working...", PHP_EOL;
    $start = $data[0];
    $end = $data[1];
    $total = 0;
    for (; $start <= $end; $start++) {
        $total += $start;
    }
    echo "task {$task_id} Process completion...", PHP_EOL;
    return $total;
});
 
$server->on('Finish', function ($server, $task_id, $data) {
    echo "task {$task_id} Process processing complete, The result is {$data}", PHP_EOL;
});
 
$server->start();

Note that we call task() to deliver tasks to the task pool, and the bottom layer of swoole will poll the delivery tasks to each task process.

When the number of tasks you post exceeds the processing speed of onTask, it will cause the task pool to be full, and then the worker process will be blocked. Therefore, the relationship between the number of tasks and the processing speed should be set reasonably.

Of course, we can also manually deliver tasks to the specified task process. The second parameter of the task() function specifies the task process ID to be posted, with an ID range of 0 to (task ﹣ worker ﹣ num - 1).

 

V. divide the task s and control the delivery process manually

<?php
 
$server = new swoole_server('0.0.0.0', 6666);
 
$server->set([
    'worker_num' => 1,
    'task_worker_num' => 10,
]);
 
$server->on('WorkerStart', function ($server, $worker_id) {
    //To facilitate the demonstration, the worker_num Set to 1, only once
    if (!$server->taskworker) {
        //adopt swoole_table Shared memory, sharing data in different processes
        $server->result = new swoole_table(10240);
        //Used for preservation task Number of processes completed
        $server->result->column('finish_nums', swoole_table::TYPE_INT);
        //Used to save final calculation results
        $server->result->column('result', swoole_table::TYPE_INT);
        $server->result->create();
        //Calculate the cumulative sum of 1000 and assign 10 calculation tasks task Process
        $num = 1000;
        $step = $num / $server->setting['task_worker_num'];
        for ($ix = 0; $ix < $server->setting['task_worker_num']; $ix++) {
            $start = $ix * $step;
            $server->task([$start, $start + $step], $ix);
        }
    }
});
 
$server->on('Receive', function ($server, $fd, $reactor_id, $data) {
 
});
 
//Be careful, task The process is completely in synchronous blocking mode
$server->on('Task', function ($server, $task_id, $src_worker_id, $data) {
    echo "task {$task_id} Process is working... Calculation {$data[0]} - {$data[1]} ", PHP_EOL;
    $start = ++$data[0];
    $end = $data[1];
    $total = 0;
    for (; $start <= $end; $start++) {
        $total += $start;
    }
    echo "task {$task_id} Process completion...", PHP_EOL;
    return $total;
});
 
$server->on('Finish', function ($server, $task_id, $data) {
    echo "task {$task_id} Process processing complete, The result is {$data}", PHP_EOL;
    $server->result->incr('finish_nums', 'finish_nums');
    $server->result->set('result', ['result' => $data + $server->result->get('result', 'result')]);
 
    if ($server->result->get('finish_nums', 'finish_nums') == $server->setting['task_worker_num']) {
        echo "Final calculation result:{$server->result->get('result', 'result')}", PHP_EOL;
    }
});
 
$server->s
tart();

Tags: PHP network

Posted on Fri, 08 Nov 2019 08:57:40 -0500 by YodaOfCamarilla