Dolphin scheduler 2.0.0 source code analysis process (02)

The last article was:

Dolphin scheduler 2.0.0 source code analysis process (01)

 

We continue with the analysis of the previous article:

 

ok, let's take a screenshot to see our current project and background database:

 

 

 

 

 

  Next, let's click the task run button to start running a task. In the background, I will only start APIApplication server and MasterServer, not WorkerServer. Observe whether there will be an error.

 

 

 

  Post the master server log at this time:

[INFO] 2021-11-24 11:29:50.248 org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[243] - find command 79, slot:1 :
[INFO] 2021-11-24 11:29:50.249 org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[186] - find one command: id: 79, type: START_PROCESS
[INFO] 2021-11-24 11:29:50.300 org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[209] - handle command end, command 79 process 79 start...
[INFO] 2021-11-24 11:29:50.338 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread:[1146] - add task to stand by list: test_shell
[INFO] 2021-11-24 11:29:50.347 org.apache.dolphinscheduler.service.process.ProcessService:[1093] - start submit task : test_shell, instance id:79, state: RUNNING_EXECUTION
[INFO] 2021-11-24 11:29:50.376 org.apache.dolphinscheduler.service.process.ProcessService:[1106] - end submit task to db successfully:82 test_shell state:SUBMITTED_SUCCESS complete, instance id:79 state: RUNNING_EXECUTION  
[INFO] 2021-11-24 11:29:50.376 org.apache.dolphinscheduler.server.master.runner.task.CommonTaskProcessor:[120] - task ready to submit: TaskInstance{id=82, name='test_shell', taskType='SHELL', processInstanceId=79, processInstanceName='null', state=SUBMITTED_SUCCESS, firstSubmitTime=Wed Nov 24 11:29:50 CST 2021, submitTime=Wed Nov 24 11:29:50 CST 2021, startTime=null, endTime=null, host='null', executePath='null', logPath='null', retryTimes=0, alertFlag=NO, processInstance=null, processDefine=null, pid=0, appLink='null', flag=YES, dependency='null', duration=null, maxRetryTimes=0, retryInterval=1, taskInstancePriority=MEDIUM, processInstancePriority=MEDIUM, dependentResult='null', workerGroup='default', environmentCode=-1, environmentConfig='null', executorId=1, executorName='null', delayTime=0, dryRun=0}
[INFO] 2021-11-24 11:29:50.438 org.apache.dolphinscheduler.server.master.runner.task.CommonTaskProcessor:[130] - master submit success, task : test_shell
[ERROR] 2021-11-24 11:29:50.448 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=19, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=19, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[INFO] 2021-11-24 11:29:50.458 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread:[1162] - remove task from stand by list, id: 82 name:test_shell
[ERROR] 2021-11-24 11:29:53.468 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=20, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=20, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:29:56.488 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=21, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=21, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:29:59.509 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=22, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=22, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:02.530 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=23, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=23, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:05.550 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=24, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=24, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:08.612 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=25, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=25, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:11.630 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=26, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=26, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:14.642 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=27, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=27, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:17.651 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=28, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=28, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:20.670 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=29, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=29, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:23.698 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=30, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=30, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)
[ERROR] 2021-11-24 11:30:26.717 org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[140] - dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=31, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
org.apache.dolphinscheduler.server.master.dispatch.exceptions.ExecuteException: fail to execute : Command [type=TASK_EXECUTE_REQUEST, opaque=31, bodyLen=1673] due to no suitable worker, current task needs worker group default to execute
    at org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher.dispatch(ExecutorDispatcher.java:89)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.dispatch(TaskPriorityQueueConsumer.java:137)
    at org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer.run(TaskPriorityQueueConsumer.java:100)

It can be seen that errors have been reported all the time. Let's analyze the logs here and explain them sentence by sentence:

The first line is: find command 79, slot:1:  

Find task 79, in slot 1. What do you mean? In fact, when we save a task, our task is saved to t_ ds_ task_ In the definition table, the MasterServer will periodically set this t according to the scheduled task time_ ds_ task_ Each task definition that meets the execution conditions in the definition table becomes a task instance and is saved to t_ ds_ In the command table. t_ds_command each record represents the task instance to be executed. Next, the MasterServer will make a hash calculation according to the number of tasks and the number of MasterServer threads to calculate which MasterServer is responsible for the whole process execution monitoring of the command. If a command is assigned to a MasterServer, the MasterServer needs to be responsible for monitoring the execution of the command. The specific execution is that the MasterServer contacts the WorkerServer process to work. When the WorkerServer finishes the work, it needs to feed back to the MasterServer, and then the MasterServer updates the task instance status to completed. The command table is generally empty when observed, because once the MasterServer takes the command, it empties the record from the command table, and then adds an executing record to t_ds_task_instance table. Therefore, when the execution is completed, the table updated by the MasterServer is t_ds_task_instance this table. Not a command table.

 

The second line is: find one command: id: 79, type: START_PROCESS   

A command with id 79 and type start was found_ process

 

The third line is: handle command end, command 79 process 79 start

The processing task ends and the sub process 79 of task 79 begins to execute

 

Posted on Tue, 23 Nov 2021 23:17:16 -0500 by daynah