Apache Doris manages processes through the supervisor (auto start is suspended)

Apache Doris manages processes through the supervisor (auto start is suspended)

catalogue

1, Overview

Supervisor is a general process management program developed in Python. It can change an ordinary command-line process into a background daemon, monitor the process status, and automatically restart in case of abnormal exit. It starts these managed processes as child processes of the supervisor through fork/exec, so as long as the path of the executable file of the process to be managed is written in the supervisor configuration file. It also realizes that when the child process hangs, the parent process can accurately obtain the hanging information of the child process, and can choose whether to start and alarm by itself. Supervisor also provides a function to set a non root user for supervisor or each child process, and this user can manage its corresponding process.

2, Install supervisor

2.1.install supervisor using yum

$ yum -y install epel-release
$ yum -y install supervisor

2.2. Modify supervisor configuration

$ vi /etc/supervisord.conf

minfds=655350                 ; (min. avail startup file descriptors;default 1024)
minprocs=65535                ; (min. avail process descriptors;default 200)

2.3. Start supervisor

$ systemctl enable supervisord # Startup self startup
$ systemctl start supervisord # Start the supervisor service
$ systemctl status supervisord # View supervisor service status

#Restart Supervisor
$ systemctl restart supervisord

3, Configure BE auto start

3.1. Create and edit BE bootstrap file doris_be.ini

$ vi /etc/supervisord.d/doris_be.ini

doris_be.ini content

[program:doris_be]      
process_name=%(program_name)s      
directory=/usr/local/doris/be
command=sh /usr/local/doris/be/bin/start_be.sh
autostart=true
autorestart=true
user=root
numprocs=1
startretries=3
stopasgroup=true
killasgroup=true
startsecs=5

3.2. Loading doris_be.ini

#Refresh configuration
$ supervisorctl update

#View status
$ supervisorctl status

If it fails, you can stop be first, and then start be through supervisorctl

$ sh /usr/local/doris/be/bin/stop_be.sh
$ supervisorctl start doris_be

#Stop be via supervisorctl
$ supervisorctl stop doris_be

#Restart be via supervisorctl
$ supervisorctl restart doris_be

#Start all applications managed by supervisor
$ supervisorctl start all 

4, Configure FE auto start

4.1. Create and edit FE bootstrap file doris_fe.ini

vi /etc/supervisord.d/doris_fe.ini

doris_fe.ini content

[program:doris_fe]
environment = JAVA_HOME="/usr/lib/jvm/java-1.8.0"
process_name=%(program_name)s                       ;Process name
directory=/usr/local/doris/fe                       ;working directory
command=sh /usr/local/doris/fe/bin/start_fe.sh      ;Commands to run
autostart=true                                      ;Automatic opening
autorestart=true                                    ;Automatic restart
user=root                                           ;user
numprocs=1                                          ;Number of processes
startretries=3                                      ;Spin Retry Count 
stopasgroup=true                                    ;Stop child process
killasgroup=true                                    ;Kill child process
;startsecs=10                                       ;After starting for 10 seconds, if it is still running, it is considered that the process has been started

4.2. Loading doris_fe.ini

#Refresh configuration
$ supervisorctl update

#View status
$supervisorctl status

If the status fails, you can stop fe first and then start fe through supervisorctl

$ sh /usr/local/doris/fe/bin/stop_fe.sh
$ supervisorctl start doris_fe

5, Configure broker bootstrap

5.1. Create and edit Broker bootstrap file doris_broker.ini

vi /etc/supervisord.d/doris_broker.ini

doris_broker.ini content

[program:doris_broker]
environment = JAVA_HOME="/usr/lib/jvm/java-1.8.0"
process_name=%(program_name)s                       ;Process name
directory=/usr/local/doris/apache_hdfs_broker                       ;working directory
command=sh /usr/local/doris/apache_hdfs_broker/bin/start_broker.sh      ;Commands to run
autostart=true                                      ;Automatic opening
autorestart=true                                    ;Automatic restart
user=root                                           ;user
numprocs=1                                          ;Number of processes
startretries=3                                      ;Spin Retry Count 
stopasgroup=true                                    ;Stop child process
killasgroup=true                                    ;Kill child process
;startsecs=10                                       ;After starting for 10 seconds, if it is still running, it is considered that the process has been started

4.2. Loading doris_broker.ini

#Refresh configuration
$ supervisorctl update

#View status
$supervisorctl status

If the status fails, you can stop the broker first, and then start the broker through supervisor CTL

$ sh /usr/local/doris/apache_hdfs_broker/bin/stop_broker.sh
$ supervisorctl start doris_broker

6, Relevant reference

Tags: Operation & Maintenance Doris

Posted on Mon, 08 Nov 2021 01:07:26 -0500 by digibrain