Spring Job? Quartz? XXL-Job? Young people make choices~

Hello, my name is grandma. I'm a little fat man in the subway~

Today, let's take a look at the technology selection of timing tasks in Spring Boot applications~

1. Overview

Among the colorful and black requirements of products, there is a kind of requirement, which needs to be executed at a fixed time. At this time, it needs to use a fixed time task. For example, scan the overtime payment order every minute, clean up the log file every hour, count the data of the previous day and generate the report every day, push the payslip at the beginning of each month, remind the birthday every year, etc.

Among them, she likes "pushing the payroll at the beginning of every month". How about you?

In the JDK, two classes are built in to implement the function of timing tasks:

  • java.util.Timer : can be created by java.util.TimerTask Scheduling tasks, in the same thread serial execution, mutual influence. That is to say, for multiple timertasks in the same Timer, if one TimerTask is in execution, other timertasks can only wait in line even when it reaches the execution time. Because Timer is serial and exists at the same time Pit , so later JDK launched the ScheduledExecutorService, and Timer was basically no longer used.
  • java.util.concurrent.ScheduledExecutorService : in JDK 1.5, a new timed task class based on the thread pool design is added. Each scheduled task will be assigned to the thread pool for concurrent execution without mutual influence. In this way, ScheduledExecutorService solves the problem of Timer serial.

In daily development, we seldom use Timer or schedulexecutiorservice directly to implement the requirements of scheduled tasks. There are several main reasons:

  • They only support the scheduling according to the specified frequency, not directly support the timing scheduling of the specified time. We need to combine Calendar to calculate by ourselves to achieve the scheduling of complex time. For example, every day, every Friday, November 11, 2019, etc.
  • They are process level, and we need to deploy multiple processes to achieve high availability of scheduled tasks. At this time, we need to wait for more consideration. Under multiple processes, the same task cannot be executed repeatedly at the same time.
  • There may be many timing tasks in the project, which need unified management. At this time, we have to conduct secondary encapsulation.

Therefore, in general, we will choose professional scheduling task middleware.

As for the name of "Task", there are also "homework". In English, there are tasks and jobs. The essence is the same, both of which will be used in this article.

Then, generally speaking, it is to schedule tasks and execute them regularly. So fat friends will see the word "scheduling" or "timing" in this article or other articles.

In the Spring system, there are two built-in solutions for timed tasks:

  • The first, Spring Framework Of Spring Task Module, which provides the realization of lightweight timing task.
  • Second, Spring Boot Version 2.0, integrated Quartz Job scheduling framework provides a powerful implementation of timed tasks.

    Note: Spring Framework has built-in integration of Quartz. Spring boot version 1. X does not provide automatic configuration of Quartz, while version 2.X does.

In the Java ecosystem, there are many excellent open-source scheduling task middleware:

At present, elastic job and XXL job are mainly used in China. From what we learned from the company, there will be more teams using XXL-JOB, mainly because it is easier to start and the operation and maintenance function is more perfect.

This article provides a complete code example, which can be seen in https://github.com/YunaiV/Spr... Of lab-28 catalog.

It's not easy to be original. Give me some Star Hey, duck!

2. Quick start Spring Task

Example code corresponding warehouse: lab-28-task-demo .

Considering that we seldom use Spring Task in actual scenarios, this section will be relatively concise. If you are interested in Spring Task, you can read it by yourself <Spring Framework Documentation - Task Execution and Scheduling> Document, which contains detailed documents related to Spring Task.

In this section, we will use the Spring Task function to implement a scheduled task that prints a row of execution logs every 2 seconds.

2.1 introduce dependency

stay pom.xml File, introduce related dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>lab-28-task-demo</artifactId>

    <dependencies>
        <!-- Realize the right Spring MVC Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
    </dependencies>

</project>

Because Spring Task is a module of Spring Framework, we do not need to introduce it specially after we introduce spring boot starter web dependency.

At the same time, considering that we want the project to start without automatically ending the JVM process, we introduce the spring boot starter web dependency.

2.2 ScheduleConfiguration

stay cn.iocoder.springboot.lab28.task.config Under package path, create ScheduleConfiguration Class to configure Spring Task. The code is as follows:

// ScheduleConfiguration.java

@Configuration
@EnableScheduling
public class ScheduleConfiguration {
}
  • On a class, add @EnableScheduling Annotation to start the function of scheduled task scheduling of Spring Task.

2.3 DemoJob

stay cn.iocoder.springboot.lab28.task.job Under package path, create DemoJob Class, sample scheduled task class. The code is as follows:

// DemoJob.java

@Component
public class DemoJob {

    private Logger logger = LoggerFactory.getLogger(getClass());

    private final AtomicInteger counts = new AtomicInteger();

    @Scheduled(fixedRate = 2000)
    public void execute() {
        logger.info("[execute][Timing ({}) Secondary execution]", counts.incrementAndGet());
    }

}
  • On the class, add the @ Component annotation to create the DemoJob Bean object.
  • Create the execute() method to print the log. At the same time, on the method, add @Scheduled Annotation, set to execute the method every 2 seconds.

Although @ Scheduled annotation can be added to multiple methods of a class, it is still a Job class and a Scheduled task. 😈

2.4 Application

establish Application.java Class, configure @ SpringBootApplication annotation. The code is as follows:

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

Run the Application class to start the sample project. The output log is simplified as follows:

# Initializing a ThreadPoolTaskScheduler task scheduler
2019-11-30 18:02:58.415  INFO 83730 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'

# Perform DemoJob tasks every 2 seconds
2019-11-30 18:02:58.449  INFO 83730 --- [ pikaqiu-demo-1] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (1) Secondary execution]
2019-11-30 18:03:00.438  INFO 83730 --- [ pikaqiu-demo-1] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (2) Secondary execution]
2019-11-30 18:03:02.442  INFO 83730 --- [ pikaqiu-demo-2] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (3) Secondary execution]
  • Through the log, we can see that a ThreadPoolTaskScheduler task scheduler is initialized. After that, every 2 seconds, perform the DemoJob task.

At this point, we have completed the introduction of Spring Task scheduling function. In fact, Spring Task also provides Asynchronous task , which we will explain in detail in other articles.

below 「2.5 @Scheduled」 and 「 2.6 application profile 」 The two sections are supplementary knowledge. I suggest you take a look.

2.5 @Scheduled

@Scheduled Annotation, set the execution plan of scheduled tasks.

Common attributes are as follows:

  • Cron attribute: Spring Cron expression. For example, "0.012 * *?" means to execute once a day at noon, "11.11.11.11?" means to execute once at 11:11:11 on November 11 (HA HA). For more examples and explanations, see Spring Cron expression article. Note that the start time is the call completion time.
  • fixedDelay property: fixed execution interval in milliseconds. Note that the start time is the call completion time.
  • fixedRate property: fixed execution interval in milliseconds. Note that the start time is the call start time.
  • These three attributes are a little similar. Let's see Differences between fixedRate, fixedDelay and cron of @ Scheduled scheduled Scheduled scheduled tasks , be sure to distinguish the differences.

The uncommon attributes are as follows:

  • initialDelay property: the execution delay of the initialized scheduled task, in milliseconds.
  • Zone attribute: resolves the time zone of the Spring Cron expression. By default, the local time zone of the server is used.
  • initialDelayString property: the string form of initialDelay.
  • fixedDelayString property: the string form of fixedDelay.
  • fixedRateString property: the string form of fixedRate.

2.6 application profile

stay application.yml Add the configuration of Spring Task timing task as follows:

spring:
  task:
    # Configuration of Spring Task scheduling tasks, corresponding to taskscheduleingproperties configuration class
    scheduling:
      thread-name-prefix: pikaqiu-demo- # Prefix for the thread name of the thread pool. The default is scheduling -. It is recommended to set it according to your own application
      pool:
        size: 10 # Thread pool size. The default is 1, set according to your own application
      shutdown:
        await-termination: true # Whether to wait for the scheduled task to complete when the application is closed. The default value is false, and it is recommended to set it to true
        await-termination-period: 60 # The maximum time, in seconds, to wait for a task to complete. The default is 0, set according to your own application
  • At spring.task.scheduling Configuration item, configuration of Spring Task scheduling task, corresponding to TaskSchedulingProperties Configuration class.
  • Spring Boot TaskSchedulingAutoConfiguration Automatic configuration class to realize automatic configuration of Spring Task and create ThreadPoolTaskScheduler Task scheduler based on thread pool. In essence, ThreadPoolTaskScheduler is based on the encapsulation of ScheduledExecutorService to enhance the function in scheduling time.

be careful, spring.task.scheduling The. Shutdown configuration item is used to implement the graceful shutdown of Spring Task scheduled tasks. Let's imagine that if during the execution of a scheduled task, if the application starts to shut down and the Spring Bean that the scheduled task needs to use is destroyed, for example, the database connection pool, then the scheduled task is still executing at this time. Once the database needs to be accessed, an error may be reported.

  • Therefore, by configuring await termination = true, when the application is closed, wait for the scheduled task to complete. In this way, when the application is closed, Spring will first wait for the ThreadPoolTaskScheduler to finish executing the task, and then start to destroy the Spring Bean.
  • At the same time, considering that it is impossible for us to wait indefinitely for all scheduled tasks to finish, we can configure await termination period = 60, and the maximum waiting time for tasks to finish, in seconds. The specific waiting time can be set according to the needs of your application.

3. Quick start Quartz stand alone

Example code corresponding warehouse: lab-28-task-quartz-memory .

At the beginning of the internship, the company used Quartz as task scheduling middleware. Considering that we need to deploy multiple JVM processes to achieve high availability of scheduled tasks. More comfortable, Quartz comes with its own clustering solution. It stores the job information in the relational database, and uses the row lock of the relational database to realize the competition of executing jobs, so as to ensure that the same task cannot be executed repeatedly in the same time in multiple processes.

Maybe many fat friends don't know much about Quartz. Let's take a look at a brief introduction:

FROM https://www.oschina.net/p/quartz

Quartz is an open source job scheduling framework, which is completely written in Java and designed for J2SE and J2EE applications. It provides great flexibility without sacrificing simplicity. You can use it to create simple or complex schedules for executing a job.

It has many features, such as database support, clustering, plug-ins, EJB job pre construction, JavaMail and others, support for cron like expressions and so on.

In the Quartz architecture, three components are important:

  • Scheduler: scheduler
  • Trigger: trigger
  • Job: task

Fat friends who don't know can have a look directly Introduction to Quartz article. Here, she does not repeat.

FROM https://medium.com/@ChamithKo...

Overall architecture of Quartz

Quartz is divided into stand-alone mode and cluster mode.

  • In this section, let's first learn the single machine mode of Quartz, which is relatively fast to get started.
  • Next time "5. Getting started with Quartz cluster again" , let's learn about the cluster mode of Quartz. In the production environment, we must use the cluster mode of Quartz to ensure the high availability of scheduled tasks.

😈 Now, let's start to travel~

3.1 introduce dependency

stay pom.xml File, introduce related dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>lab-28-task-quartz-memory</artifactId>

    <dependencies>
        <!-- Realize the right Spring MVC Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- Realize the right Quartz Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-quartz</artifactId>
        </dependency>
    </dependencies>

</project>

For the specific role of each dependency, fat friend takes a look at all the comments added by grandma.

3.2 example Job

stay cn.iocoder.springboot.lab28.task.config.job Under package path, let's create a sample Job.

establish DemoJob01 Class, example timing task 01 class. The code is as follows:

// DemoJob01.java

public class DemoJob01 extends QuartzJobBean {

    private Logger logger = LoggerFactory.getLogger(getClass());

    private final AtomicInteger counts = new AtomicInteger();

    @Autowired
    private DemoService demoService;

    @Override
    protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
        logger.info("[executeInternal][Timing ({}) Secondary execution, demoService by ({})]", counts.incrementAndGet(),
                demoService);
    }

}
  • inherit QuartzJobBean Abstract class, which implements the method of ා executeInternal(JobExecutionContext context), and implements the logic of customized timing tasks.
  • QuartzJobBean implemented org.quartz.Job Interface, which provides Quartz to inject the dependency properties of the Job Bean every time it creates the Job execution timing logic. For example, DemoJob01 needs @ Autowired injected demoService Property. The core code is as follows:

    // QuartzJobBean.java
    
    public final void execute(JobExecutionContext context) throws JobExecutionException {
        try {
            // Wrap the current object as a BeanWrapper object
            BeanWrapper bw = PropertyAccessorFactory.forBeanPropertyAccess(this);
            // Set properties to bw
            MutablePropertyValues pvs = new MutablePropertyValues();
            pvs.addPropertyValues(context.getScheduler().getContext());
            pvs.addPropertyValues(context.getMergedJobDataMap());
            bw.setPropertyValues(pvs, true);
        } catch (SchedulerException ex) {
            throw new JobExecutionException(ex);
        }
    
        // Execute abstract methods provided to subclass implementation
        this.executeInternal(context);
    }
    
    protected abstract void executeInternal(JobExecutionContext context) throws JobExecutionException;
    • At this point of view, it's a lot clearer. Don't be afraid of the source code of middleware. If you are curious about which class or method, just click it. Anyway, it doesn't cost money.
  • Count property, counter. As we will show later, each time DemoJob01 is created by Quartz, a new Job object will execute the task. This is very important, and we should be very careful.

establish DemoJob02 Class, example timing task 02 class. The code is as follows:

// DemoJob02.java

public class DemoJob02 extends QuartzJobBean {

    private Logger logger = LoggerFactory.getLogger(getClass());

    @Override
    protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
        logger.info("[executeInternal][I started]");
    }

}
  • It's relatively simple, for the purpose of demonstrating the case later.

3.3 ScheduleConfiguration

stay cn.iocoder.springboot.lab28.task.config Under package path, create ScheduleConfiguration Class to configure the two sample jobs above. The code is as follows:

// ScheduleConfiguration.java

@Configuration
public class ScheduleConfiguration {

    public static class DemoJob01Configuration {

        @Bean
        public JobDetail demoJob01() {
            return JobBuilder.newJob(DemoJob01.class)
                    .withIdentity("demoJob01") // The name is demoJob01
                    .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                    .build();
        }

        @Bean
        public Trigger demoJob01Trigger() {
            // A simple constructor for scheduling
            SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
                    .withIntervalInSeconds(5) // Frequency.
                    .repeatForever(); // Number of times.
            // Trigger constructor
            return TriggerBuilder.newTrigger()
                    .forJob(demoJob01()) // The corresponding Job is demoJob01
                    .withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
                    .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                    .build();
        }

    }

    public static class DemoJob02Configuration {

        @Bean
        public JobDetail demoJob02() {
            return JobBuilder.newJob(DemoJob02.class)
                    .withIdentity("demoJob02") // The name is demoJob02
                    .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                    .build();
        }

        @Bean
        public Trigger demoJob02Trigger() {
            // Constructor of scheduling plan based on Quartz Cron expression
            CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * * ? *");
            // Trigger constructor
            return TriggerBuilder.newTrigger()
                    .forJob(demoJob02()) // The corresponding Job is demoJob02
                    .withIdentity("demoJob02Trigger") // The name is demoJob02Trigger
                    .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                    .build();
        }

    }

}
  • Two configuration classes, DemoJob01Configuration and DemoJob02Configuration, are created internally, and two quartz jobs, DemoJob01 and DemoJob02, are configured respectively.
  • ========== DemoJob01Configuration ==========
  • #demoJob01() method to create the JobDetail Bean object of DemoJob01.
  • #demoJob01Trigger() method to create the Trigger Bean object of DemoJob01. Among them, we use SimpleScheduleBuilder A simple scheduling plan builder creates a scheduling plan that executes every 5 seconds and repeats infinitely.
  • ========== DemoJob2Configuration ==========
  • #demoJob2() method to create the JobDetail Bean object of DemoJob02.
  • #demoJob02Trigger() method to create the Trigger Bean object of DemoJob02. Among them, we use CronScheduleBuilder The constructor of scheduling plan based on Quartz Cron expression creates a scheduling plan that is executed every 10th second. Here, recommend one Quartz/Cron/Crontab expression online generation tool , which is convenient for us to generate the Quartz Cron expression and calculate the last 5 run times.

😈 Because JobDetail and Trigger usually appear in pairs, grandma is used to configuring them as a Configuration class.

3.4 Application

establish Application.java Class, configure @ SpringBootApplication annotation. The code is as follows:

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

Run the Application class to start the sample project. The output log is simplified as follows:

# Created and started the quartz quartz scheduler
2019-11-30 23:40:05.123  INFO 92812 --- [           main] org.quartz.impl.StdSchedulerFactory      : Using default implementation for ThreadExecutor
2019-11-30 23:40:05.130  INFO 92812 --- [           main] org.quartz.core.SchedulerSignalerImpl    : Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
2019-11-30 23:40:05.130  INFO 92812 --- [           main] org.quartz.core.QuartzScheduler          : Quartz Scheduler v.2.3.2 created.
2019-11-30 23:40:05.131  INFO 92812 --- [           main] org.quartz.simpl.RAMJobStore             : RAMJobStore initialized.
2019-11-30 23:40:05.132  INFO 92812 --- [           main] org.quartz.core.QuartzScheduler          : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'quartzScheduler' with instanceId 'NON_CLUSTERED'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
  Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

2019-11-30 23:40:05.132  INFO 92812 --- [           main] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler 'quartzScheduler' initialized from an externally provided properties instance.
2019-11-30 23:40:05.132  INFO 92812 --- [           main] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler version: 2.3.2
2019-11-30 23:40:05.132  INFO 92812 --- [           main] org.quartz.core.QuartzScheduler          : JobFactory set to: org.springframework.scheduling.quartz.SpringBeanJobFactory@203dd56b
2019-11-30 23:40:05.158  INFO 92812 --- [           main] o.s.s.quartz.SchedulerFactoryBean        : Starting Quartz Scheduler now
2019-11-30 23:40:05.158  INFO 92812 --- [           main] org.quartz.core.QuartzScheduler          : Scheduler quartzScheduler_$_NON_CLUSTERED started.

# DemoJob01
2019-11-30 23:40:05.164  INFO 92812 --- [eduler_Worker-1] c.i.springboot.lab28.task.job.DemoJob01  : [executeInternal][Timing (1) Secondary execution, demoService by (cn.iocoder.springboot.lab28.task.service.DemoService@23d75d74)]
2019-11-30 23:40:09.866  INFO 92812 --- [eduler_Worker-2] c.i.springboot.lab28.task.job.DemoJob01  : [executeInternal][Timing (1) Secondary execution, demoService by (cn.iocoder.springboot.lab28.task.service.DemoService@23d75d74)]
2019-11-30 23:40:14.865  INFO 92812 --- [eduler_Worker-4] c.i.springboot.lab28.task.job.DemoJob01  : [executeInternal][Timing (1) Secondary execution, demoService by (cn.iocoder.springboot.lab28.task.service.DemoService@23d75d74)]

# DemoJob02
2019-11-30 23:40:10.004  INFO 92812 --- [eduler_Worker-3] c.i.springboot.lab28.task.job.DemoJob02  : [executeInternal][I started]
2019-11-30 23:40:20.001  INFO 92812 --- [eduler_Worker-6] c.i.springboot.lab28.task.job.DemoJob02  : [executeInternal][I started]
2019-11-30 23:40:30.002  INFO 92812 --- [eduler_Worker-9] c.i.springboot.lab28.task.job.DemoJob02  : [executeInternal][I started]
  • When the project starts, the quartz quartz scheduler is created and started.
  • Considering the convenience of reading the logs, grandma separated the logs of DemoJob01 and DemoJob02 here.
  • For DemoJob01, every 5 seconds or so. At the same time, we can see that demoService is injected successfully, while counts are 1 each time, indicating that DemoJob01 is newly created each time.
  • For DemoJob02, every 10th second.

below 「 3.5 application profile 」 The two sections are supplementary knowledge. I suggest you take a look.

3.5 application profile

stay application.yml In, add the configuration of Quartz as follows:

spring:
  # Quartz configuration, corresponding to the QuartzProperties configuration class
  quartz:
    job-store-type: memory # Job storage type. The default value is memory, and optional jdbc uses database.
    auto-startup: true # Whether Quartz starts automatically
    startup-delay: 0 # Delay N seconds to start
    wait-for-jobs-to-complete-on-shutdown: true # Whether to wait for the scheduled task to complete when the application is closed. The default value is false, and it is recommended to set it to true
    overwrite-existing-jobs: false # Whether to overwrite the configuration of an existing Job
    properties: # Add additional properties of Quartz Scheduler. For more information, see http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/configuration.html  file
      org:
        quartz:
          threadPool:
            threadCount: 25 # Thread pool size. The default is 10.
            threadPriority: 5 # thread priority 
            class: org.quartz.simpl.SimpleThreadPool # Thread pool type
#    jdbc: # It's not explained here for the moment. When using JDBC's JobStore, you need to configure it
  • At spring.quartz Configuration item, configuration of Quartz, corresponding to QuartzProperties Configuration class.
  • Spring Boot QuartzAutoConfiguration Automatic configuration class to realize automatic configuration of Quartz and create Quartz Scheduler (scheduler) Bean.

be careful, spring.Quartz.wait -For jobs to complete on shutdown configuration item is for elegant shutdown of Quartz. It is recommended to enable it. About this, and our Spring Task 「 2.6 application profile 」 It is consistent.

4. Getting started with Quartz cluster again

Example code corresponding warehouse: lab-28-task-quartz-memory .

In the actual scenario, we must consider the high availability of scheduled tasks, so basically, we must use the Quartz cluster scheme. So in this section, we use Quartz's JDBC storage JobStoreTX , and uses MySQL as the database.

Here's a comparison of two types of Quartz memory:

FROM https://blog.csdn.net/Evankak...

type advantage shortcoming
RAMJobStore No external database, easy to configure and fast to run Because the scheduler information is stored in the memory allocated to the JVM, all scheduling information will be lost when the application stops running. In addition, because it is stored in the JVM memory, how many jobs and triggers can be stored will be limited
JDBC job store Cluster is supported, because all task information will be saved in the database, and things can be controlled. In addition, if the application server is shut down or restarted, task information will not be lost, and tasks that fail due to server shutdown or restart can be recovered The speed of running depends on the speed of connecting to the database

In fact, there is a solution that can realize the advantages of both of these two ways 666. Eggs Middle.

In addition, the examples and "3. Quick start Quartz stand alone" Basically the same. 😈 Now, let's start to travel~

4.1 introduce dependency

stay pom.xml File, introduce related dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.10.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>lab-28-task-quartz-jdbc</artifactId>

    <dependencies>
        <!-- Automatic configuration of database connection pool -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency> <!-- In this example, we use MySQL -->
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.48</version>
        </dependency>

        <!-- Realize the right Spring MVC Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- Realize the right Quartz Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-quartz</artifactId>
        </dependency>

        <!-- Easy to write unit tests later -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>
  • and 「 3.1 introduce dependency 」 Basically the same, except for the introduction of spring boot starter test dependency, two unit test methods will be written later.

4.2 example Job

stay cn.iocoder.springboot.lab28.task.config.job Under package path, create DemoJob01 and DemoJob02 Class. The code is as follows:

// DemoJob01.java

@DisallowConcurrentExecution
public class DemoJob01 extends QuartzJobBean {

    private Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private DemoService demoService;

    @Override
    protected void executeInternal(JobExecutionContext context) {
        logger.info("[executeInternal][I started, demoService by ({})]", demoService);
    }

}

// DemoJob02.java

@DisallowConcurrentExecution
public class DemoJob02 extends QuartzJobBean {

    private Logger logger = LoggerFactory.getLogger(getClass());

    @Override
    protected void executeInternal(JobExecutionContext context) {
        logger.info("[executeInternal][I started]");
    }

}

Notice, not with Quartz Job Is the dimension, which ensures that in multiple JVM processes, there is only one node executing, but with JobDetail as the dimension. Although, in most cases, we will ensure that a Job and Job detail are one-to-one correspondence. 😈 So, fat friends who don't understand the concept, it's better to understand the concept. It's a bit silly. It's right to make sure that a Job corresponds to a Job detail one by one.

The only identification of JobDetail is JobKey , using the name + group attributes. In general, we only need to set name, and Quartz will default group = DEFAULT.

However, there is one more thing to add here. It is also important to note that in Quartz, nodes with the same Scheduler name form a Quartz cluster. In the following, we can spring.quartz.scheduler-name configuration item, set the name of the Scheduler.

[important] Why do you say that? Because we need to improve the above statement: by adding @ disallowcurrentexecution annotation to the Job implementation class, we can implement the JobDetail of the same JobKey in the same Quartz Scheduler cluster to ensure that there is only one node executing in multiple JVM processes.

4.3 application profile

stay application.yml In, add the configuration of Quartz as follows:

spring:
  datasource:
    user:
      url: jdbc:mysql://127.0.0.1:3306/lab-28-quartz-jdbc-user?useSSL=false&useUnicode=true&characterEncoding=UTF-8
      driver-class-name: com.mysql.jdbc.Driver
      username: root
      password:
    quartz:
      url: jdbc:mysql://127.0.0.1:3306/lab-28-quartz-jdbc-quartz?useSSL=false&useUnicode=true&characterEncoding=UTF-8
      driver-class-name: com.mysql.jdbc.Driver
      username: root
      password:

  # Quartz configuration, corresponding to the QuartzProperties configuration class
  quartz:
    scheduler-name: clusteredScheduler # Scheduler name. The default is schedulerName
    job-store-type: jdbc # Job storage type. The default value is memory, and optional jdbc uses database.
    auto-startup: true # Whether Quartz starts automatically
    startup-delay: 0 # Delay N seconds to start
    wait-for-jobs-to-complete-on-shutdown: true # Whether to wait for the scheduled task to complete when the application is closed. The default value is false, and it is recommended to set it to true
    overwrite-existing-jobs: false # Whether to overwrite the configuration of an existing Job
    properties: # Add additional properties of Quartz Scheduler. For more information, see http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/configuration.html  file
      org:
        quartz:
          # JobStore related configuration
          jobStore:
            # Data source name
            dataSource: quartzDataSource # Data sources used
            class: org.quartz.impl.jdbcjobstore.JobStoreTX # JobStore implementation class
            driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
            tablePrefix: QRTZ_ # Quartz table prefix
            isClustered: true # Cluster mode
            clusterCheckinInterval: 1000
            useProperties: false
          # Thread pool related configuration
          threadPool:
            threadCount: 25 # Thread pool size. The default is 10.
            threadPriority: 5 # thread priority 
            class: org.quartz.simpl.SimpleThreadPool # Thread pool type
    jdbc: # When using JDBC's JobStore, JDBC configuration
      initialize-schema: never # Whether to automatically initialize the Quartz table structure with SQL. This is set to never, and we create the table structure manually.
  • There are many configuration items. We mainly compare them 「 3.5 application profile 」 Come and have a look.
  • At spring.datasource Under configuration, the configuration used to create multiple data sources.

    • User configuration, connect to lab-28-quartz-jdbc-user library. The purpose is to simulate the business database used in our general project.
    • Quartz configuration, connect lab-28-quartz-jdbc-quartz library. The goal is that quartz will use a separate database. 😈 If we have multiple projects that need to use the quartz database, we can use one uniformly, but pay attention to the configuration spring.quartz.scheduler-name sets different Scheduler names to form different quartz clusters.
  • At spring.quartz Under configuration item, some additional configuration items are added. Let's take a look at them one by one.

    • Scheduler name configuration, scheduler name. We have explained this many times. If you don't understand it, please pat yourself to death.
    • Job store type configuration, which sets the job store using "jdbc".
    • properties.org.quartz.jobStore configuration, added JobStore related configuration. It is important to set the datasource named "quartzDataSource" as the data source through the datasource configuration item. 😈 stay 「4.4 DataSourceConfiguration」 We will use spring.datasource.quartz Configuration to create the data source.
    • jdbc configuration item, although it is called this, is mainly used to set up the SQL initialization Quartz table structure. Here, we set initialize schema = never, and we create the table structure manually.

There are a lot of configuration items. If you don't understand the fat friend for the time being, you can simply spring.datasource Modify the data source to your own.

4.4 initializing the Quartz table structure

stay Quartz Download Address, download the release package of the corresponding version. After decompression, we can see the initialization script of the Quartz table structure of various databases in the Src / org / Quartz / impl / JDBC jobstore / directory. Here, because we use MySQL, we use tables_mysql_innodb.sql script.

Execute the script in the database to finish initializing the Quartz table structure. As shown in the figure below:

For a description of the structure of each Quartz table, see Quartz framework (2) - details of table fields in JobStore database article. 😈 In fact, I don't want to watch it. Hahahaha.

We will find that each table has a SCHED_NAME field, the name of the Quartz Scheduler. In this way, the data level of each Quartz cluster can be split.

4.5 DataSourceConfiguration

stay cn.iocoder.springboot.lab28.task.config Under package path, create DataSourceConfiguration Class to configure the data source. The code is as follows:

// DataSourceConfiguration.java

@Configuration
public class DataSourceConfiguration {

    /**
     * Create configuration object for user data source
     */
    @Primary
    @Bean(name = "userDataSourceProperties")
    @ConfigurationProperties(prefix = "spring.datasource.user") // Read spring.datasource.user  Configure to DataSourceProperties object
    public DataSourceProperties userDataSourceProperties() {
        return new DataSourceProperties();
    }

    /**
     * Create user data source
     */
    @Primary
    @Bean(name = "userDataSource")
    @ConfigurationProperties(prefix = "spring.datasource.user.hikari") // Read spring.datasource.user  Configure to HikariDataSource object
    public DataSource userDataSource() {
        // Get DataSourceProperties object
        DataSourceProperties properties =  this.userDataSourceProperties();
        // Create HikariDataSource object
        return createHikariDataSource(properties);
    }

    /**
     * Creating a configuration object for a quartz data source
     */
    @Bean(name = "quartzDataSourceProperties")
    @ConfigurationProperties(prefix = "spring.datasource.quartz") // Read spring.datasource.quartz  Configure to DataSourceProperties object
    public DataSourceProperties quartzDataSourceProperties() {
        return new DataSourceProperties();
    }

    /**
     * Create a quartz data source
     */
    @Bean(name = "quartzDataSource")
    @ConfigurationProperties(prefix = "spring.datasource.quartz.hikari")
    @QuartzDataSource
    public DataSource quartzDataSource() {
        // Get DataSourceProperties object
        DataSourceProperties properties =  this.quartzDataSourceProperties();
        // Create HikariDataSource object
        return createHikariDataSource(properties);
    }

    private static HikariDataSource createHikariDataSource(DataSourceProperties properties) {
        // Create HikariDataSource object
        HikariDataSource dataSource = properties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
        // Set thread pool name
        if (StringUtils.hasText(properties.getName())) {
            dataSource.setPoolName(properties.getName());
        }
        return dataSource;
    }

}
  • be based on spring.datasource.user Configuration item, created DataSource Bean named "userDataSource". In addition, we added @ primary annotation on it to indicate that it is the main data source.
  • be based on spring.datasource.quartz Configuration item. A DataSource Bean named "quartzDataSource" is created. And, on it, we added @QuartzDataSource Annotation indicating that it is the data source of Quartz. 😈 Attention, must be configured ah, here the grandma card for a long time!!!!

4.6 timing task configuration

After completing the above work, we need to configure the timing task of Quartz. At present, there are two ways:

4.6.1 Bean automatic setting

stay cn.iocoder.springboot.lab28.task.config Under package path, create ScheduleConfiguration Class to configure the two sample jobs above. The code is as follows:

// ScheduleConfiguration.java

@Configuration
public class ScheduleConfiguration {

    public static class DemoJob01Configuration {

        @Bean
        public JobDetail demoJob01() {
            return JobBuilder.newJob(DemoJob01.class)
                    .withIdentity("demoJob01") // The name is demoJob01
                    .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                    .build();
        }

        @Bean
        public Trigger demoJob01Trigger() {
            // A simple constructor for scheduling
            SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
                    .withIntervalInSeconds(5) // Frequency.
                    .repeatForever(); // Number of times.
            // Trigger constructor
            return TriggerBuilder.newTrigger()
                    .forJob(demoJob01()) // The corresponding Job is demoJob01
                    .withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
                    .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                    .build();
        }

    }

    public static class DemoJob02Configuration {

        @Bean
        public JobDetail demoJob02() {
            return JobBuilder.newJob(DemoJob02.class)
                    .withIdentity("demoJob02") // The name is demoJob02
                    .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                    .build();
        }

        @Bean
        public Trigger demoJob02Trigger() {
            // A simple constructor for scheduling
            CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * * ? *");
            // Trigger constructor
            return TriggerBuilder.newTrigger()
                    .forJob(demoJob02()) // The corresponding Job is demoJob02
                    .withIdentity("demoJob02Trigger") // The name is demoJob02Trigger
                    .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                    .build();
        }

    }

}

When the Quartz scheduler is started, the following methods will be automatically called according to the configuration:

  • The scheduler ා addjob (JobDetail JobDetail, Boolean replace) method persists the JobDetail to the database.
  • Schedulejob (Trigger trigger) method to persist the Trigger to the database.

4.6.2 Scheduler manual setting

In general, it is recommended that you use the Scheduler to set it manually.

establish QuartzSchedulerTest Class to create a Quartz timer task configuration that adds DemoJob01 and DemoJob02, respectively. The code is as follows:

// QuartzSchedulerTest.java

@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class QuartzSchedulerTest {

    @Autowired
    private Scheduler scheduler;

    @Test
    public void addDemoJob01Config() throws SchedulerException {
        // Create JobDetail
        JobDetail jobDetail = JobBuilder.newJob(DemoJob01.class)
                .withIdentity("demoJob01") // The name is demoJob01
                .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                .build();
        // Create Trigger
        SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
                .withIntervalInSeconds(5) // Frequency.
                .repeatForever(); // Number of times.
        Trigger trigger = TriggerBuilder.newTrigger()
                .forJob(jobDetail) // The corresponding Job is demoJob01
                .withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
                .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                .build();
        // Add schedule task
        scheduler.scheduleJob(jobDetail, trigger);
//        scheduler.scheduleJob(jobDetail, Sets.newSet(trigger), true);
    }

    @Test
    public void addDemoJob02Config() throws SchedulerException {
        // Create JobDetail
        JobDetail jobDetail = JobBuilder.newJob(DemoJob02.class)
                .withIdentity("demoJob02") // The name is demoJob02
                .storeDurably() // Whether the task is reserved when there is no Trigger Association. Because no Trigger points to JobDetail when it is created, it needs to be set to true to indicate reservation.
                .build();
        // Create Trigger
        CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * * ? *");
        Trigger trigger = TriggerBuilder.newTrigger()
                .forJob(jobDetail) // The corresponding Job is demoJob01
                .withIdentity("demoJob02Trigger") // The name is demoJob01Trigger
                .withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
                .build();
        // Add schedule task
        scheduler.scheduleJob(jobDetail, trigger);
//        scheduler.scheduleJob(jobDetail, Sets.newSet(trigger), true);
    }

}
  • The code to create JobDetail and Trigger is actually the same as 「 4.6.1 Bean automatic setting 」 Is consistent.
  • At the end of each unit test method, the scheduler schedulejob (JobDetail JobDetail, Trigger trigger Trigger) method is called to persist the JobDetail and Trigger to the database.
  • If you want to overwrite the configuration of the Quartz timing task in the database, you can call the scheduler ා schedulejob (JobDetail JobDetail, set <? Extends trigger > triggers for job, Boolean replace) method, and pass in replace = true to overwrite the configuration.

4.7 Application

establish Application.java Class, configure @ SpringBootApplication annotation. The code is as follows:

// Application.java

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}
  • Run the Application class to start the sample project. Specific execution logs, and 「3.4 Application」 It's basically the same. Grandma won't list it again.

If fat friends want to test the running status of the cluster, they can create another one Application02.java Class, configure @ SpringBootApplication annotation. The code is as follows:

// Application02.java

@SpringBootApplication
public class Application02 {

    public static void main(String[] args) {
        // Set Tomcat random port
        System.setProperty("server.port", "0");

        // Start the Spring Boot application
        SpringApplication.run(Application.class, args);
    }

}
  • Run the Application02 class and start a sample project again. Then, observe the output log. You can see that the two sample projects started will have the execution logs of DemoJob01 and DemoJob02.

5. Quick start XXL job

Example code corresponding warehouse: lab-28-task-xxl-job .

Although the functions of Quartz can meet our demands for timed tasks, there is still a certain distance from the availability and ease of production. At the beginning of the internship, because Quartz only provides the function of task scheduling, and does not provide the management and monitoring console of management tasks, we need to do the second encapsulation by ourselves. At that time, because there was no suitable open source project in the community to realize this function, we simply encapsulated ourselves to meet our management and monitoring needs.

But now, there are many excellent scheduling Middleware in the open source community. Among them, the representative ones are XXL-JOB . It defines itself as follows:

XXL-JOB is a lightweight distributed task scheduling platform. Its core design goal is to develop quickly, learn simply, lightweight and easy to expand.

For getting started with XXL job, we have "Introduction to XXL job in taro road" Write in, fat friends jump to read this article first. The key point is to build a XXL job dispatching center first. 😈 Because, in this article, we are going to implement a XXL job executor in the Spring Boot project.

5.1 introduce dependency

stay pom.xml File, introduce related dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>lab-28-task-xxl-job</artifactId>

    <dependencies>
        <!-- Realize the right Spring MVC Automatic configuration of -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- XXL-JOB Related dependence -->
        <dependency>
            <groupId>com.xuxueli</groupId>
            <artifactId>xxl-job-core</artifactId>
            <version>2.1.1</version>
        </dependency>
    </dependencies>

</project>

For the specific role of each dependency, fat friend takes a look at all the comments added by grandma. Unfortunately, at present, XXL-JOB does not provide the Spring Boot Starter package officially, which is a bit awkward. However, the community has already submitted a Pull Request, which can be seen in detail https://github.com/xuxueli/xx... .

5.2 application profile

stay application.yml In, add the configuration of Quartz as follows:

server:
  port: 9090 #Specify a port to avoid port conflicts with XXL-JOB scheduling center. For testing purposes only

# xxl-job
xxl:
  job:
    admin:
      addresses: http://127.0.0.1:8080 / XXL job admin - dispatch center deployment and address [optional]: if there are multiple addresses in the dispatch center cluster deployment, they are separated by commas. The actuator will use this address for "actuator heartbeat registration" and "task result callback"; if it is blank, the automatic registration will be turned off;
    executor:
      appname: lab-28-executor # Actuator AppName [optional]: actuator heartbeat registration grouping basis; if it is blank, turn off automatic registration
      ip: # Actuator IP [optional]: if it is blank by default, the IP will be automatically obtained. When multiple network cards are used, the specified IP can be manually set. The IP will not bind the Host only as a communication utility; the address information is used for "actuator registration" and "dispatch center requests and triggers tasks";
      port: 6666 # ### Actuator port number [optional]: automatically obtained when it is less than or equal to 0; the default port is 9999. When deploying multiple actuators on a single machine, pay attention to configuring different actuator ports;
      logpath: /Users/yunai/logs/xxl-job/lab-28-executor # Path to the disk where the running log file of the actuator is stored [optional]: you need to have read and write permission to the path; if it is blank, the default path will be used;
      logretentiondays: 30 # Actuator log file saving days [optional]: expired logs will be automatically cleaned up when the limit value is greater than or equal to 3; otherwise, for example - 1, turn off the automatic cleaning function;
    accessToken: yudaoyuanma # Actuator communication TOKEN: enabled when not empty;
  • For the specific function of each parameter, fat friend has a look at the detailed notes.

5.3 XxlJobConfiguration

stay cn.iocoder.springboot.lab28.task.config Under package path, create DataSourceConfiguration Class to configure the XXL-JOB actuator. The code is as follows:

// XxlJobConfiguration.java

@Configuration
public class XxlJobConfiguration {

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;
    @Value("${xxl.job.executor.appname}")
    private String appName;
    @Value("${xxl.job.executor.ip}")
    private String ip;
    @Value("${xxl.job.executor.port}")
    private int port;
    @Value("${xxl.job.accessToken}")
    private String accessToken;
    @Value("${xxl.job.executor.logpath}")
    private String logPath;
    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;

    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        // Create the XxlJobSpringExecutor executor
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppName(appName);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
        // return
        return xxlJobSpringExecutor;
    }

}
  • In the ා xxlJobExecutor() method, the XXL job executor Bean object under the Spring container is created. Note that the @ Bean annotation added to the method configures the start and destroy methods.

5.4 DemoJob

stay cn.iocoder.springboot.lab28.task.job Under package path, create DemoJob Class, sample scheduled task class. The code is as follows:

// DemoJob.java

@Component
@JobHandler("demoJob")
public class DemoJob extends IJobHandler {

    private Logger logger = LoggerFactory.getLogger(getClass());

    private final AtomicInteger counts = new AtomicInteger();

    @Override
    public ReturnT<String> execute(String param) throws Exception {
        // Print log
        logger.info("[execute][Timing ({}) Secondary execution]", counts.incrementAndGet());
        // Return to successful execution
        return ReturnT.SUCCESS;
    }

}
  • Inherit XXL job IJobHandler Abstract class, through the implementation of ා execute(String param) method, to achieve the logic of timed tasks.
  • On the method, add @JobHandler Annotation to set the name of the JobHandler. Later, we need to use this name when adding tasks in the console of the dispatch center.

#The return result of the execute(String param) method is ReturnT Type. When the return value matches“ ReturnT.code == ReturnT.SUCCESS_CODE "indicates that the task is successfully executed, otherwise, it indicates that the task fails to execute, and you can use the“ ReturnT.msg ”Call back the error information to the scheduling center, so that the task execution results can be easily controlled in the task logic.

#The method parameter of the execute(String param) method is the "task parameter" configured in the console of the dispatch center when a task is added. Generally, it will not be used.

5.5 Application

establish Application.java Class, configure @ SpringBootApplication annotation. The code is as follows:

// Application.java

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

Run the Application class to start the sample project. The output log is simplified as follows:

# XXL job startup log
2019-11-29 00:58:42.429  INFO 46957 --- [           main] c.xxl.job.core.executor.XxlJobExecutor   : >>>>>>>>>>> xxl-job register jobhandler success, name:demoJob, jobHandler:cn.iocoder.springboot.lab28.task.job.DemoJob@3af9aa66
2019-11-29 00:58:42.451  INFO 46957 --- [           main] c.x.r.r.provider.XxlRpcProviderFactory   : >>>>>>>>>>> xxl-rpc, provider factory add service success. serviceKey = com.xxl.job.core.biz.ExecutorBiz, serviceBean = class com.xxl.job.core.biz.impl.ExecutorBizImpl
2019-11-29 00:58:42.454  INFO 46957 --- [           main] c.x.r.r.provider.XxlRpcProviderFactory   : >>>>>>>>>>> xxl-rpc, provider factory add service success. serviceKey = com.xxl.job.core.biz.ExecutorBiz, serviceBean = class com.xxl.job.core.biz.impl.ExecutorBizImpl
2019-11-29 00:58:42.565  INFO 46957 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-11-29 00:58:42.629  INFO 46957 --- [       Thread-7] com.xxl.rpc.remoting.net.Server          : >>>>>>>>>>> xxl-rpc remoting server start success, nettype = com.xxl.rpc.remoting.net.impl.netty_http.server.NettyHttpServer, port = 6666

At this time, DemoJob will not execute because we have not configured it in the XXL job scheduling center. Next, let's configure it in the XXL-JOB dispatch center.

5.6 add actuator

Browser open http://127.0.0.1:8080/xxl-job... Address, which is the "actuator management" menu. As shown below:

Click the "add actuator" button to open the "add actuator" interface. As shown below:

Fill in the information of "lab-28-executor", click "save" to save. Wait patiently for a while, and the actuator will automatically register. As shown below:

  • The OnLine actuator list is displayed in the actuator list. You can view the cluster machine of the corresponding actuator through "OnLine machine".

Same actuator, only need to be configured once.

5.7 new task

Browser open http://127.0.0.1:8080/xxl-job... Address, which is the task management menu. As shown below:

Click the "add" button on the far right to open the "add" interface. As shown below:

Fill in the information of "demoJob" task, click "save" to save. As shown below:

Click the "operation" button of the "demoJob" task, and select "start". After confirmation, the status of the "demoJob" task will become RUNNING. As shown below:

At this point, we open the IDE interface of the actuator, and you can see that DemoJob has been executed every minute. The logs are as follows:

2019-11-29 01:30:00.161  INFO 48374 --- [      Thread-18] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (1) Secondary execution]
2019-11-29 01:31:00.012  INFO 48374 --- [      Thread-18] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (2) Secondary execution]
2019-11-29 01:32:00.009  INFO 48374 --- [      Thread-18] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (3) Secondary execution]
2019-11-29 01:33:00.010  INFO 48374 --- [      Thread-18] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (4) Secondary execution]
2019-11-29 01:34:00.005  INFO 48374 --- [      Thread-18] c.i.springboot.lab28.task.job.DemoJob    : [execute][Timing (5) Secondary execution]

In addition, on the dispatch center interface, click the "operation" button of the "demoJob" task and select "query log" to see the corresponding dispatch log. As shown below:

At this point, we have finished getting started with the XXL-JOB actuator.

6. Getting started with elastic job

Maybe many fat friends don't know about elastic job middleware. Let's take a look at an introduction to its official documents:

Elastic job is a distributed scheduling solution, which consists of two independent subprojects, elastic job lite and elastic job cloud.

Elastic job Lite is positioned as a lightweight and decentralized solution, which provides coordination services of distributed tasks in the form of jar packages.

Elastic job is basically one of the best middleware for scheduling tasks open-source in China. It may not be one of them. At present, it is in a state of "breaking and changing", which can be seen in detail https://github.com/elasticjob... .

So for this example, we will not provide it for the time being. If you are interested in elastic job source code, you can see the following two series written by Grandma:

666. Eggs

① How to choose?

Maybe chubby would like to know the comparison of different scheduling middleware. The table is as follows:

characteristic quartz elastic-job-lite xxl-job LTS
rely on MySQL,jdk jdk,zookeeper mysql,jdk jdk,zookeeper,maven
High availability In multi node deployment, only one node can perform tasks by competing database locks Through the registration and discovery of zookeeper, the server can be added dynamically Based on the competitive database lock, only one node can perform the task, which supports the horizontal expansion. You can manually add timed tasks, start and pause tasks, and monitor them Cluster deployment can add servers dynamically. You can manually add scheduled tasks, start and pause tasks. With monitoring
Task segmentation ×
Management interface ×
Difficulty degree simple simple simple Slightly complex
Advanced features - Elastic capacity expansion, multiple job modes, failover, running state collection, multithreading data processing, idempotence, fault tolerance processing, spring namespace support Elastic capacity expansion, fragment broadcasting, failover, Rolling real-time log, GLUE (support online code editing, no release), task progress monitoring, task dependency, data encryption, email alarm, operation report, internationalization Support spring, spring boot, business loggers, SPI extension support, failover, node monitoring, diversified task execution result support, FailStore fault tolerance, dynamic capacity expansion.
Version update Not updated in half a year Not updated in 2 years Recent updates Not updated in 1 year

The following articles are also recommended:

At present, if you really don't know how to choose, you can try it first XXL-JOB .

② Centralization V.S decentralization

Next, let's talk about the implementation of distributed scheduling middleware. A distributed scheduling middleware has two roles:

  • Scheduler: responsible for scheduling tasks and sending them to the executor.
  • Actuator: responsible for receiving tasks and performing specific tasks.

Then, from the perspective of scheduling system, it can be divided into two categories:

  • Centralization: the scheduling center is separated from the actuator. The scheduling center schedules uniformly and informs an actuator to process tasks.
  • Decentralization: the dispatching center and the actuator are integrated to schedule their own processing tasks.

Therefore, XXL job belongs to the centralized task scheduling platform. At present, this kind of program is also adopted:

  • Catenary kob
  • Crane of meituan

The decentralized task scheduling platform currently includes:

Grandma: if fat friends want to understand better, you can see what grandma wrote Centralized V.S decentralized scheduling design

③ Task competition V.S task pre allocation

Then, from the perspective of task allocation, it can be divided into two categories:

  • Task competition: the scheduler will send tasks to the executor through competing tasks.
  • Task pre allocation: the scheduler assigns tasks to different executors in advance without competition.

Therefore, XXL job belongs to the task scheduling platform of task competition. At present, this kind of program is also adopted:

  • Catenary kob
  • Crane of meituan
  • Quartz Database based clustering scheme

The task scheduling platform for task pre allocation currently includes:

Generally speaking, the task scheduling platform based on task pre allocation will choose to use Zookeeper to coordinate the allocation of tasks to different nodes. At the same time, the task scheduling platform must be a decentralized scheme, each node is both a scheduler and an executor. In this way, after the task is preassigned in each node, it will be scheduled to execute by itself.

In comparison, with more and more nodes, the performance of the scheme based on task competition will decline because of task competition. However, this problem does not exist in the scheme based on task pre allocation. Moreover, the performance of the scheme based on task pre allocation is better than that based on task competition.

Here is an article about Zhang Liang, the developer of Elastic Job Detailed explanation of the distributed job framework elastic job of Dangdang network Awesome!

④ Quartz is an excellent scheduling kernel

In most cases, we will not directly use Quartz as our choice of scheduling middleware. However, all of the distributed scheduling middleware use Quartz as the scheduling kernel, because Quartz provides a strong function in task scheduling itself.

However, with the gradual improvement of a distributed scheduling middleware, it will gradually consider abandoning Quartz as the scheduling kernel and turning to self-development. For example, in the version of 2.1.0 RELEASE, XXL-JOB has been replaced with a self-developed scheduling module. The reasons for its replacement are as follows:

XXL job finally chooses self-developed scheduling component (early scheduling component is based on Quartz);

  • On the one hand, it is to simplify the system and reduce redundant dependence.
  • On the other hand, it is to provide the controllability and stability of the system.

In the development plan of elastic job 3. X, there is also a plan, i.e. self-developed scheduling component to replace Quartz.

Recommended reading

Tags: Java Spring SpringBoot Database

Posted on Tue, 09 Jun 2020 01:27:40 -0400 by aftabn10