Use of APScheduler in python timing framework

Article directory

APScheduler

install

pip install apscheduler

Composition of apscheduler

  1. trigger
    Each job is its own trigger to decide which job to run next. Except for the initial configuration, the trigger is stateless.
  2. Job store
    Store the scheduled jobs. The default job is to simply save the jobs in memory. Other job stores are to save the jobs in the database. A job's data will be saved in the persistent job store is serialized and deserialized at load time. The scheduler cannot share the same job store.
  3. Actuator
    To handle the running of jobs, they usually submit the specific callable objects to a thread or process pool in the jobs. When the jobs are completed, the executor will notify the scheduler.
  4. Scheduler
    It is responsible for connecting the above components. Generally, there is only one scheduler in the application. Our development will not directly operate the trigger, job storage and actuator, but use the scheduler to provide the appropriate interface to handle these. Job storage and actuator are configured in the scheduler

Common schedulers

  • BlockingScheduler: suitable for running a single task only in a process

  • BackgroundScheduler: suitable for situations where you want to execute in the background of a program without running other frameworks

  • Asyncio scheduler: suitable for use with asyncio framework

  • GeventScheduler: suitable for using gevent framework

  • Tornado scheduler: suitable for applications using tornado framework

  • Twisted scheduler: application suitable for twisted framework

  • QtScheduler: suitable for QT

Configure scheduler

The Apscheduler framework provides many configuration methods for schedulers, which can use the configuration dictionary or directly pass configuration parameters to the schedulers. At the same time, it supports initializing the schedulers first, adding job tasks, and then configuring the schedulers.

Here is a small example of using the default scheduler

from apscheduler.schedulers.blocking import BlockingScheduler

def task():
    print('this is a job')

sch = BlockingScheduler()
sch.add_job(task,'interval',seconds=5)
sch.start()

Another complex one, using two job stores and two actuators

from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.jobstores.mongodb import MongoDBJobStore
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
from apscheduler.executors.pool import ThreadPoolExecutor,ProcessPoolExecutor
#Modify default job store
jobstores = {
    'mongo':MongoDBJobStore(),
    'default':SQLAlchemyJobStore(url='sqlite://jobs.sqlite')
}

#Modify actuator
executors = {
    'defaults':ThreadPoolExecutor(10),
    'processpool':ProcessPoolExecutor(10)
}


job_default= {
    'coalesce':False, #If coalesce is True, the next time the job is submit ted to the executor, it will only be executed once, that is, the last time. If it is False, it will be executed five times (not necessarily, because there are other conditions, see the later explanation of migration gracetime)
    'max_instances':3 #Maximum instances of job running
}

sc = BackgroundScheduler(jobstores=jobstores,executors=executors,job_default=job_default)

Start / stop scheduler

Use the start() method to start the scheduler. Note that BlockingScheduler needs to be initialized before start() can be executed. For other schedulers, calling the start() method will return directly, and then the subsequent initialization can be continued. At the same time, after the scheduler is started, its configuration can no longer be changed.

By default, the scheduler will automatically close all schedulers and job stores after all job tasks are completed. If you don't want to wait during use, you can set the wait parameter option to False, which means it is closed directly:

sched.shutdown(wait=False)

event listeners

Event listeners can be added to the scheduler, which can only be triggered in some cases, and can carry some useful information. By passing the appropriate mask parameter to add_listener(), you can only listen to several specific event types. For the specific types, you can see the event.exception or event.code values in the source code for identification.

from apscheduler.events import EVENT_JOB_EXECUTED,EVENT_JOB_ERROR
from apscheduler.schedulers.blocking import BlockingScheduler

def lister(event):    
    if event.code == EVENT_JOB_EXECUTED:
        print('job down')
    else:
        print('error')
        
sched = BlockingScheduler()
sched.add_listener(lister,EVENT_JOB_EXECUTED|EVENT_JOB_ERROR)

Operation and operation storage

The job store provides a unified abstraction of serialized jobs to the scheduler, and provides an interface for adding, deleting, modifying and querying jobs in the scheduler. According to the different storage backend s, it can be divided into the following types:

  • MemoryJobStore: without serialization, jobs exist in memory, and add, delete, modify and query are all in memory
  • SQLAlchemyJobStore: all databases supported by sqlalchemy can be used as backend, and the add, delete, modify and query operations can be converted into the corresponding backend sql statements
  • MongoDBJobStore: mongodb as backend
  • RedisJobStore: use redis as the backend

Job is the framework to undertake the work and tasks that need to be performed at present. We can dynamically add, modify, delete, query and other operations during the operation of the system.

1. Add job

Add ABCD Job() to add a Job, and you can also use the decorator scheduled ABCD Job to dynamically decorate the actual functions of a Job

@sched.scheduled_job('interval',seconds=5)
def task():
    print('t')

2. Removal

#Multiple tasks can use job id. objects can be identified according to job id. remove is only valid before start
job = sched.add_job(task,'interval',seconds=5,id='2')
job.remove()
sched.remove_job(job_id='2')

3. Suspend and resume operation

job.pause() #suspend
job.resume() #recovery

4. Get the list of scheduling jobs

Get the list of scheduled jobs, which can be completed by get_jobs(), which will return all job instances, and print_jobs(), which can also be used to output all formatted job lists. You can also use get job (task ID) to get the job list of a specified task

print(sched.get_jobs())
print(sched.get_job(job_id='2'))

5. Modify operation

Except for id, it can't be modified

sched.reschedule_job('2',trigger='cron',minutes='*/5')
job.modify()

Job control

The second parameter of the add ﹣ job() method is trigger, which manages the scheduling mode of job tasks. It can be set into three categories: data, interval and corn. For different settings, the corresponding parameters are different, as follows:
1.corn scheduling

#Executed at 16:13:00 on February 14, 2020
job = sch.add_job(task,'cron',year=2020,month=2,day=14,hour=16,minute=13,second=0)
sch.start()

# 0, 1, 2, 3 on the third Friday of 2, 3, 4, 5, 10, 11
job = sch.add_job(task,'cron',month='2-5,10-11',day='3rd fri',hour='0-3')
sch.start()

#3:00 a.m. from Monday to Friday until March 30, 2019
job = sch.add_job(task,'cron',day_of_week='mon-fri',hour='3',end_date='2019-03-30')
sch.start()

##parameter
"""
year  year
month  month
day  day
week  Monday-Friday
day_of_week What day of the week-Zhou Ji
hour Time
minutes branch
second second
"""

2.interval scheduling

# interval
#Every 4 days and 3 hours
job = sch.add_job(task,'interval',days=4,hours=3)
sch.start()

#parameter
'''
weeks week
days day
hours Time
minutes branch
seconds second
'''

3.data scheduled scheduling (only once to the time)

from datetime import date
import datetime
#data
# interval
#Every 4 days and 3 hours
job = sch.add_job(task,'date',run_date='2020-02-14 16:38:05',args=["text"])
sch.start()
import apscheduler
#parameter
'''
run_date Date tuple or fill in time directly
'''
Published 11 original articles, won praise 2, visited 244
Private letter follow

Tags: MongoDB SQLite pip Database

Posted on Fri, 14 Feb 2020 04:26:11 -0500 by kenrbnsn