springcloud: Quick start timing task framework xxl-job (15)

0 Preface

In actual development, we often encounter tasks that need to be executed regularly. We can use the timing thread pool or the schedule framework to implement timing tasks, but these methods have defects in efficiency and performance. Under the microservice framework, we expect a A more regular, lightweight, and reliable timing task framework to help us implement timing tasks and visually manage timing tasks.

Under such demands, the domestic timing task framework xxl-jobcame into being.

1. Introduction to xxl-job

xxl-job is a distributed task scheduling platform, named after the author, because of its light weight, visibility, and ease of use, it quickly gained a foothold under the microservice framework. xxl-job is divided into server and client. The client is the implementation of our scheduled task method, also known as the executor, while the server is used to manage the configuration of scheduled tasks and record execution, also known as the scheduler.

Official document: https://www.xuxueli.com/xxl-job/

2. xxl-job installation

1. In the xxl-job github warehouse](https://github.com/xuxueli/xxl-job/releases), download the source code, I directly choose the latest version to download here

2. Open the project in the form of maven in idea, which is the xxl-job-adminserver we want to install. You can see that the server is actually a springboot project, which xxl-job-coreis a public dependency and xxl-job-executor-samplesa sample code for the client

tables_xxl_job.sql3. Find the file under the doc folder

Use this file to create a databasexxl-job

4. In xxl-job-adminthe application.properties configuration file under the resource folder in the project, modify its database address, service port number, database connection pool, warning sending mailbox and other information

### web 端口
server.port=8080
server.servlet.context-path=/xxl-job-admin

### actuator
management.server.servlet.context-path=/actuator
management.health.mail.enabled=false

### resources
spring.mvc.servlet.load-on-startup=0
spring.mvc.static-path-pattern=/static/**
spring.resources.static-locations=classpath:/static/

### freemarker
spring.freemarker.templateLoaderPath=classpath:/templates/
spring.freemarker.suffix=.ftl
spring.freemarker.charset=UTF-8
spring.freemarker.request-context-attribute=request
spring.freemarker.settings.number_format=0.##########

### mybatis
mybatis.mapper-locations=classpath:/mybatis-mapper/*Mapper.xml
#mybatis.type-aliases-package=com.xxl.job.admin.core.model

### xxl-job, datasource 数据库
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=123456
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

### datasource-pool 连接池
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.hikari.minimum-idle=10
spring.datasource.hikari.maximum-pool-size=30
spring.datasource.hikari.auto-commit=true
spring.datasource.hikari.idle-timeout=30000
spring.datasource.hikari.pool-name=HikariCP
spring.datasource.hikari.max-lifetime=900000
spring.datasource.hikari.connection-timeout=10000
spring.datasource.hikari.connection-test-query=SELECT 1
spring.datasource.hikari.validation-timeout=1000

### xxl-job, email 预警发送邮箱
spring.mail.host=smtp.qq.com
spring.mail.port=25
[email protected]
[email protected]
spring.mail.password=xxx
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.properties.mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory

### xxl-job, access token 
xxl.job.accessToken=default_token

### xxl-job, i18n (default is zh_CN, and you can choose "zh_CN", "zh_TC" and "en")
xxl.job.i18n=zh_CN

## xxl-job, triggerpool max size
xxl.job.triggerpool.fast.max=200
xxl.job.triggerpool.slow.max=100

### xxl-job, log retention days
xxl.job.logretentiondays=30

5. logback.xmlIt is a log configuration file. If you want to modify the log storage path, you can configure it here. If there is no configured log folder when starting the project, an error will be reported, so modify the log path in advance

6. If you start the server locally, you only need to run the xxl-job project. If you want to publish the server to the server, package it, and then run it on the server

In order to take care of some beginners, here is a detailed explanation of the packaging and publishing process. If you have mastered it, you can skip it directly.

2.1 Publish xxl-job-admin to the server

1. Modify the configuration file database address

2. Modify the email for sending the warning. If you don’t know how to get the email authorization code, you can refer to my article: https://wu55555.blog.csdn.net/article/details/127156042

No need for warning or configuration

3. Modify the log path

4. Packingxxl-job-admin

5. Upload the packaged jar package to the target server (because it is a java project, please install jdk on the server in advance)

6. Create a log folder

mkdir -p /data/logs/xxl-job-2.4.0/log

7. Because we configured the default port 8080, port 8080 needs to be opened. If the firewall is not opened, there is no need to open the port

firewall-cmd --add-port=8080/tcp --permanent
firewall-cmd --reload

8. Startxxl-job-admin

# 前台启动，关闭窗口或者ctrl+c会关闭项目
java -jar xxl-job-admin-2.4.0.jar
# 后台启动，关闭窗口或者ctrl+c不会关闭项目
nohup java -jar xxl-job-admin-2.4.0.jar & 

9. Access project: server ip: 8080/xxl-job-admin , default login account password: admin/123456

3. Client scheduled task writing

1. Here we create a springboot project to write timing tasks

2. Introduce dependencies, and keep the version consistent with the server version

<dependency>
            <groupId>com.xuxueli</groupId>
            <artifactId>xxl-job-core</artifactId>
            <version>2.4.0</version>
</dependency>

3. Modify the configuration file

# 服务端地址
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### xxl-job, access token，与服务端配置的保持一致
xxl.job.accessToken=default_token
### xxl-job executor appname
xxl.job.executor.appname=job-server
# 执行器注册地址，默认使用address，否则使用ip:port，这是注册到服务端的地址
xxl.job.executor.address=
# 执行器ip
xxl.job.executor.ip=
# 执行器端口
xxl.job.executor.port=9998
### xxl-job executor log-path
xxl.job.executor.logpath=/data/logs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30

4. Create a configuration classXxlJobConfig

@Configuration
public class XxlJobConfig {
    
    
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${xxl.job.accessToken}")
    private String accessToken;

    @Value("${xxl.job.executor.appname}")
    private String appname;

    @Value("${xxl.job.executor.address}")
    private String address;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private int port;

    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;


    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
    
    
        logger.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

        return xxlJobSpringExecutor;
    }

    /**
     * 针对多网卡、容器内部署等情况，可借助 "spring-cloud-commons" 提供的 "InetUtils" 组件灵活定制注册IP；
     *
     *      1、引入依赖：
     *          <dependency>
     *             <groupId>org.springframework.cloud</groupId>
     *             <artifactId>spring-cloud-commons</artifactId>
     *             <version>${version}</version>
     *         </dependency>
     *
     *      2、配置文件，或者容器启动变量
     *          spring.cloud.inetutils.preferred-networks: 'xxx.xxx.xxx.'
     *
     *      3、获取IP
     *          String ip_ = inetUtils.findFirstNonLoopbackHostInfo().getIpAddress();
     */


}

5. We create a timing method. We can see that we only need to add an annotation to the method that needs to be executed regularly. @XxlJobThe attribute value is the name of the timing method, which will be configured on the server later, and must be unique

At the same time, note that the scheduled task class should be declared as a bean, that is, add@Component

@Component
public class JobService {
    
    

    @XxlJob("testJob")
    public void testJob() throws Exception {
    
    
        String param = XxlJobHelper.getJobParam();
     
        System.out.println("执行定时任务，参数为："+param);

    }
}

4. Server configuration

The xxl-job-admin server has the following menus

Operation report: statistics on the running status of scheduled tasks
Task management: configured scheduled tasks
Scheduling log: specific execution log of each scheduled task
User management: account management, which can configure permissions for executors
Executor management: executors are clients, that is, Background timing task method

1. First, we need to add an executor. Here we choose automatic registration. Sometimes we need to perform joint debugging and need to specify the executor as our local executor. At this time, we can use the registration form of manually entering the ip, and the address is what we configured before. or + xxl.job.executor.address_xxl.job.executor.ipxxl.job.executor.port

Note: The appName here should xxl.job.executor.appnamebe consistent with that configured in the client

Manual entry form:

2. Start the executor project, that is, the job-server project created above

3. Enter the task management, switch the executor to the newly created executor, and then create a new task

4. Configure the task name, cron expression (here I configure it to execute once per second), JobHandler is the name of the scheduled task we wrote on the client before, that is, the @XxlJobvalue of the annotation

The configuration descriptions are as follows:

• Scheduling type:

Supports 3 scheduling modes: corn expression
cron: free configuration timing interval; none: manual trigger execution; fixed: fixed cycle execution, such as once every 30s

• operating mode

The default is BEAN. From the executor scheduling method, you can also configure the GLUE mode. After configuration, a GULE button will appear on the list page. Click to enter the online writing code of the corresponding language, which is equivalent to an online writing executor method, that is, the customer End tasks are maintained in the dispatch center in the form of source code, supporting languages ​​such as java, shell, and python

• Task parameters

You can set fixed task parameters, or leave them blank, or fill in the task parameters during manual execution, such as some timed tasks that are executed according to the date. If you want to go back to execute the historical date, you can set the parameter as the date. Default is today

• routing policy

When there are multiple executors, it is necessary to configure a routing strategy, that is, a load balancing strategy, and the more common one is polling

• Subtask ID

If you need to trigger the next task after the execution of the current task is completed, you can configure subtasks, and the ID can be viewed in the list

• Scheduling Expiration Policy

Two strategies are supported. We will explain why there is an expiration.
Ignore: After the schedule expires, ignore the expired task, and recalculate the next trigger time from the current time;
Execute once immediately: After the schedule expires, execute it once immediately. And recalculate the next trigger time from the current time;

• blocking handling strategy

When there are too many tasks and too many intensive tasks, the strategy adopted by the executor supports the following three types of
stand-alone serial (default): enter the waiting queue, first in first out, waiting for execution;
discard follow-up scheduling: discard, don’t care
Override previous scheduling: Terminate the running scheduled task, clear the waiting queue, and then run the current task

• task timeout

If the task runs beyond the timeout period, it will be directly interrupted

• failed retries

Number of times to retry after task failure

For students who have not mastered cron expressions, you can use this website to help us write cron expressions, and you can generate the desired cron expressions by checking

https://cron.qqe2.com/

5. The created task is closed by default, and the task needs to be started. If the task only needs to be executed once, you can choose not to start it, just click it 执行一次manually

6. The status of the started project isrunning

7. We can see that the task is being executed by looking at the job-server

8. You can see that the parameters here are empty. This is because we did not configure parameters when configuring tasks on the server side. We execute them manually once and execute the parameters to show you the effect.

Summarize

At this point, our explanation of the timing task framework xxl-job is over, let’s build it up and try it yourself