About the author: Hello everyone, I am Brother Smart, a former architect of ZTE and Meituan, and now the CTO of an Internet company.
Contact qq: 184480602, add me to the group, let’s learn together, make progress together, and fight against the cold winter of the Internet together
I personally think that making wheels is almost the best way to learn, if not even one. Because making a wheel requires doing at least the following two things:
- Understand design thinking (design level)
- A brief look at the source code (code level)
If you don’t understand design, you can’t grasp the whole. If you haven't seen the code, you can't perfect the details. In addition, from the creator's perspective, it is sometimes too difficult to directly analyze the source code, there are too many codes, and the level of abstraction is too deep. If we can reinvent the wheel, reduce the level of abstraction, and present it in a straightforward manner, it will be easier for readers to understand.
Since it's so good to make wheels, let's get it. Today I will show you how to create a thread pool.
Design ideas
You can't avoid the vulgarity, just "picture from Baidu":
Pooling technology
Everyone has heard of the so-called "pooling technology", such as database connection pool, constant pool, thread pool, object pool, etc. Pooling technology is a commonly used and effective optimization method in the computer world. So I want to ask you, what does the "pool" in the thread pool refer to?
Putting aside the insignificant little shrimps, there are two most important things in the thread pool: the tasks we submit to the Executor and the Thread maintained by the Executor itself. Among them, the thread "pool" obviously refers to the collection of Threads.
But unlike general pooling technologies such as database connection pools, ThreadPool's role is not only "pooling", but its more important responsibility is actually to "do work", that is, to execute tasks. For example, when we usually use a database connection pool, we actually take out a Connection from the pool. After executing the SQL, the rewritten close() will be called to return the Connection. But have you ever seen someone asking for Thread from ThreadPool? Will it give it to you? What ThreadPool does is:
Want to get Thread from the pool? No way! Don’t you know that you have a lot of knowledge about multi-threading? Be careful to play with fire and burn yourself. If you want to perform a task, you can throw it in yourself and I will protect you.
In other words, ThreadPool never thought of letting you take away Thread from the beginning! But what if you want to return the results? I return a FutureTask. If you need the result, use FutureTask#get() yourself. But the initiative still lies with ThreadPool. It has the final say whether it can get the result and whether it wants to block!
At this point, it makes sense why ExecutorService provides return values and why AbstractExecutorService introduces FutureTask! However, most people find the thread pool difficult, not because they don't understand the thread "pool", but because they don't understand how it "does work". In other words: How does the thread pool perform tasks?
This involves the biggest difference between thread pools and general pooling technology: internalized execution operations, and tasks are executed through the production and consumption model, which you can see in the following Demo.
Production and consumption model
If you continuously submit tasks to the thread pool, you will generally go through 4 stages:
- Core thread processing tasks
- The task enters the task queue and waits
- Non-core thread processing tasks
- Deny policy
Especially in the second stage, tasks that are too late to be processed will be temporarily stored in the workQueue (task queue), so a typical production and consumption model emerges.
The caller delivers the Task ====> ThreadPool.workQueue ====> workerThread blocks to obtain the Task execution
Several important concepts
JVM’s Thread and operating system’s thread resources
Usually we start a thread through new Thread().start(), and threads are essentially operating system resources. Java, as a programming language, cannot allocate threads by itself. So, what is the relationship between the JVM's Thread object and the operating system's thread resources? You might as well imagine a scene:
In a windy and rainy suburb, Zhang Tianshi of Longhu Mountain faced all kinds of monsters alone! I saw Zhang Tianshi slowly taking out a "Sky Thunder Talisman" from his arms, gently twisting it with his fingertips, and the spell suddenly burst into blue flames. Soon enough, the sky suddenly became covered with dark clouds and roared, and then a purple lightning bolt descended from the sky like a dragon and a snake, striking at the group of monsters and monsters.
The "Thunder Talisman" is not a real thunder, but it can summon thunder through the Thunder Talisman. It is a contract binding relationship. Thread is the same as the thread of the operating system.
How does the thread pool reuse threads?
Sometimes, to solve a problem, it may be easier to start in the opposite direction. Let’s not worry about how to reuse threads for now, let me ask you: How to recycle/destroy threads? (Knowing under what circumstances the thread will be destroyed, then as long as you avoid destruction, you can reuse it)
The word "thread" actually has two levels of reference: ThreadObject, JVMThread resources (essentially operating system threads). There is a binding relationship between the Thread object and the thread resource. After a thread resource is allocated, Thread#run() will be found as the code execution entry.
When will the thread be destroyed? Normally, after new Thread(tartget).start(), the operating system will allocate thread resources, and when the thread finishes executing the code in Thread#run(), it will die naturally. As for the Thread object, if there is no reference, it will also be recycled by GC.
Seeing this, I think everyone should understand:As long as the task never ends, the thread will never die. How can the task never end? Either do the task in a loop or block.
The essence of the thread pool is also Thread, but the difference is between a single entity and a collection. Since Thread's characteristic of "destroying after completing the task" is innate and destined, the thread pool cannot change this. Therefore, if the thread pool wants to keep the internal threads alive, it must keeps threads busy working, that is, keep them working. What should I do if I really have no work to do? Then block it (you can use blocking queue)! In short, you cannot be allowed to "complete execution", otherwise it will be destroyed.
How to ensure that only "non-core threads" are destroyed
Everyone has heard some formulas in eight-part essays, such as "During idle time, if a non-core thread is idle for more than keepAliveTime, it will be recycled." How is this achieved?
First of all, there is a common misunderstanding that many people think that the thread pool will mark each Thread when creating a thread, such as marking core threads as coreThread, non-core threads as nonCoreThread, and then recycling nonCoreThread during idle time. However, Doug Lea of JDK doesn’t think so. The solution he adopted is simpler and more crude:
- The current number of threads <= corePoolSize, then all threads are core threads and will not be recycled.
- Current number of threads > corePoolSize, recyclingExcessivethreads
Look, when corePoolSize is exceeded,the excess threads are "redundant" and no specific selection will be made during actual recycling. Recycle one by one until the number of threads is not greater than corePoolSize. (There is a schematic diagram later to help understand how threads are recycled)
Why is queue priority given after corePoolSize is exceeded?
Many interviewers like to ask: Why does the thread pool choose to put tasks into the queue first when the number of core threads is full, instead of immediately allocating non-core threads for processing?
In fact, this is reversing cause and effect. The thread pool was originally intended to use queue buffering to execute tasks, but at the same time, in order to ensure "elasticity", it allowed the expansion of non-core threads to speed up efficiency after the queue was full. So instead of asking "Why should corePoolSize be added to the queue first instead of allocating non-core threads directly when the corePoolSize is full?", you should think about "What are the benefits of allocating non-core threads after the queue is full?"
In addition, thread resources are relatively precious (especially if they are frequently created and destroyed), so if you can use queue buffering, don't create additional threads.
Simple version of TheadPool
In order to reduce the difficulty, we first write a minimalist version of the thread pool.
public class SimpleThreadPool {
/**
* 任务队列
*/
BlockingQueue<Runnable> workQueue;
/**
* 工作线程
*/
List<Worker> workers = new ArrayList<>();
/**
* 构造器
*
* @param poolSize 线程数
* @param workQueue 任务队列
*/
SimpleThreadPool(int poolSize, BlockingQueue<Runnable> workQueue) {
this.workQueue = workQueue;
// 创建线程，并加入线程池
for (int i = 0; i < poolSize; i++) {
Worker work = new Worker();
work.start();
workers.add(work);
}
}
/**
* 提交任务
*
* @param command
*/
void execute(Runnable command) {
try {
// 任务队列满了则阻塞
workQueue.put(command);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
/**
* 工作线程，负责执行任务
*/
class Worker extends Thread {
public void run() {
// 循环获取任务，如果任务为空则阻塞等待
while (true) {
try {
Runnable task = workQueue.take();
task.run();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}Schematic diagram
Test Case
public class SimpleThreadPoolTest {
public static void main(String[] args) {
SimpleThreadPool simpleThreadPool = new SimpleThreadPool(2, new ArrayBlockingQueue<Runnable>(2));
simpleThreadPool.execute(() -> {
System.out.println("第1个任务开始");
sleep(3);
System.out.println("第1个任务结束");
});
simpleThreadPool.execute(() -> {
System.out.println("第2个任务开始");
sleep(4);
System.out.println("第2个任务结束");
});
simpleThreadPool.execute(() -> {
System.out.println("第3个任务开始");
sleep(5);
System.out.println("第3个任务结束");
});
}
private static void sleep(int seconds) {
try {
TimeUnit.SECONDS.sleep(seconds);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
There is only one core of SimpleThreadPool above: the production and consumption model, which does not involve various logical judgments when submitting tasks (directly joining the blocking queue), and there is no destruction of non-core threads. In addition, put is used when submitting tasks to the blocking queue. When the queue is full, it will be blocked. This "rejection strategy" is obviously unreasonable. The complex version of ThreadPool will be expanded on this basis to complete some details.
Complex version of ThreadPool
Since the rejection strategy is not the core logic, throwing an exception is used instead. In addition, the code logic and structure, and even the variable names are basically consistent with ThreadPoolExecutor. It can be said that although the sparrow is small, it has all the internal organs. It is recommended to debug and read it:
public class ThreadPool {
private final ReentrantLock mainLock = new ReentrantLock();
/**
* 工作线程
*/
private final List<Worker> workers = new ArrayList<>();
/**
* 任务队列
*/
private BlockingQueue<Runnable> workQueue;
/**
* 核心线程数
*/
private final int corePoolSize;
/**
* 最大线程数
*/
private final int maximumPoolSize;
/**
* 非核心线程最大空闲时间（否则销毁线程）
*/
private final long keepAliveTime;
public ThreadPool(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit timeUnit,
BlockingQueue<Runnable> workQueue) {
this.workQueue = workQueue;
this.corePoolSize = corePoolSize;
this.maximumPoolSize = maximumPoolSize;
this.keepAliveTime = timeUnit.toNanos(keepAliveTime);
}
public void execute(Runnable task) {
Assert.notNull(task, "task is null");
// 创建核心线程处理任务
if (workers.size() < corePoolSize) {
this.addWorker(task, true);
return;
}
// 尝试加入任务队列
boolean enqueued = workQueue.offer(task);
if (enqueued) {
return;
}
// 创建非核心线程处理任务
if (!this.addWorker(task, false)) {
// 非核心线程数达到上限，触发拒绝策略
throw new RuntimeException("拒绝策略");
}
}
private boolean addWorker(Runnable task, boolean core) {
int wc = workers.size();
if (wc >= (core ? corePoolSize : maximumPoolSize)) {
return false;
}
boolean workerStarted = false;
try {
Worker worker = new Worker(task);
final Thread thread = worker.getThread();
if (thread != null) {
mainLock.lock();
workers.add(worker);
thread.start();
workerStarted = true;
}
} catch (Exception e) {
e.printStackTrace();
} finally {
mainLock.unlock();
}
return workerStarted;
}
private void runWorker(Worker worker) {
Runnable task = worker.getTask();
try {
// 循环处理任务
while (task != null || (task = getTask()) != null) {
task.run();
task = null;
}
} finally {
// 从循环退出来，意味着当前线程是非核心线程，而且需要被销毁
// Java的线程，既可以指代Thread对象，也可以指代JVM线程，一个Thread对象绑定一个JVM线程
// 因此，线程的销毁分为两个维度：1.把Thread对象从workers移除 2.JVM线程执行完当前任务，会自然销毁
workers.remove(worker); // 这里前后应该加锁，否则线程不安全。由于是demo，很多处理比较随意
}
}
private Runnable getTask() {
boolean timedOut = false;
// 循环获取任务
for (; ; ) {
// 是否需要检测超时：当前线程数超过核心线程
boolean timed = workers.size() > corePoolSize;
// 需要检测超时 && 已经超时了
if (timed && timedOut) {
return null;
}
try {
// 是否需要检测超时
// 1.需要：poll阻塞获取，等待keepAliveTime，等待结束就返回，不管有没有获取到任务
// 2.不需要：take持续阻塞，直到获取结果
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
}
}
@Getter
@Setter
private class Worker implements Runnable {
private Thread thread;
private Runnable task;
public Worker(Runnable task) {
this.task = task;
thread = new Thread(this);
}
@Override
public void run() {
runWorker(this);
}
}
}Code diagram (asynchronous execution by Thread in the dotted box):
This picture is flawed. In fact, the thread pool does not distinguish between coreThread and nonCoreThread. It only depends on whether the current number of threads is greater than corePoolSize.
Test Case
@Slf4j
public class ThreadPoolTest {
public static void main(String[] args) {
// 创建线程池，核心线程1，最大线程2
// 提交4个任务：第1个任务交给核心线程、第2个任务入队、第3个任务交给非核心线程、第4个任务被拒绝
ThreadPool threadPoolExecutor = new ThreadPool(
1,
2,
1,
TimeUnit.SECONDS,
new ArrayBlockingQueue<>(1)
);
threadPoolExecutor.execute(() -> {
log.info("{}:执行第1个任务...", Thread.currentThread().getName());
sleep(10);
});
sleep(1);
threadPoolExecutor.execute(() -> {
log.info("{}:执行第2个任务...", Thread.currentThread().getName());
sleep(10);
});
sleep(1);
threadPoolExecutor.execute(() -> {
log.info("{}:执行第3个任务...", Thread.currentThread().getName());
sleep(10);
});
sleep(1);
threadPoolExecutor.execute(() -> {
log.info("{}:执行第4个任务...", Thread.currentThread().getName());
sleep(10);
});
sleep(1);
log.info("main结束");
}
private static void sleep(int seconds) {
try {
TimeUnit.SECONDS.sleep(seconds);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
You can replace the thread pool in the test case with JDK's ThreadPoolExecutor, and the execution effect will be very similar:
About the author: Hello everyone, I am Brother Smart, a former architect of ZTE and Meituan, and now the CTO of an Internet company.
Join the group, let’s learn together, make progress together, and fight the Internet winter together