Preface

Almost all of the previous sections on the Java multithreading basics are summarized. Starting with this blog, let's summarize the contents of Java concurrency tools, starting with thread pools

Why there is a thread pool

There are two main reasons: 1. Repeatedly creating threads is expensive (which I really don't want to explain); 2. Too many threads will consume too much memory

To address these two issues, Java introduced the concept of pools.

Use a small number of threads to avoid the hassle of consuming too much memory; Always keep only some threads working and can repeat the submitted tasks, which avoids the wastage of frequent thread switching.

Thread pools can speed up response to some extent, make reasonable use of CPU and memory, and facilitate the unified management of resources. In practical development, if more than five threads need to be created, we should generally consider using thread pools to achieve this.

Thread pool parameters and task addition rules

Creating a thread pool is slightly more cumbersome, has more parameters, and involves more concepts than creating a single thread. Summarize step by step

Parameters for Thread Pool Constructor

First, familiarize yourself with the parameters used to create the thread pool, perhaps by sweeping it, some of which may not be very interesting.

Parameter Name	type	Meaning
corePoolSize	int	Number of Core Threads
maxPoolSize	int	Maximum Threads
keepAliveTime	long	Thread Lifetime
workQueue	BlockingQueue	Queue to store tasks
threadFactory	ThreadFactory	Generate a new thread's factory class
handler	RejectedExecutionHandler	Thread pool rejection policy

corePoolSize is the number of core threads. After initialization, there are no threads in the thread pool. The thread pool waits for tasks to arrive before creating threads (taking tasks out of the task queue and creating threads afterwards).

maxPoolSize has a maximum number of threads, and one of the advantages of a thread pool is that you can flexibly expand the number of active threads you have based on how many tasks you have. The number of tasks to be executed at different times in the same thread pool is also different, because the number of core threads is not enough to handle too many tasks, at which point we need to extend the threads that process tasks in the thread pool

KeepAliveTime If the thread pool currently has more threads than corePoolSize, they will be terminated if the extra thread is idle longer than keepAliveTime

The workQueue stores the queue of tasks. There are usually three kinds of queues to store tasks: 1. SynchronousQueue, the directly handed-over task queue; 2. Unbounded Queue - LinkedBlockingQueue; 3. Bounded Queue--ArrayBlockingQueue

ThreadFactory new threads are created by ThreadFactory, using Executors.defaultThreadFactory() by default, and create threads in the same thread group with the same NORM_PRIORITY priority, and none of them are daemon threads. If you specify the ThreadFactory yourself, you can change properties such as thread name, thread group, priority, whether it is a daemon thread, and so on, but this is not usually necessary.

handler rejection policy, when the maximum number of thread pools is reached and the task queue is full, the newly submitted task will be rejected, and the rejection policy will execute the specific rejection action

Rules for adding tasks

It doesn't make much sense to memorize these parameters separately to create a thread pool. We need attributes and understand the rules for adding tasks to a thread pool to concatenate them

Thread pools have rules for adding tasks because they are flexible to extend

1. If the number of threads in the current thread pool is less than corePoolSize, the thread pool will still create a thread to run new tasks even if other worker threads are idle.

2. If the number of threads is equal to (or greater than) corePoolSize but less than maxPoolSize, put the task in the queue first, and the number of active threads in the thread pool is the value of corePoolSize

3. If the queue is full at this time and the number of threads in the thread pool is less than maxPoolSize, the thread pool creates a new thread to run the task

4. If the queue is full and the number of threads in the thread pool has reached maxPoolSize, the thread pool will execute the rejection policy directly if there are tasks committed to the thread pool outside at this time.

5. If the number of tasks in the thread pool is small and there are many active but idle threads at this time, those idle threads will be terminated if they are idle longer than the keepAliveTime set.

A diagram of rules for adding tasks to a thread pool

You can see that the order of judgment is corePoolSize->workQueue->maxPoolSize.

Take a simple example:

If the number of core threads in a thread pool is set to 5, the maximum number of threads is 10, and the queue length is 100.

At first, there weren't many threads, three tasks came, and the thread pool created three threads to handle the three tasks submitted. Later tasks continue to be added to the thread pool, where they are processed first with five core threads. If they cannot be processed, they are temporarily stored in the task queue. If, over time, the task queue is full and subsequent tasks are still submitted, the thread pool will continue to create more threads to process tasks. If the number of active threads exceeds the maximum number of threads set at 10, the thread pool rejects processing subsequent submitted tasks.

Based on the rules that thread pools handle tasks, summarize them further:

1. If we set the corePoolSize to be the same as the maxPoolSize, can we create a thread pool with a fixed number of active threads?

2. Thread pools generally want to maintain a small number of active threads, only when the load (the number of tasks in the task queue) becomes large.

3. If maxPoolSize is set to a high value (such as Integer.MAX_VALUE), in theory, the thread pool can add many new threads to handle tasks.

4. The thread pool will create more active threads only when the task queue is full. If we use an unbounded queue (such as LinkedBlockingQueue) to store tasks, that is, the task queue can never be full, then the number of active threads in the thread pool cannot exceed corePoolSize at this time.

So according to the characteristics of thread pools, JDK customizes thread pools for some threads, each of which has its own characteristics.

Some thread pools provided by JDK

JDK actually provides us with some commonly used thread pools, each of which has its own characteristics. Let's start with some combing

newFixedThreadPool

Its constructor source code is as follows

public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
    //The first parameter is corePoolSize, the second parameter is maxPoolSize, and the third and fourth parameters are keepAliveTime and its units
    //The fourth parameter is the task queue, and the fifth parameter is the ThreadFactory
    return new ThreadPoolExecutor(nThreads, nThreads,
                                  0L, TimeUnit.MILLISECONDS,
                                  new LinkedBlockingQueue<Runnable>(),
                                  threadFactory);
}

You can see that this is a large thread pool such as corePoolSize and maxPoolSize, and its task queue is an unbounded task queue. Therefore, when there are too many tasks, the number of threads in this thread pool is fixed, so it is called FixedThreadPool.

Basic use cases

/**
 * autor:liman
 * createtime:2021-11-01
 * comment: newFixThreadPool Examples
 */
public class NewFixThreadPoolDemo {

    public static void main(String[] args) {
        ExecutorService executorService = Executors.newFixedThreadPool(4);
        for(int i=0;i<1000;i++){
            executorService.execute(new Task());
        }
    }

}

class Task implements Runnable {

    @Override
    public void run() {
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(Thread.currentThread().getName()+"End of run");
    }
}

Because the number of active threads in this thread pool is fixed when there are too many tasks, and there is no capacity limit on the task queue, when requests are increasing and cannot be processed in time, it is easy to cause a large amount of memory to be consumed when requests are piled up, which may lead to OOM

For example, if you set the JVM parameter a little smaller, you will get an OOM exception

/**
 * autor:liman
 * createtime:2021-11-01
 * comment: newFixedThreadPool An instance of OOM appears
 * JVM Parameter-Xmx8m-Xms8m
 */
public class FixThreadPoolOOM {

    private static ExecutorService executorService = Executors.newFixedThreadPool(1);

    public static void main(String[] args) {
      for(int i=0;i<Integer.MAX_VALUE;i++){
          executorService.execute(new OOMTask());
      }
    }

}

class OOMTask implements Runnable{

    @Override
    public void run() {
        try {
            Thread.sleep(1000000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

newSingleThreadExecutor

Constructor Source

public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
    //You can see that the corePoolSize and maxPoolSize parameters are set to 1 only on the basis of newFixedThreadPool
    return new FinalizableDelegatedExecutorService
        (new ThreadPoolExecutor(1, 1,
                                0L, TimeUnit.MILLISECONDS,
                                new LinkedBlockingQueue<Runnable>(),
                                threadFactory));
}

You can see that only on the basis of newFixedThreadPool, the corePoolSize and maxPoolSize parameters are set to 1. There is a problem with newFixedThreadPool, which is a thread pool with a simple understanding of a low-profile version of newFixedThreadPool.

newCachedThreadPool

A thread pool that can be cached, is a boundless thread pool, and has the ability to automatically recycle extra threads. Its constructor source code is as follows

public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
    //corePoolSize is set to 0 and maxPoolSize to Integer.MAX_VALUE
    //The task queue is using SynchronousQueue
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                  60L, TimeUnit.SECONDS,
                                  new SynchronousQueue<Runnable>(),
                                  threadFactory);
}

As you can see from the constructor, its corePoolSize is set to 0, its maxPoolSize is set to the maximum of the integer, and its task queue is a SynchronousQueue, which we mentioned earlier is a directly handed-over queue with an internal capacity of 0. Task queues do not essentially store tasks, but are handed directly to threads for execution, so the thread pool needs to constantly create threads to handle submitted tasks.

Over time, idle threads that do not have tasks are recycled.

Instance Code

/**
 * autor:liman
 * createtime:2021-11-01
 * comment:
 */
public class CacheThreadPoolDemo {

    public static void main(String[] args) {
        ExecutorService executorService = Executors.newCachedThreadPool();
        for(int i=0;i<1000;i++){
            executorService.execute(new CacheThreadTask());
        }
    }
}

class CacheThreadTask implements Runnable{

    @Override
    public void run() {
        System.out.println(Thread.currentThread().getName());
    }
}

Unlike other thread pools, there are a few more threads working on tasks here

OOM also occurs in this thread pool because new threads are created every time a task comes in.

newScheduledThreadPool

Thread pool supporting periodic or periodic task execution

One of the constructor sources

public ScheduledThreadPoolExecutor(int corePoolSize,
                                   ThreadFactory threadFactory) {
    //I don't want to introduce it anymore.
    //Task queues use delayed queues
    super(corePoolSize, Integer.MAX_VALUE,
          DEFAULT_KEEPALIVE_MILLIS, MILLISECONDS,
          new DelayedWorkQueue(), threadFactory);
}

Basic use cases

/**
 * autor:liman
 * createtime:2021-11-01
 * comment:scheduledThreadPool Examples
 */
public class ScheduledThreadPoolDemo {

    public static void main(String[] args) {
        ScheduledExecutorService executorService
                = Executors.newScheduledThreadPool(10);
        //Delay execution of specified task by five seconds
        //executorService.schedule(new ScheduledTask(),5, TimeUnit.SECONDS);
        //Start the first time, execute the task after 1 second, then execute it every 3 seconds
        executorService.scheduleAtFixedRate(new ScheduledTask(),1,3,TimeUnit.SECONDS);
    }
}
class ScheduledTask implements Runnable{

    @Override
    public void run() {
        System.out.println(Thread.currentThread().getName());
    }
}

There are other thread pools added to JDK1.8, but they are not particularly common, so no summary is made here.

Automatic or manual creation

As for whether thread pools are created automatically or manually, when you come here, you should have an answer. Although JDK provides some available thread pools, they have some drawbacks and are highly designed. If our business scenario fits this own thread pool, you can actually try it. However, in most cases it is recommended that we create the thread pool manually.

How to set the number of threads in the thread pool

1. If our program is CPU intensive, such as encryption and frequent hash operations, the optimal number of threads at this time is 1-2 times the number of CPU cores.

2. If our program is IO-type, such as frequent reading and writing files, frequent network transfers, etc., the optimal number of threads at this time will generally be many times larger than the number of CPU cores, based on the busy situation displayed by JVM thread monitoring, to ensure that the idle threads can be converged.

The general formula is:
Line Check number = C P U nucleus heart number ∗ ( 1 + flat all etc. stay time between / flat all work do time between ) Number of threads = Number of CPU cores* (1 + Average wait time / Average working time) Number of threads = Number of CPU cores (1 + Average wait time / Average working time)
The so-called average wait time is only the time a thread spends waiting for data, and the average working time is the time a thread spends processing data.

Stop Thread Pool Correctly

Stopping the thread pool is easier than stopping individual threads normally.

Related methods

shutdown

A direct call to the shutdown method of the thread pool is equivalent to simply telling the thread pool an interrupt signal that the thread pool will wait for the task in the queue and that the task being executed in the active thread will be executed before it stops. However, during the period when the thread pool responds to an interrupt signal, submitting tasks to the thread pool will result in an error.

Instance Code

/**
 * autor:liman
 * createtime:2021-11-02
 * comment:
 */
public class ShutDownDemo {

    public static void main(String[] args) throws InterruptedException {
        ExecutorService fixedThreadPool = Executors.newFixedThreadPool(10);
        for (int i = 0; i < 1000; i++) {
            fixedThreadPool.execute(new ShutDownTask());
        }
        Thread.sleep(1500);
        //By calling shutdown here, you can see that the thread pool is still not slow enough to finish processing its own tasks before stopping
        fixedThreadPool.shutdown();
        fixedThreadPool.execute(new ShutDownTask());//Exception thrown here, thread pool does not accept new tasks when handling interrupts
    }

}

class ShutDownTask implements Runnable {

    @Override
    public void run() {
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(Thread.currentThread().getName());
    }
}

isShutdown

Determines if thread pool eating is processing interrupts.

isTerminated

Determine if the thread pool is completely stopped

awaitTermination

Determines if the thread pool stops completely after a period of time, returns true if it stops completely, or false if it does not

shutDownNow

Close immediately. Compare violence to shut down ongoing tasks directly.

If we want to stop the thread pool normally, it is recommended that we use the judgment to stop instead of using the last method to stop the thread pool directly and violently.

Rejection Policy

Rejection Policies There are not many examples to illustrate here, just a summary of several rejection strategies

Rejection Policy	Rejection Method
AbortPolicy	Throw an exception directly and it will not be submitted successfully
DiscardPolicy	Thread pool silently discards tasks
DiscardOldestPolicy	Drop the oldest tasks in the queue
CallerRunsPolicy	Threads submitting tasks execute rejection policies

Developer-defined rejection policies are also supported, simply by implementing an interface to the corresponding rejection policy

Thread Pool Principle and Source Code Analysis

In fact, this summary almost covers a few common points of knowledge in thread pools, and here we will briefly review the source code and the principles.

A thread pool consists of a thread pool manager, a work queue, a task queue, and a task interface.

Relationships between classes

Executor, ThreadPoolExecutor, Executor Service, Executors and so on. Let's first sort through the relationships

Just look at the picture first

Executor is a top-level interface where there is only one method

public interface Executor {

    /**
     * Executes the given command at some time in the future.  The command
     * may execute in a new thread, in a pooled thread, or in the calling
     * thread, at the discretion of the {@code Executor} implementation.
     *
     * @param command the runnable task
     * @throws RejectedExecutionException if this task cannot be
     * accepted for execution
     * @throws NullPointerException if command is null
     */
    void execute(Runnable command);
}

ExecutorService is an interface that inherits Executor, with new methods such as closing thread pools added

Executors is a tool class, just a tool class for creating a thread pool

ThreadPoolExecutor is the thread pool subclass, but we created the thread pool through Executors and returned the class ExecutorService, which is the parent of ThreadPoolExecutor

Principle of Task Reuse by Thread Pool

Enter from executor method

public void execute(Runnable command) {
    if (command == null)
        throw new NullPointerException();
    //Comments in the associated source code also describe the steps for thread pools to perform tasks and extend core threads
    /*
     * Proceed in 3 steps:
     *
     * 1. If fewer than corePoolSize threads are running, try to
     * start a new thread with the given command as its first
     * task.  The call to addWorker atomically checks runState and
     * workerCount, and so prevents false alarms that would add
     * threads when it shouldn't, by returning false.
     *
     * 2. If a task can be successfully queued, then we still need
     * to double-check whether we should have added a thread
     * (because existing ones died since last checking) or that
     * the pool shut down since entry into this method. So we
     * recheck state and if necessary roll back the enqueuing if
     * stopped, or start a new thread if there are none.
     *
     * 3. If we cannot queue task, then we try to add a new
     * thread.  If it fails, we know we are shut down or saturated
     * and so reject the task.
     */
    int c = ctl.get();
    if (workerCountOf(c) < corePoolSize) {
        //If the number of currently running threads is less than the number of core threads, a thread is created, where the second parameter of addWorker is true to indicate whether the number of currently active threads is greater than the number of core threads when creating a thread, and false to indicate whether the number of running threads is greater than maxPoolSize when creating a thread
        if (addWorker(command, true))
            return;
        c = ctl.get();
    }
    //Check if the thread pool is running and, if so, place the task in a waiting queue
    if (isRunning(c) && workQueue.offer(command)) {
        int recheck = ctl.get();
        //If the thread is not running, delete the current task and execute the rejection policy
        if (! isRunning(recheck) && remove(command))
            reject(command);//Execute Rejection Policy
        else if (workerCountOf(recheck) == 0)//If no thread is currently executing a task, you need to create a thread
            addWorker(null, false);
    }
    //If addWorker returns false, it means it has not been added to the task queue and will be rejected
    else if (!addWorker(command, false))
        reject(command);
}

The source code is explained in the code comments, so let's continue with addWorker

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (int c = ctl.get();;) {
        // Check if queue empty only if necessary.
        if (runStateAtLeast(c, SHUTDOWN)
            && (runStateAtLeast(c, STOP)
                || firstTask != null
                || workQueue.isEmpty()))
            return false;

        for (;;) {
            if (workerCountOf(c)
                >= ((core ? corePoolSize : maximumPoolSize) & COUNT_MASK))
                return false;
            if (compareAndIncrementWorkerCount(c))
                break retry;
            c = ctl.get();  // Re-read ctl
            if (runStateAtLeast(c, SHUTDOWN))
                continue retry;
            // else CAS failed due to workerCount change; retry inner loop
        }
    }

    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        w = new Worker(firstTask);
        final Thread t = w.thread;
        if (t != null) {
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                // Recheck while holding lock.
                // Back out on ThreadFactory failure or if
                // shut down before lock acquired.
                int c = ctl.get();

                if (isRunning(c) ||
                    (runStateLessThan(c, STOP) && firstTask == null)) {
                    if (t.getState() != Thread.State.NEW)
                        throw new IllegalThreadStateException();
                    //Build a Worker object from the incoming Runnable and add it to the task queue
                    workers.add(w);
                    workerAdded = true;
                    int s = workers.size();
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                }
            } finally {
                mainLock.unlock();
            }
            if (workerAdded) {
                t.start();
                workerStarted = true;
            }
        }
    } finally {
        if (! workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

What really runs the thread is the runWorker method

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); // allow interrupts
    boolean completedAbruptly = true;
    try {
        //Call threads in workQueue iteratively, then call their run methods
        while (task != null || (task = getTask()) != null) {
            w.lock();
            // If pool is stopping, ensure thread is interrupted;
            // if not, ensure thread is not interrupted.  This
            // requires a recheck in second case to deal with
            // shutdownNow race while clearing interrupt
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() &&
                  runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                beforeExecute(wt, task);
                try {
                    task.run();
                    afterExecute(task, null);
                } catch (Throwable ex) {
                    afterExecute(task, ex);
                    throw ex;
                }
            } finally {
                task = null;
                w.completedTasks++;
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        processWorkerExit(w, completedAbruptly);
    }
}

The state of the thread pool

Thread pools are stateful just like threads. Here is a simple list of the states of a thread pool

The state of the thread pool	Meaning
RUNNING	Accept new tasks and process queued tasks
SHUTDOWN	Do not accept new tasks, but handle queued tasks
STOP	Do not accept new tasks, do not process queued tasks, and interrupt running tasks
TIDYING	All tasks have been terminated, workerCount is 0, start running terminate() method
TERMINATED	Thread pool completely stopped

summary

A simple comb of the contents of the thread pool means that tasks are reused through several active threads, but when using the thread pool, you need to avoid too many threads to create and too many tasks to pile up. The hook functions for thread pools are not summarized here, so come back after you have combed the concept of locks in concurrent programming.