Analysis of eight rejection strategies for java thread pool ThreadPoolExecutor

See a very good article to share with you.

preface

When it comes to Java's thread pool, the most familiar one is the ExecutorService interface, which is newly added in jdk1.5 java.util.concurrent The api under the package greatly simplifies the development of multithreaded code. Whether you use FixedThreadPool or CachedThreadPool, the implementation behind it is ThreadPoolExecutor. ThreadPoolExecutor is a typical product of cache pooling design. Because the pool has size, when the pool is not large enough to carry, it involves rejection strategy. Four kinds of thread pool rejection strategies have been preset in JDK. Let's talk about the usage scenarios of these strategies in detail and what rejection strategies we can extend.

Pool design idea

Pool design should not be a new term. Our common designs such as java thread pool, jdbc connection pool and redis connection pool are representative implementations of this kind of design. This design will initially preset resources, and the problem to be solved is to offset the consumption of each resource acquisition, such as the cost of creating a thread, the cost of acquiring a remote connection, etc. It's like when you go to the canteen to cook, the mother of the rice maker will put several portions of the rice there first. When you come, you can directly take the lunch box and add vegetables. You don't need to cook and cook temporarily, so the efficiency is high. In addition to initializing resources, the pooling design also includes the following features: the initial value of the pool, the active value of the pool, the maximum value of the pool, etc., which can be directly mapped to the member properties of the java thread pool and the database connection pool.

When the thread pool triggers the denial policy

Unlike the data source connection pool, the thread pool has an additional blocking queue to buffer in addition to its initial size and pool maximum. Generally, when the number of connections requested by the data source connection pool exceeds the maximum value of the connection pool, the rejection policy will be triggered. Generally, the policy is to block the waiting time or directly throw exceptions. The trigger time of thread pool is as follows:

As shown in the figure, if you want to know when the thread pool triggers coarse rejection, you need to clarify the specific meaning of the above three parameters, which is the result of the overall coordination of these three parameters, rather than simply exceeding the maximum number of threads, which will trigger coarse rejection. When the number of tasks submitted is greater than the corePoolSize, it will be placed in the queue buffer first, and only when the buffer is filled can you judge when the Whether the previously run task is greater than maxPoolSize, if less than, a new thread will be created for processing. When it is greater than, the rejection policy is triggered. The summary is: when the current number of submitted tasks is greater than (maxPoolSize + queueCapacity), the rejection policy of thread pool will be triggered.

JDK has four built-in thread pool rejection strategies

Reject policy interface definition
Before analyzing the thread pool rejection policy of JDK, first look at the rejection policy interface defined by JDK, as follows:

public interface RejectedExecutionHandler {
    void rejectedExecution(Runnable r, ThreadPoolExecutor executor);
}

The interface definition is very clear. When the rejection policy is triggered, the thread pool will call the specific policy you set, and pass the currently submitted task and the thread pool instance itself to you for processing. What is the specific processing? Different scenarios will have different considerations. Let's see what implementation JDK has built in for us:

CallerRunsPolicy (caller running policy)

    public static class CallerRunsPolicy implements RejectedExecutionHandler {

        public CallerRunsPolicy() { }

        public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
            if (!e.isShutdown()) {
                r.run();
            }
        }
    }

Function: when the rejection policy is triggered, as long as the thread pool is not closed, it will be processed by the current thread submitting the task.

Usage scenario: it is generally used in scenarios where failure is not allowed, performance requirements are not high, and concurrency is small, because the thread pool will not be closed in general, that is to say, the submitted task will be run, but because it is executed by the caller thread itself, when the task is submitted several times, the subsequent task execution will be blocked, and the performance and efficiency will be slow naturally.

AbortPolicy

    public static class AbortPolicy implements RejectedExecutionHandler {

        public AbortPolicy() { }

        public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
            throw new RejectedExecutionException("Task " + r.toString() +
                                                 " rejected from " +
                                                 e.toString());
        }
    }

Function: when the rejection policy is triggered, an exception of rejection execution is thrown directly. The meaning of the suspension policy is to interrupt the current execution process

Usage scenario: there is no special scenario, but you need to handle the exceptions thrown correctly. The default policy in ThreadPoolExecutor is AbortPolicy. This is the default policy in the ThreadPoolExecutor series of ExecutorService interfaces because there is no set rejection policy displayed. However, please note that the thread pool instance queue in ExecutorService is unbounded, that is to say, the denial policy will not be triggered if the memory is full. When you customize a thread pool instance, you must handle the exceptions thrown when using this strategy, because it will interrupt the current execution process.

DiscardPolicy

    public static class DiscardPolicy implements RejectedExecutionHandler {

        public DiscardPolicy() { }

        public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
        }
    }

Function: discard the task directly and quietly without triggering any action

Usage scenario: if the task you submit doesn't matter, you can use it. Because it is an empty realization, will quietly devour your task. So this strategy is basically not used

DiscardOldestPolicy

    public static class DiscardOldestPolicy implements RejectedExecutionHandler {

        public DiscardOldestPolicy() { }

        public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
            if (!e.isShutdown()) {
                e.getQueue().poll();
                e.execute(r);
            }
        }
    }

Function: if the thread pool is not closed, the element in the queue head will pop up, and then try to execute

Usage scenario: this strategy still discards tasks, and there is no sound when discarding them. However, it is characterized by discarding old unimplemented tasks and tasks with higher priority to be executed. Based on this feature, the scenario I can think of is to publish and modify the message. When the message is published, it has not been executed, and the updated message comes again. At this time, the version of the unexecuted message is lower than the version of the message submitted now, and it can be discarded. Because there may be messages with lower message versions in the queue that will be queued for execution, it is necessary to do a good job of message version comparison when actually processing messages

Rejection strategy implemented by the third party

Thread rejection policy in dubbo

public class AbortPolicyWithReport extends ThreadPoolExecutor.AbortPolicy {

    protected static final Logger logger = LoggerFactory.getLogger(AbortPolicyWithReport.class);

    private final String threadName;

    private final URL url;

    private static volatile long lastPrintTime = 0;

    private static Semaphore guard = new Semaphore(1);

    public AbortPolicyWithReport(String threadName, URL url) {
        this.threadName = threadName;
        this.url = url;
    }

    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
        String msg = String.format("Thread pool is EXHAUSTED!" +
                        " Thread Name: %s, Pool Size: %d (active: %d, core: %d, max: %d, largest: %d), Task: %d (completed: %d)," +
                        " Executor status:(isShutdown:%s, isTerminated:%s, isTerminating:%s), in %s://%s:%d!",
                threadName, e.getPoolSize(), e.getActiveCount(), e.getCorePoolSize(), e.getMaximumPoolSize(), e.getLargestPoolSize(),
                e.getTaskCount(), e.getCompletedTaskCount(), e.isShutdown(), e.isTerminated(), e.isTerminating(),
                url.getProtocol(), url.getIp(), url.getPort());
        logger.warn(msg);
        dumpJStack();
        throw new RejectedExecutionException(msg);
    }

    private void dumpJStack() {
       //Omit implementation
    }
}

As you can see, when dubbo's worker thread triggers thread rejection, it mainly does three things. The principle is to let the user know the real reason of triggering thread rejection strategy as much as possible

A warning level log is output. The log content is the detailed setting parameters of the thread pool, the current status of the thread pool, and some details of the current task rejection. It can be said that those who have experience in production and operation and maintenance using dubbo have seen this log more or less. This log is just a model of log printing. Other models of log printing include spring. Thanks to such a detailed log, it is easy to locate the problem
Output the details of the current thread stack. This is very useful. When you can't locate the problem through the above log information, the dump thread context information at the crime scene is the life-saving straw for you to find the problem. For this, please refer to "dubbo thread pool exhaustion event -" the disaster caused by CyclicBarrier "
Continue to throw the reject execution exception to fail this task, which inherits the default reject policy of JDK
Thread pool rejection policy in Netty

    private static final class NewThreadRunsPolicy implements RejectedExecutionHandler {
        NewThreadRunsPolicy() {
            super();
        }

        public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
            try {
                final Thread t = new Thread(r, "Temporary task executor");
                t.start();
            } catch (Throwable e) {
                throw new RejectedExecutionException(
                        "Failed to start a new thread", e);
            }
        }
    }

The implementation in Netty is similar to the CallerRunsPolicy in JDK, and is reluctant to discard tasks. The difference is that CallerRunsPolicy is a task that is executed directly in the caller thread. And Netty created a new thread to handle it. Therefore, the implementation of Netty can be extended to support efficient and high-performance scenarios compared with the use of caller execution strategy. But it should also be noted that in the implementation of Netty, there is no judgment constraint when creating a thread, that is to say, as long as the system has resources, it will create a new thread to handle, and only when the new thread cannot be created, will it throw the exception of creating a thread failure

Thread pool rejection policy in activeMq

 new RejectedExecutionHandler() {
                @Override
                public void rejectedExecution(final Runnable r, final ThreadPoolExecutor executor) {
                    try {
                        executor.getQueue().offer(r, 60, TimeUnit.SECONDS);
                    } catch (InterruptedException e) {
                        throw new RejectedExecutionException("Interrupted waiting for BrokerService.worker");
                    }

                    throw new RejectedExecutionException("Timed Out while attempting to enqueue Task.");
                }
            });

The strategy in activeMq is the best effort task type. When the rejection strategy is triggered, the task will be put into the task queue again within one minute of the attempt. When the one minute timeout is not successful, an exception will be thrown

Thread pool rejection policy in pinpoint

public class RejectedExecutionHandlerChain implements RejectedExecutionHandler {
    private final RejectedExecutionHandler[] handlerChain;

    public static RejectedExecutionHandler build(List<RejectedExecutionHandler> chain) {
        Objects.requireNonNull(chain, "handlerChain must not be null");
        RejectedExecutionHandler[] handlerChain = chain.toArray(new RejectedExecutionHandler[0]);
        return new RejectedExecutionHandlerChain(handlerChain);
    }

    private RejectedExecutionHandlerChain(RejectedExecutionHandler[] handlerChain) {
        this.handlerChain = Objects.requireNonNull(handlerChain, "handlerChain must not be null");
    }

    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        for (RejectedExecutionHandler rejectedExecutionHandler : handlerChain) {
            rejectedExecutionHandler.rejectedExecution(r, executor);
        }
    }
}

The implementation of the rejection strategy of pinpoint has its own characteristics, which is different from other implementations. He defines a rejection policy chain and wraps a rejection policy list. When the rejection policy is triggered, the rejectedExecution in the policy chain will be executed one by one

epilogue
In the previous paper, the definition of java thread pool rejection policy interface is introduced from the design idea of thread pool and the time when the thread pool triggers the rejection policy. With the help of four kinds of JDK built-in rejection policies and four third-party open source software definitions, this paper describes various ideas and use scenarios of the implementation of thread pool rejection policies. I hope that reading this article can make you have a deeper understanding of the java thread pool rejection strategy and more flexible application according to different use scenarios.

Source: http://rrd.me/en3Wp

Tags: JDK Java Netty Dubbo

Posted on Fri, 19 Jun 2020 07:08:43 -0400 by tllewellyn