Learning the source code of ThreadPoolExecutor and feeling about brother Guanxiong's blog

Article directory

Preface

Today, I saw xiaxiong's How to make the thread pool full before putting the queue in Java Although it's a bit coquettish to see the realization, but I'm sorry to say that this is a good way to save the country.

ThreadPookExecutor part source code

execute(Runnable command)

Take a look at the following notes in Chinese

public void execute(Runnable command) {
        if (command == null)
            throw new NullPointerException();
        /*
         * Proceed in 3 steps:
         *
         * 1. If fewer than corePoolSize threads are running, try to
         * start a new thread with the given command as its first
         * task.  The call to addWorker atomically checks runState and
         * workerCount, and so prevents false alarms that would add
         * threads when it shouldn't, by returning false.
         *
         * 2. If a task can be successfully queued, then we still need
         * to double-check whether we should have added a thread
         * (because existing ones died since last checking) or that
         * the pool shut down since entry into this method. So we
         * recheck state and if necessary roll back the enqueuing if
         * stopped, or start a new thread if there are none.
         *
         * 3. If we cannot queue task, then we try to add a new
         * thread.  If it fails, we know we are shut down or saturated
         * and so reject the task.
         */
         //Count by AtomicInteger
        int c = ctl.get();
        //If the number of threads is less than the number of core threads, it means that a new thread is created, and it is dropped into the thread pool addWorker(). Then the count will be + 1
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        //If the thread pool is still running, try to throw it into the queue. Use the offer method to determine whether the queue is full. This is also the reason why the unbounded queue will cause the cpu to burst and insert it into the queue.
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        //If the maximum number of threads is reached, an exception RejectedExecutionException will be thrown
        else if (!addWorker(command, false))
            reject(command);
    }

Is RejectedExecutionException a little familiar? When there are too many tasks, the thread pool will throw this exception, which can also be understood as the rejection strategy of the thread pool.

Impressions of after reading

                    ~

Let's see how it is realized

Override thread pool

public class EagerThreadPoolExecutor extends ThreadPoolExecutor {

    /**
     * Define a member variable to record the number of tasks submitted in the current thread pool
     */
    private final AtomicInteger submittedTaskCount = new AtomicInteger(0);

    public EagerThreadPoolExecutor(int corePoolSize,
                                   int maximumPoolSize,
                                   long keepAliveTime,
                                   TimeUnit unit, TaskQueue<Runnable> workQueue,
                                   ThreadFactory threadFactory,
                                   RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
    }


    public int getSubmittedTaskCount() {
        return submittedTaskCount.get();
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {
        // The checkmark method of ThreadPoolExecutor. After task execution, the number of tasks submitted in the pool needs to be - 1
        submittedTaskCount.decrementAndGet();
    }

    @Override
    public void execute(Runnable command) {
        if (command == null) {
            throw new NullPointerException();
        }
        // do not increment in method beforeExecute!
        // Number of tasks submitted in the pool + 1
        submittedTaskCount.incrementAndGet();
        try {
            super.execute(command);
        } catch (RejectedExecutionException rx) {
            // retry to offer the task into queue.
            final TaskQueue queue = (TaskQueue) super.getQueue();
            try {
                if (!queue.retryOffer(command, 0, TimeUnit.MILLISECONDS)) {
                    submittedTaskCount.decrementAndGet();
                    throw new RejectedExecutionException("Queue capacity is full.", rx);
                }
            } catch (InterruptedException x) {
                submittedTaskCount.decrementAndGet();
                throw new RejectedExecutionException(x);
            }
        } catch (Throwable t) {
            // decrease any way
            submittedTaskCount.decrementAndGet();
            throw t;
        }
    }
}

Main code

                          ?

                              .

                   . When it is false, it indicates that the queue is full, and a RejectedExecutionException exception is thrown.

I feel that this operation is very coquettish, but it's also very strong. Ha ha, it's so powerful~

                      

Rewriting queue

public class TaskQueue<R extends Runnable> extends LinkedBlockingQueue<Runnable> {

    private static final long serialVersionUID = -2635853580887179627L;
    
    // Custom thread pool class, inherited from ThreadPoolExecutor
    private EagerThreadPoolExecutor executor;

    public TaskQueue(int capacity) {
        super(capacity);
    }

    public void setExecutor(EagerThreadPoolExecutor exec) {
        executor = exec;
    }

    // The meaning of the offer method is: submit the task to the queue, and the return value is true/false, which respectively represents the success / failure of submission
    @Override
    public boolean offer(Runnable runnable) {
        if (executor == null) {
            throw new RejectedExecutionException("The task queue does not have executor!");
        }
        // Current number of threads in the thread pool
        int currentPoolThreadSize = executor.getPoolSize();
        if (executor.getSubmittedTaskCount() < currentPoolThreadSize) {
            // The number of submitted tasks is less than the current number of threads, which means that there are idle threads in the thread pool, which are directly thrown into the queue for processing
            return super.offer(runnable);
        }

        // return false to let executor create new worker.
        if (currentPoolThreadSize < executor.getMaximumPoolSize()) {
            // Key points: if the current thread number is less than the maximum thread number, false will be returned, implying failure to join the queue. Let the thread pool create a new thread
            return false;
        }

        // Important: the code runs here, indicating that the current number of threads > = the maximum number of threads, which needs to be submitted to the queue
        return super.offer(runnable);
    }

    public boolean retryOffer(Runnable o, long timeout, TimeUnit unit) throws InterruptedException {
        if (executor.isShutdown()) {
            throw new RejectedExecutionException("Executor is shutdown!");
        }
        return super.offer(o, timeout, unit);
    }
}

Key code

// return false to let executor create new worker.
        if (currentPoolThreadSize < executor.getMaximumPoolSize()) {
            // Key points: if the current thread number is less than the maximum thread number, false will be returned, implying failure to join the queue. Let the thread pool create a new thread
            return false;
        }
226 original articles published, praised by 31 and visited 210000+
Private letter follow

Tags: less Java

Posted on Sat, 14 Mar 2020 23:05:18 -0400 by Jenski