How do I set the size of the Java thread pool?

Source: ifeve

Concurrent functionality is used more or less in our daily business development process.So when you use concurrency, you're bound to encounter the following issues

What is the exact size of the concurrent thread pool?

Typically, older programmers may have heard of this statement (where N represents the number of CPU s)

  1. CPU intensive application with thread pool size set to N + 1

  2. IO-intensive applications with thread pool size set to 2N

Is this statement correct?

In fact, this is extremely incorrect.So why?

First of all, let's say the opposite, assuming this is true, it doesn't matter how many services we deploy on a single server.This is incorrect because the size of the thread pool can only be related to the number of cores on the server.How do I set the size?

Assuming that this application is a hybrid of both, where tasks are both CPU-intensive and IO-intensive, how do we set it up?Can I just throw my hard drive?

So how do we actually set the thread pool size?Are there any specific practical methods to guide you to land?Let's take a closer look.

Little's Law

The number of requests per system is equal to the product of the arrival rate of requests and the average time spent per individual request

Assuming that the server is single-core and the corresponding business needs to guarantee the number of requests (QPS): 10, it takes 1 second to actually process a request, then the server has 10 requests to process at each time, that is, it needs 10 threads

Similarly, we can use Little's law to determine the size of the thread pool.We only need to calculate request arrival rates and average request processing time.Then the average number of requests can be calculated by putting the above values into Little's law.The estimation formula is as follows

Thread pool size = ((thread IO time + thread CPU time) / thread CPU time) Number of CPUs*

Specific Practice

From the formula, we know that three specific values are needed

  1. Time consumed by a request (thread IO time + thread CPU time)

  2. The request calculation time (thread CPU time)

  3. Number of CPU s

Request time consumed

In a Web service container, you can use Filter to intercept the time it takes to get the request

public class MoniterFilter implements Filter {

private static final Logger logger = LoggerFactory.getLogger(MoniterFilter.class);

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException,
            ServletException {
long start = System.currentTimeMillis();

        HttpServletRequest httpRequest = (HttpServletRequest) request;
        HttpServletResponse httpResponse = (HttpServletResponse) response;
        String uri = httpRequest.getRequestURI();
        String params = getQueryString(httpRequest);

try {
            chain.doFilter(httpRequest, httpResponse);
        } finally {
long cost = System.currentTimeMillis() - start;
  "access url [{}{}], cost time [{}] ms )", uri, params, cost);

private String getQueryString(HttpServletRequest req) {
        StringBuilder buffer = new StringBuilder("?");
        Enumeration<String> emParams = req.getParameterNames();
try {
    while (emParams.hasMoreElements()) {
                String sParam = emParams.nextElement();
                String sValues = req.getParameter(sParam);
return buffer.substring(0, buffer.length() - 1);
        } catch (Exception e) {
            logger.error("get post arguments error", buffer.toString());
return "";

CPU calculation time

CPU calculation time = total request time - CPU IO time

Assuming that the request has a DB query operation, once you know the CPU IO time of the DB query, the calculation time will not come out. Let's see how to keep the DB query time concise and clear.

Get thread IO time by adding AOP facets (JDK dynamic proxy / CGLIB).The code is as follows, see:

public class DaoInterceptor implements MethodInterceptor {

private static final Logger logger = LoggerFactory.getLogger(DaoInterceptor.class);

public Object invoke(MethodInvocation invocation) throws Throwable {
        StopWatch watch = new StopWatch();
        Object result = null;
        Throwable t = null;
try {
            result = invocation.proceed();
        } catch (Throwable e) {
            t = e == null ? null : e.getCause();
throw e;
        } finally {
  "({}ms)", watch.getTotalTimeMillis());
return result;


Number of CPU s

Number of logical CPUs, number of CPUs referenced when setting thread pool size

cat /proc/cpuinfo| grep "processor"| wc -l


Configuring the appropriate thread pool size is not easy, but with the above formulas and code, we can quickly and accurately figure out how large the thread pool should be.

However, in the end, we still need to fine-tune through the pressure test, only after the test of the pressure test, can we finally ensure that the configuration size is accurate.

Recommend going to my blog to read more:

1.Java JVM, Collections, Multithreaded, New Features Series Tutorial

2.Spring MVC, Spring Boot, Spring Cloud series tutorials

3.Maven, Git, Eclipse, Intellij IDEA Series Tools Tutorial

4.Latest Interview Questions for Java, Backend, Architecture, Alibaba, etc.

Feel good, don't forget to say yes + forward!

Tags: Java Spring less JDK

Posted on Sun, 10 May 2020 23:38:30 -0400 by thebadbad