Noseparte says: How is a gateway thread pool created in online games

ThreadPool ThreadPool

1. Definition of thread pool:

(From Job Q ) In object-oriented programming, creating and destroying objects is time-consuming because creating an object requires memory resources or more.This is especially true in Java, where virtual machines will attempt to track each object so that they can be garbage collected after the object has been destroyed.Therefore, one of the ways to improve the efficiency of service programs is to minimize the number of objects created and destroyed, especially those that are very resource intensive, which is why the "pooled resources" technology comes into being.Thread pools, as the name implies, are precreated by creating several executable threads into a pool (container) and retrieving threads from the pool when needed without creating them on their own, thus reducing the overhead of creating and destroying Thread objects without destroying them.

2. How to create a thread pool:

Using ThreadPoolExecutor:

ThreadPoolExecutor is a flexible and stable thread pool that allows customization.

Using Executors:

One of the static factory methods in Executors to create a thread pool:
newSingleThreadExecutor: is a single-threaded Executor that creates a single worker thread to perform tasks and, if the thread ends abnormally, creates another thread to replace it.newSingleThreadExecutor ensures that tasks are executed serially in the order in which they are queued (for example, FIFO, LIFO, priority).
newFixedThreadPool: A fixed-length thread pool will be created, creating a thread each time a task is submitted, until the maximum number of thread pools is reached, and the size of the thread pool will no longer change (if a thread ends up with an unexpected Exception, the thread pool will be supplemented with a new thread).
newCachedThreadPool: A cacheable thread pool will be created, and idle threads will be recycled if the current size of the thread pool exceeds processing requirements, and new threads can be added when demand increases. There is no limit to the size of the thread pool.
newScheduledThreadExecutor: Creates a fixed-length thread pool and executes tasks in a delayed or timed manner, similar to Timer.

Configure ThreadPoolExecutor

public class ThreadPoolExecutor { // Minimum number of threads maintained by thread pool private volatile int corePoolSize; // Maximum number of threads that a thread pool can accommodate private volatile int maximumPoolSize; // New threads need to wait after the thread pool reaches the threshold private volatile long keepAliveTime; // Create a new thread in factory mode private volatile ThreadFactory threadFactory; // context private final AccessControlContext acc; // Blocking Queue private final BlockingQueue<Runnable> workQueue; // Rejection Policy private volatile RejectedExecutionHandler handler; /** * ThreadPoolExecutor Core Constructor */ public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) { if (corePoolSize < 0 || maximumPoolSize <= 0 || maximumPoolSize < corePoolSize || keepAliveTime < 0) throw new IllegalArgumentException(); if (workQueue == null || threadFactory == null || handler == null) throw new NullPointerException(); this.acc = System.getSecurityManager() == null ? null : AccessController.getContext(); this.corePoolSize = corePoolSize; this.maximumPoolSize = maximumPoolSize; this.workQueue = workQueue; this.keepAliveTime = unit.toNanos(keepAliveTime); this.threadFactory = threadFactory; this.handler = handler; } }

Manage Task Queue BlockingQueue

ThreadPoolExecutor allows you to provide a BlockingQueue to hold tasks waiting to be executed.There are three basic task queuing methods: bounded queue, bounded queue, and synchronous handoff.

Unbounded Queue: Unbounded LinkedBlockingQueue is a common queue with unlimited size. Use this queue as a blocking queue with special care. A large number of new tasks may accumulate in the queue and eventually cause OOM when the task takes a long time.

Reading the code, Executors.newFixedThreadPool uses LinkedBlockingQueue, which is the pit the landlord stepped into. When QPS is high and sending data is large, a large number of tasks are added to this unbounded LinkedBlockingQueue, causing the cpu and memory to soar and the server to hang up.

Bounded queues: There are two common types,

One is a FIFO-compliant queue such as ArrayBlockingQueue and a bounded LinkedBlockingQueue.
Another type is a priority queue such as PriorityBlockingQueue.Priority in Priority BlockingQueue is determined by the Comparator of the task.
With bounded queues, the queue size needs to match the thread pool size. With smaller bounded queues and larger thread pools, memory consumption is reduced, cpu usage and context switching are reduced, but system throughput may be limited.In our repair scenario, this type of queue is chosen. Although some tasks will be lost, we are sorting log collection tasks online, so partial pair loss is tolerated.

Synchronous handover queue: If you do not want tasks to wait in the queue but want to hand them over directly to the worker thread, you can use SynchronousQueue as the waiting queue.SynchronousQueue is not a real queue, but a mechanism for handing over between threads.To place an element in a SynchronousQueue, another thread must be waiting to receive it.This queue is recommended only when using an unbounded thread pool or when there is a saturation policy.

Saturation Policy RejectedExecutionHandler

ThreadPoolExecutor provides the following four saturation strategies:

CallerRunsPolicy handles the task by the calling thread (the thread submitting the task)
AbortPolicy discards the task and throws the RejectedExecutionException exception directly (default thread pool rejection policy)
DiscardPolicy only discards tasks and does not throw exceptions
DiscardOldestPolicy discards the top task in the queue and resubmits the rejected task

Customize the saturation policy by implementing the RejectedExecutionHandler interface and overriding the void rejectedExecution(Runnable r, ThreadPoolExecutor executor) method

public class ThreadPoolExecutor{ /** * Default thread pool rejection policy AbortPolicy */ private static final RejectedExecutionHandler defaultHandler = new AbortPolicy(); /* ThreadPoolExecutor Provide a rejection policy as follows: */ /** * The task is handled by the calling thread (the thread submitting the task) */ public static class CallerRunsPolicy implements RejectedExecutionHandler {} /** * Discard task and throw RejectedExecutionException directly */ public static class AbortPolicy implements RejectedExecutionHandler {} /** * Drop tasks only and do not throw exceptions */ public static class DiscardPolicy implements RejectedExecutionHandler {} /** * Discard the top task in the queue and resubmit the rejected task */ public static class DiscardOldestPolicy implements RejectedExecutionHandler {} }

Executors (not recommended)

As mentioned in the Alibaba Java Development Manual, creating a thread pool with Executors may result in OOM(OutOfMemory, memory overflow)

ExecutorService

public interface ExecutorService extends Executor { void shutdown(); List<Runnable> shutdownNow(); boolean isShutdown(); boolean isTerminated(); boolean awaitTermination(long timeout, TimeUnit unit)throws InterruptedException; // .... Other methods for task submission }

To address the lifecycle issue of executing services,
ExecutorService extends the Executor interface and adds some methods for life cycle management.
The life cycle of ExecutorService has three states: running, shutting down, and terminated.
ExecutorService was running when it was initially created.
The shutdown method performs a gentle shutdown process: it stops accepting new tasks and waits for submitted tasks to complete, including those that have not yet started executing.
The shutdownNow method performs a rough shutdown process: it will attempt to cancel all running tasks and will no longer start tasks that have not yet started in the queue.

ThreadFactory

DefaultThreadFactory

/** * The default thread factory */ static class DefaultThreadFactory implements ThreadFactory { private static final AtomicInteger poolNumber = new AtomicInteger(1); private final ThreadGroup group; private final AtomicInteger threadNumber = new AtomicInteger(1); private final String namePrefix; DefaultThreadFactory() { SecurityManager s = System.getSecurityManager(); group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup(); namePrefix = "pool-" + poolNumber.getAndIncrement() + "-thread-"; } public Thread newThread(Runnable r) { Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0); if (t.isDaemon()) t.setDaemon(false); if (t.getPriority() != Thread.NORM_PRIORITY) t.setPriority(Thread.NORM_PRIORITY); return t; } }

PrivilegedThreadFactory

/** * Permission Access and Class Loading */ static class PrivilegedThreadFactory extends DefaultThreadFactory { private final AccessControlContext acc; private final ClassLoader ccl; PrivilegedThreadFactory() { super(); SecurityManager sm = System.getSecurityManager(); if (sm != null) { // Calls to getContextClassLoader from this class // never trigger a security check, but we check // whether our callers have this permission anyways. sm.checkPermission(SecurityConstants.GET_CLASSLOADER_PERMISSION); // Fail fast sm.checkPermission(new RuntimePermission("setContextClassLoader")); } this.acc = AccessController.getContext(); this.ccl = Thread.currentThread().getContextClassLoader(); } public Thread newThread(final Runnable r) { return super.newThread(new Runnable() { public void run() { AccessController.doPrivileged(new PrivilegedAction<Void>() { public Void run() { Thread.currentThread().setContextClassLoader(ccl); r.run(); return null; } }, acc); } }); } }

ThreadFactoryBuilder using guava

public class ThreadFactoryBuilder{ private static ThreadFactory doBuild(ThreadFactoryBuilder builder) { final String nameFormat = builder.nameFormat; final Boolean daemon = builder.daemon; final Integer priority = builder.priority; final UncaughtExceptionHandler uncaughtExceptionHandler = builder.uncaughtExceptionHandler; final ThreadFactory backingThreadFactory = (builder.backingThreadFactory != null) ? builder.backingThreadFactory : Executors.defaultThreadFactory(); final AtomicLong count = (nameFormat != null) ? new AtomicLong(0) : null; return new ThreadFactory() { @Override public Thread newThread(Runnable runnable) { Thread thread = backingThreadFactory.newThread(runnable); if (nameFormat != null) { thread.setName(format(nameFormat, count.getAndIncrement())); } if (daemon != null) {// Daemon Threads thread.setDaemon(daemon); } if (priority != null) {// priority thread.setPriority(priority); } if (uncaughtExceptionHandler != null) { thread.setUncaughtExceptionHandler(uncaughtExceptionHandler); } return thread; } }; } }

Create the correct posture for the thread pool

/** * @Auther: Noseparte * @Date: 2019/11/27 10:35 * @Description: * * <p>Custom Protocol Gateway Thread Pool </p> */ public class ThreadPool { protected final static Logger _LOG = LogManager.getLogger(ThreadPool.class); private List<ExecutorService> workers = new ArrayList<>(); private int threadCount; private ThreadFactory threadFactory; public ThreadPool(int threadCount) { this(threadCount, new UserThreadFactory("Gateway Game Logic Protocol Thread Pool")); } public ThreadPool(int threadCount, ThreadFactory threadFactory) { this.threadCount = threadCount; this.threadFactory = threadFactory; if (threadCount <= 0 || null == threadFactory) throw new IllegalArgumentException(); for (int i = 0; i < threadCount; i++) { workers.add(new ThreadPoolExecutor(threadCount, 200, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(1024), threadFactory, new ThreadPoolExecutor.AbortPolicy())); } } public Future execute(Runnable task, int mold) { int index = Math.abs(mold) % threadCount; ExecutorService executor = workers.get(index); if (null == executor) { _LOG.error("sid=" + mold + ", tid=" + index); return null; } return executor.submit(task); } public void shutdown() { int count = 0; for (ExecutorService worker : workers) { _LOG.error("close thread{}.", ++count); worker.shutdown(); } } static class UserThreadFactory implements ThreadFactory { private static final AtomicInteger poolNumber = new AtomicInteger(1); private final ThreadGroup group; private final AtomicInteger threadNumber = new AtomicInteger(1); private final String namePrefix; UserThreadFactory(String poolName) { SecurityManager s = System.getSecurityManager(); group = (s != null) ? s.getThreadGroup() : Thread.currentThread().getThreadGroup(); namePrefix = poolName + "-" + poolNumber.getAndIncrement() + "-thread-"; } public Thread newThread(Runnable r) { Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0); if (t.isDaemon()) t.setDaemon(false); if (t.getPriority() != Thread.NORM_PRIORITY) t.setPriority(Thread.NORM_PRIORITY); return t; } } }

summary

Considerations for creating thread pools:

Customize ThreadFactory, Saturation Policy, Task Queue, ThreadPoolExecutor to business scenarios
Note that the increasing number of task blockages in BlockingQueue can lead to memory depletion (OOM), so set an upper limit on the queue

Source address:
Almost-Famous: How the Gateway Thread Pool in Game was created