one Introduction to multithreading
one point one Evolution of operating system
Batch operating system (single channel and multi-channel):
The computer can automatically and batch process the jobs of one or more users. However, when a single channel batch processing system performs I/O operations, CPU resources are not utilized. In order to make more efficient use of CPU resources, a multi-channel batch processing system was born.
Working diagram of single channel batch processing operating system:
Working diagram of multi-channel (two channel) batch processing operating system:
Time sharing operating system
Batch processing system is lack of interaction and independence between work for the computer system required by users. And be able to complete the tasks required by users within the expected time. The time-sharing operating system came into being. It supports multi-user interactive operating system. Each user can send various operation control commands to the system through his own terminal to complete the operation of the job. Time sharing is to divide the running time of cpu into short time slices, and allocate cpu resources to each task in turn.
Working diagram of time-sharing operating system:
Real time system: it is often used in aircraft flight, missile launch and other scenes, which will not be introduced in detail here.
1.2 relationship between thread and process
After introducing the operating system, let's look at processes and threads. With the development of computer hardware, the memory of computer is larger and larger, and the number of cpu is also more and more. Processes and threads are born naturally in order to make better use of computer resources.
Process is the basic unit of resource (CPU, memory, etc.) allocation. It is an instance of program execution. When the program runs, the system will create a process. The system will allocate independent memory address space to each process, and the addresses of each process will not interfere with each other. For a single core CPU, only one task will be executed at any time, and the parallel execution is completed by switching time slices. Multiple processes are executed alternately through the continuous switching of CPU time slices (it gives the impression that the application is carried out at the same time, so you can write code and listen to music without perception).
thread
Thread is the smallest unit of program execution. It is an execution flow of a process and the basic unit of CPU scheduling and dispatching. A process can be composed of many threads. Each thread will be responsible for an independent subtask. A process can only do one thing at a time. If you want to do multiple things at the same time, It is necessary to divide multiple sub tasks in the process into multiple threads and use threads to complete tasks more efficiently.
two Suitable scenario for multithreading
When do you need to use multithreading? For us to write programs, generally a main thread can handle all things, but when we need to optimize some time-consuming operations, we can consider multithreading.
- You need to fetch data from other rpc services many times. You can consider using multiple threads to obtain data, and then process them uniformly after they are returned.
- If you need to get data from multiple tables in the database, you can consider using multithreading to get data from the table and then process it.
- When we need to import a large table data, we can consider using multithreading to import data in batches, etc.
We need to pay attention when using multithreading ⚠️ Because multithreading is out of order, we need to make sure that parallel tasks are independent. If the tasks between two threads are not independent, you need to use thread safe variables to communicate before the thread.
three Multithreading practice (java)
3.1 let's first look at how to create a thread
By inheriting the Thread class
public class TestThread extends Thread {
public TestThread() {
}
@Override
public void run() {
System.out.print("The current execution thread is:" + this.getName() + " ");
}
public static void main(String[] args) {
System.out.println("The current thread is" + Thread.currentThread().getName());
TestThread testThread1 = new TestThread();
TestThread testThread2 = new TestThread();
testThread1.start();
testThread2.start();
}
}
Implement Runnable interface
public class TestRunnable implements Runnable {
private int count = 10;
@Override
public void run() {
for (int i = 0; i < 5; i++) {
System.out.println(Thread.currentThread().getName() + "function count= " + count--);
}
}
public static void main(String[] args) {
TestRunnable testRunnable = new TestRunnable();
new Thread(testRunnable, "C").start();
new Thread(testRunnable, "D").start();
}
}
Implement Callable interface
import java.util.concurrent.*;
public class TestCallable implements Callable<Integer> {
private final static ExecutorService executor = Executors.newCachedThreadPool();
private Integer number;
public TestCallable(Integer number) {
this.number = number;
}
@Override
public Integer call() throws Exception {
System.out.println("thread number=" + number);
return number;
}
public static void main(String[] args) {
executor.submit(new TestCallable(1));
}
}
After JDK 1.5, Callable and Future are provided, through which task execution results can be obtained after task execution. Examples of combined use of Callable and Future:
package com.bytedance.cg.robot.web.practise.multithreading;
import java.util.concurrent.*;
public class TestCallable implements Callable<Integer> {
private final static ExecutorService executor = Executors.newCachedThreadPool();
private Integer number;
public TestCallable(Integer number) {
this.number = number;
}
@Override
public Integer call() throws Exception {
System.out.println("thread number=" + number);
return number;
}
public static void main(String[] args) {
Callable<Integer> callable = () -> getDoubleNumber(100);
Future future = executor.submit(callable);
try {
System.out.println(future.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
private static Integer getDoubleNumber(Integer i) {
return i * 2;
}
}
Use of FutureTask
FutureTask is the implementation class of the Future interface. The construction parameters need to be passed into a Callable interface or the implementation class that implements Callable.
FutureTask class mainly implements the Future interface (you can get the return value of Callable as Future) and the Runnable interface (you can be executed by threads as Runnable).
package com.bytedance.cg.robot.web.practise.multithreading;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
public class TestCallable implements Callable<Integer> {
private final static ExecutorService executor = Executors.newCachedThreadPool();
private Integer number;
public TestCallable(Integer number) {
this.number = number;
}
@Override
public Integer call() {
System.out.println("thread number=" + number);
return number;
}
public static void main(String[] args) {
List<FutureTask<Integer>> futureTasks = new ArrayList<>();
for (int i = 1; i <= 3; i++) {
FutureTask<Integer> integerFutureTask = new FutureTask<>(new TestCallable(i));
futureTasks.add(integerFutureTask);
// Executing FutureTask with thread pool
executor.submit(integerFutureTask);
// Self built thread pool executes FutureTask
new Thread(new FutureTask<>(new TestCallable(i * 5)), "Thread with return value").start();
}
// Interface to get the return value (blocking the main thread)
try {
futureTasks.stream().forEach(task -> {
try {
System.out.println("Return value of child thread:" + task.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
);
} catch (Exception e) {
e.printStackTrace();
}
}
}
3.2 thread pool
Definition: Thread Pool is a tool to manage threads based on the idea of pooling. It often appears in multi-threaded servers, such as MySQL. Too many threads will bring additional overhead, including the overhead of creating and destroying threads, the overhead of scheduling threads, etc. at the same time, it also reduces the overall performance of the computer. The Thread Pool maintains multiple threads, waiting for the supervisor to assign tasks that can be executed concurrently. On the one hand, it avoids the cost of creating and destroying threads when processing tasks, on the other hand, it avoids the excessive scheduling problem caused by the expansion of the number of threads, and ensures the full utilization of the kernel.
Benefits:
- Reuse the created thread pool to avoid performance loss caused by thread creation and destruction. Thread pool can manage threads in an unlimited way. Creating threads leads to imbalance in resource scheduling. Threads can be allocated, tuned and monitored uniformly.
- When the task reaches, it is directly allocated to the existing threads in the thread pool for execution.
- Thread pool is extensible, allowing developers to add more functions to it. For example, the delayed timed thread pool ScheduledThreadPoolExecutor allows tasks to be delayed or executed periodically.
Introduction to thread pool:
The core implementation class of thread pool in Java is ThreadPoolExecutor.
Diagram of inherited classes of ThreadPoolExecutor:
Executor provides an idea: decouple task submission and task execution. Users do not need to pay attention to how to create threads and how to schedule threads to execute tasks. Users only need to provide Runnable objects to submit the running logic of tasks to the executor, and the executor framework completes the deployment of threads and the execution of tasks.
ExecutorService interface adds some capabilities: it expands the ability to execute tasks and supplements the methods that can generate Future for one or a batch of asynchronous tasks; It provides methods to control the thread pool, such as stopping the operation of the thread pool.
AbstractExecutorService is an abstract class of the upper layer, which connects the processes of executing tasks to ensure that the implementation of the lower layer only needs to focus on a method of executing tasks.
The lowest implementation class ThreadPoolExecutor implements the most complex running part. On the one hand, ThreadPoolExecutor will maintain its own life cycle, on the other hand, it will manage threads and tasks at the same time, so as to make a good combination of the two, so as to execute parallel tasks.
Operation mechanism of ThreadPoolExecutor:
Thread pool has three important elements: the number of core threads, the maximum number of threads, and the blocking queue.
Execution process of task scheduling:
There are many types of blocking queues: common blocking queues
ArrayBlockingQueue: a bounded blocking queue implemented by an array. The queue uses the first in first out (FIFO) principle to sort and add elements.
LinkedBlockingQueue: a bounded blocking queue implemented with a one-way linked list. The default and maximum length of this queue is Integer.MAX_VALUE. This queue sorts elements on a first in, first out basis. Pay attention to the length of the blocking queue. If it is not a special service, remember to define the queue capacity when using LinkedBlockingQueue.
Priority blocking queue: an unbounded blocking queue that supports priority.
DelayQueue: a delay unbounded blocking queue implemented using priority queue. Queues are implemented using PriorityQueue. The elements in the queue must implement the Delayed interface. When creating elements, you can specify how long it takes to get the current element from the queue. Elements can only be extracted from the queue when the delay expires.
Synchronous queue: a blocking queue that does not store elements, that is, a queue of single elements. Each put operation must wait for a take operation, otherwise you cannot continue to add elements. When the thread is idle for 60s, it will be recycled.
LinkedTransferQueue: an unbounded blocking queue composed of linked list structure (which implements the TransferQueue inherited from BlockingQueue) adopts a preemption mode. This means that when the consumer thread takes an element, if the queue is not empty, it directly takes the data. If the queue is empty, it generates a node (the node element is null) to join the queue. Then the consumer thread is waiting on this node. When the producer thread joins the queue, it finds a node with null element, and the producer thread does not join the queue, Directly fill the element into the node and wake up the waiting thread of the node. The awakened consumer thread takes the element and returns from the called method. We call this node operation "matching". The tryTransfer and transfer methods are overridden and have matching functions
LinkedBlockingDeque: a two-way blocking queue composed of linked list structure. When multithreading is concurrent, the contention lock is reduced.
Task rejection policy: (takes effect when the maximum capacity of blocking queue and the maximum number of threads in thread pool are reached)
Common ThreadPoolExecutor implementation of ExecutorService class
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
} public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}// This thread pool jdk8 appears. paramStream uses this thread pool.
public static ExecutorService newWorkStealingPool() {
return new ForkJoinPool
(Runtime.getRuntime().availableProcessors(),
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}3.3 ThreadPoolTaskExecutor of springboot thread pool
When we create a thread pool, we don't want the default thread pools of jdk ExecutorService class. We can select the ThreadPoolTaskExecutor of springboot to define the thread pool we need. Compared with ThreadPoolTaskExecutor, ThreadPoolExecutor adds the submitListenable method, which returns the ListenableFuture interface object and ListenableFuture interface object, and adds callback methods for success and failure after thread execution. This avoids the need for Future to call get in a blocking manner, and then execute successful and failed methods.
Class diagram structure of ListenableFuture
Compared with Future, ListenableFuture provides a callback method (addCallback) after thread execution, and the result will not block the main thread.
We can customize the number of core threads, the maximum number of threads, and the length of the blocking queue to initialize the bean and hand it over to the spring container for management. The thread pool is obtained through annotation later. Use posture
import org.springframework.beans.factory.annotation.Configurable;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;
@Configurable
@EnableAsync
public class ThreadPoolConfig {
@Bean("goodsWriteExecutor")
public Executor goodsWriteExecutor() {
return buildExecutor(40, 60, 100, 60, "goodsWriteExecutor");
}
@Bean("brandWriteExecutor")
public Executor brandWriteExecutor() {
return buildExecutor(40, 60, 100, 60, "brandWriteExecutor");
}
@Bean("supplierWriteExecutor")
public Executor supplierWriteExecutor() {
return buildExecutor(40, 60, 100, 60, "supplierWriteExecutor");
}
private Executor buildExecutor(int corePoolSize, int maxPoolSize, int queueCapacity, int keepAliveSeconds, String name) {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
//Setting thread pool parameter information
taskExecutor.setCorePoolSize(corePoolSize);
taskExecutor.setMaxPoolSize(maxPoolSize);
taskExecutor.setQueueCapacity(queueCapacity);
taskExecutor.setKeepAliveSeconds(keepAliveSeconds);
taskExecutor.setThreadNamePrefix(name);
taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
taskExecutor.setAwaitTerminationSeconds(60);
//Modify the reject policy to execute with the current thread
taskExecutor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//Initialize thread pool
taskExecutor.initialize();
return taskExecutor;
}
}
@Autowired
private Executor brandWriteExecutor; // Gets the thread pool from the spring container by name.3.4 usage of completable future and parallel stream.
Introduction to completable future:
Future has certain limitations. Before obtaining the thread execution result, our main thread get() needs to block and wait all the time to get the result, wasting cpu resources. And it is not convenient for us to do the choreography and combination of asynchronous computing.
Common methods of completable future:
runAsync asynchronous operation, return value is not supported
supplyAsync asynchronous operation, support return value
Exceptionally: when an exception occurs in the current task, execute the callback method in exceptionally.
whenComplete Consuming the processing results (including exceptions) of the previous task will not affect the results of the previous task. Although the returned value is actually the result of the previous task (whencomplete uses the main thread).
thenAccept consumes the processing result, receives the processing result of the task, and consumes the processing result. No result is returned.
thenApply Method, which consumes the processing result of the previous task and has a return value. If the task has an exception, the method will not be executed. (a self built thread pool or forkjoinpool is used, depending on the thread pool parameter of the completable future task.)
The handle and thenApply methods are handled in basically the same way. The difference is that the handle is executed after the task is completed. It can also handle abnormal tasks.
thenCombine merges tasks. thenCombine will execute both completion stage tasks, and then hand over the results of the two tasks to thenCombine for processing.
Thenpose method. Thenpose method allows you to pipeline two completionstages. When the first operation is completed, the result is passed to the second operation as a parameter.
allof can combine multiple completable future tasks, use stream to obtain results, and then use thenApply to execute business logic.
anyof can combine multiple completable future tasks. The combined completable future.get can obtain the results of the first completed task.
Completable future supports obtaining results through callback, and the main thread will not be blocked. Simple asynchronous processing demo
@SneakyThrows
@Override
public BaseUploadResponse upload(StoreUploadReq uploadReq) {
// 1 get the attribute information of api layer parsing
List<AttributeUploadTDTO> attributeUploadTDTOs = uploadReq.getAttributeUploadTDTOList();
List<UploadFailedResponse> failedResponses = attributeUploadTDTOs.parallelStream().map((attributeUploadTDTO) -> {
// 2. Perform parameter verification on the entity class (including the verification of association relationship) - > convert the entity class to the parameter type required for creation - > record the record of creation failure
UploadFailedResponse verifyResult = verifyUploadAttribute(attributeUploadTDTO, Integer.parseInt(attributeUploadTDTO.getRowNumber()),
uploadReq.getGoodsType());
// If the verification is successful, write directly with completabilefuture. Runasync()!!!
if (Objects.isNull(verifyResult)) {
CompletableFuture<Void> voidCompletableFuture =
CompletableFuture.runAsync(() -> attributeModifyServiceImpl.create(Convert2basicGoodsData(attributeUploadTDTO,
uploadReq.getGoodsType()), uploadReq.getCparam()), attrWriteExecutor);
// Because it is processed asynchronously, the thread may not be able to listen to the returned results of the asynchronous write thread, and the log may not be printed
if (voidCompletableFuture.isCompletedExceptionally()) {
log.error("create attribute error, AttributeUploadTDTO={}", JacksonUtils.marshalToString(attributeUploadTDTO));
return baseUploadServiceImpl.packFailedResponse(attributeUploadTDTO.getName(),
Integer.parseInt(attributeUploadTDTO.getRowNumber()), "Attribute name", Constant.SYSTEM_ERROR);
}
}
return verifyResult;
}).filter(Objects::nonNull).collect(Collectors.toList());Here, the incoming parameters will be verified. After the verification is passed, the write operation will be directly carried out with completable future. The write operation will be processed in other threads through the thread pool and will not block the progress of the main process
@SneakyThrows
private ResponseDTO<Object> parallelBatchHandle(StoreModuleTEnum moduleTEnum, StoreOperationTEnum operation,
GoodsTypeTEnum goodsTypeTEnum, List<String> ids) {
//Parallel mode is used here to improve efficiency
List<CompletableFuture<Pair<Long, StoreStandardRes>>> futures = ids.stream()
.map(Long::parseLong)
.map(id -> CompletableFuture.supplyAsync(() -> invoke(moduleTEnum, operation, goodsTypeTEnum, id), Executors.newCachedThreadPool())) //Note that using cachedThreadPool here may have the risk of memory leakage
.collect(Collectors.toList());
//Wait for all results to return for calculation
return sequence(futures)
.thenApply(pairList -> buildResponse(pairList, operation, moduleTEnum, goodsTypeTEnum))
.get(2000, TimeUnit.MILLISECONDS);
}
private <T> CompletableFuture<List<T>> sequence(List<CompletableFuture<T>> futures) {
CompletableFuture<Void> allDoneFuture = CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
return allDoneFuture.thenApply(v -> futures.stream().map(CompletableFuture::join).collect(Collectors.<T>toList()));
}It is a clever way to use completable future asynchronous processing, collect processing results with stream, allof combine all completable future, wait for all results to return, and then process.
parallelStream introduction:
Parallel stream is actually a stream executed in parallel. It improves the speed of multithreaded tasks through the default ForkJoinPool.
The default ForkJoinPool is used inside the parallel stream, and its default number of threads is the number of your processors (cpu cores). You can change the number of threads in ForkJoinPool by setting the variable [java.util.concurrent.ForkJoinPool.common.parallelism].
matters needing attention:
1: The java.util.concurrent.ForkJoinPool.common.parallelism variable is of type final and can only be set once in the entire JVM
2: Multiple parallel streams use the thread pool of the global ForkJoinPool. Try not to put IO operations into the pool, which will block other parallel streams.
The core of ForkJoinPool: Fork/Join framework
The core of Fork/Join framework is to use the idea of divide and conquer method to divide a large task into several independent subtasks, put these subtasks into different queues, and create a separate thread for each queue to execute the tasks in the queue. (you can execute the parent task after executing the sub task)
The Fork/Join framework uses the work stealing algorithm to run tasks, that is, when a thread processes tasks in its own work queue, it tries to steal a task from the work queue of other threads to execute until all tasks are processed. In order to reduce the competition between threads, the task will use a double ended queue. Steal the task queue and steal the task execution from the tail.
Simple usage of parallelStream:
List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5); numbers.parallelStream().forEach(System.out::println);