1. Background
Most systems read more than write less. To reduce the pressure on the database, you can create multiple slave libraries from the main library, synchronize data from the main library automatically, send write operations to the main library in the program, and send read operations to the slave library to execute.
Today's main goal is to achieve read-write separation through spring.
Read-write separation requires the following two functions:
1. The method of reading, which is controlled by the caller
2. Transactional methods, all internal read and write operations are in the main library
2. Think about three questions
1. The method of reading is controlled by the caller whether it is a slave library or a master library. How can it be implemented?
You can add a parameter to all methods of reading to control whether the read slave library or the main library.
2. How is the data source routed?
An abstract class is provided in the spring-jdbc package: AbstractRoutingDataSource, which implements the javax.sql.DataSource interface. We use this class as a data source class. The key point is that this class can be used as a routing for data sources, multiple real data sources can be configured internally, and the developer decides which data source to use ultimately.
AbstractRoutingDataSource has a map for storing multiple target data sources
private Map<Object, DataSource> resolvedDataSources;
For example, a master-slave library can be stored like this
resolvedDataSources.put("master",Primary Database Data Source); resolvedDataSources.put("salave",From Library Data Source);
There is also an abstract method, determineCurrentLookupKey, in AbstractRoutingDataSource that uses the return value of this method as the key to find the corresponding data source in the resolvedDataSources above and as the data source for the current operation db
protected abstract Object determineCurrentLookupKey();
3. Where is the read-write separation controlled?
Read-write separation is a common function, which can be achieved by spring's aop. Add an interceptor. Before intercepting the target method, before the target method is executed, get which library you need to go to, store this flag in ThreadLocal, and use this flag as
The return value of the AbstractRoutingDataSource.determineCurrentLookupKey() method, which is cleared from ThreadLocal by the interceptor after the target method has been executed.
3. Code implementation
3.1. Engineering Structure Drawing
3.2,DsType
Represents the type of data source with two values to distinguish between primary and secondary libraries.
package com.javacode2018.readwritesplit.base; public enum DsType { MASTER, SLAVE; }
3.3,DsTypeHolder
There is a ThreadLocal inside to record whether the current master library or slave library, place this flag in the dsTypeThreadLocal
package com.javacode2018.readwritesplit.base; public class DsTypeHolder { private static ThreadLocal<DsType> dsTypeThreadLocal = new ThreadLocal<>(); public static void master() { dsTypeThreadLocal.set(DsType.MASTER); } public static void slave() { dsTypeThreadLocal.set(DsType.SLAVE); } public static DsType getDsType() { return dsTypeThreadLocal.get(); } public static void clearDsType() { dsTypeThreadLocal.remove(); } }
3.4, IService interface
This interface acts as a flag, and when a class needs to enable read-write separation, it needs to be implemented, and the classes that implement this interface are intercepted by the read-write separation interceptor.
package com.javacode2018.readwritesplit.base; //A service that requires read-write separation needs to implement the interface public interface IService { }
3.5,ReadWriteDataSource
Read-write detached data source, inherit ReadWriteDataSource, note its internal
The determineCurrentLookupKey method, which gets the flag from the ThreadLocal above whether you currently need to go from the main library or from the library.
package com.javacode2018.readwritesplit.base; import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource; import org.springframework.lang.Nullable; public class ReadWriteDataSource extends AbstractRoutingDataSource { @Nullable @Override protected Object determineCurrentLookupKey() { return DsTypeHolder.getDsType(); } }
3.6,ReadWriteInterceptor
Read-write detachment interceptors, which need to be executed in front of transaction interceptors, are set to Integer.MAX_VALUE-2 by @1 code, and later we set the order of transaction interceptors to Integer.MAX_VALUE-1. Transaction interceptors are executed in small order, so ReadWriteInterceptor will execute in transaction interceptorsPreviously executed by org.springframework.transaction.interceptor.TransactionInterceptor.
Since there are calls to each other in business methods, such as service2.m2 in service1.m1 and service2.m3 in service2.m2, we only need to get the specific data source to use before the M1 method can be executed, so the code below will record whether the master library or the slave library is taken when the interceptor first enters.
The following method takes the last parameter of the current target method, which can be of type DsType, through which the developer can control whether the master or slave library is taken.
package com.javacode2018.readwritesplit.base; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.aspectj.lang.annotation.Pointcut; import org.springframework.core.annotation.Order; import org.springframework.stereotype.Component; import java.util.Objects; @Aspect @Order(Integer.MAX_VALUE - 2) //@1 @Component public class ReadWriteInterceptor { @Pointcut("target(IService)") public void pointcut() { } //Gets the last parameter of the current target method private Object getLastArgs(final ProceedingJoinPoint pjp) { Object[] args = pjp.getArgs(); if (Objects.nonNull(args) && args.length > 0) { return args[args.length - 1]; } else { return null; } } @Around("pointcut()") public Object around(final ProceedingJoinPoint pjp) throws Throwable { //Determine if this is the first time in, for handling transaction nesting boolean isFirst = false; try { if (DsTypeHolder.getDsType() == null) { isFirst = true; } if (isFirst) { Object lastArgs = getLastArgs(pjp); if (DsType.SLAVE.equals(lastArgs)) { DsTypeHolder.slave(); } else { DsTypeHolder.master(); } } return pjp.proceed(); } finally { //On exit, clean up if (isFirst) { DsTypeHolder.clearDsType(); } } } }
3.7,ReadWriteConfiguration
spring configuration class, role
1, @3: Used to register some classes in the com.javacode2018.readwritesplit.base package into the spring container, such as the interceptor ReadWriteInterceptor above
2, @1: turn on spring aop
3, @2: Turn on spring's ability to automatically manage transactions, @EnableTransactionManagement's order is used to specify transaction interceptors
org.springframework.transaction.interceptor.TransactionInterceptor order, where we set the order to Integer.MAX_VALUE-1 and the order of ReadWriteInterceptor above is Integer.MAX_VALUE-2, so ReadWriteInterceptor executes before the transaction interceptor.
package com.javacode2018.readwritesplit.base; import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.EnableAspectJAutoProxy; import org.springframework.transaction.annotation.EnableTransactionManagement; @Configuration @EnableAspectJAutoProxy //@1 @EnableTransactionManagement(proxyTargetClass = true, order = Integer.MAX_VALUE - 1) //@2 @ComponentScan(basePackageClasses = IService.class) //@3 public class ReadWriteConfiguration { }
3.8,@EnableReadWrite
This annotation turns on read-write separation in two ways, @1 imports ReadWriteConfiguration into the spring container through @Import, which automatically enables read-write separation. Read-write separation is required in business, simply by adding the @EnableReadWrite annotation to the spring configuration class.
package com.javacode2018.readwritesplit.base; import org.springframework.context.annotation.Import; import java.lang.annotation.*; @Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME) @Documented @Import(ReadWriteConfiguration.class) //@1 public @interface EnableReadWrite { }
4. Cases
The key code for read-write separation has been written. Let's go to a case to verify the effect.
4.1. Executing sql scripts
Here are two databases: javacode2018_master and javacode2018_slave.
Both libraries create a t_user table, which inserts a single piece of data that you can use later to verify whether you are going from the primary library to the secondary library.
DROP DATABASE IF EXISTS javacode2018_master; CREATE DATABASE IF NOT EXISTS javacode2018_master; USE javacode2018_master; DROP TABLE IF EXISTS t_user; CREATE TABLE t_user ( id INT PRIMARY KEY AUTO_INCREMENT, name VARCHAR(256) NOT NULL DEFAULT '' COMMENT 'Full name' ); INSERT INTO t_user (name) VALUE ('master library'); DROP DATABASE IF EXISTS javacode2018_slave; CREATE DATABASE IF NOT EXISTS javacode2018_slave; USE javacode2018_slave; DROP TABLE IF EXISTS t_user; CREATE TABLE t_user ( id INT PRIMARY KEY AUTO_INCREMENT, name VARCHAR(256) NOT NULL DEFAULT '' COMMENT 'Full name' ); INSERT INTO t_user (name) VALUE ('slave library');
4.2, spring Configuration Class
@1: Enable read-write separation
masterDs() method: Define the primary database data source
slaveDs() method: Define a slave database data source
dataSource(): Defines read-write split routing data sources
There are two other methods to define JdbcTemplate and Transaction Manager, each of which restricts the injected bean name to dataSource by @Qualifier("dataSource"): i.e., injecting the read-write detached routing data source returned by dataSource() above.
package com.javacode2018.readwritesplit.demo1; import com.javacode2018.readwritesplit.base.DsType; import com.javacode2018.readwritesplit.base.EnableReadWrite; import com.javacode2018.readwritesplit.base.ReadWriteDataSource; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.datasource.DataSourceTransactionManager; import org.springframework.transaction.PlatformTransactionManager; import javax.sql.DataSource; import java.util.HashMap; import java.util.Map; @EnableReadWrite //@1 @Configuration @ComponentScan public class MainConfig { //Primary Database Data Source @Bean public DataSource masterDs() { org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource(); dataSource.setDriverClassName("com.mysql.jdbc.Driver"); dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_master?characterEncoding=UTF-8"); dataSource.setUsername("root"); dataSource.setPassword("root123"); dataSource.setInitialSize(5); return dataSource; } //From Library Data Source @Bean public DataSource slaveDs() { org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource(); dataSource.setDriverClassName("com.mysql.jdbc.Driver"); dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_slave?characterEncoding=UTF-8"); dataSource.setUsername("root"); dataSource.setPassword("root123"); dataSource.setInitialSize(5); return dataSource; } //Read-Write Separated Routing Data Source @Bean public ReadWriteDataSource dataSource() { ReadWriteDataSource dataSource = new ReadWriteDataSource(); //Set the primary library as the default, which is used when routing without finding a corresponding data source in the map of the datasource dataSource.setDefaultTargetDataSource(this.masterDs()); //Set up multiple target Libraries Map<Object, Object> targetDataSources = new HashMap<>(); targetDataSources.put(DsType.MASTER, this.masterDs()); targetDataSources.put(DsType.SLAVE, this.slaveDs()); dataSource.setTargetDataSources(targetDataSources); return dataSource; } //JdbcTemplate, dataSource is the injection read-write detached data source defined above @Bean public JdbcTemplate jdbcTemplate(@Qualifier("dataSource") DataSource dataSource) { return new JdbcTemplate(dataSource); } //Define the transaction manager, dataSource is the injection read-write detached data source defined above @Bean public PlatformTransactionManager transactionManager(@Qualifier("dataSource") DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } }
4.3,UserService
This class is equivalent to the service we usually write. I use JdbcTemplate to operate the database directly in it for the purpose of method. The real project operation db will be placed in dao.
getUserNameById method: Query name by id.
Insert method: insert data, all internal operations will take the main library, in order to verify whether queries will also take the main library, after inserting data, we will call this.userService.getUserNameById (id, DsType.SLAVE)Method to perform the query operation, the second parameter intentionally uses SLAVE, if the query has results, it means that the main library is going, otherwise it is going from the library, why do you need to call getUserNameById through this.userService?
this.userService is ultimately a proxy object, and access to its internal methods through the proxy object is blocked by read-write detached interceptors.
package com.javacode2018.readwritesplit.demo1; import com.javacode2018.readwritesplit.base.DsType; import com.javacode2018.readwritesplit.base.IService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.stereotype.Component; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; import java.util.List; @Component public class UserService implements IService { @Autowired private JdbcTemplate jdbcTemplate; @Autowired private UserService userService; @Transactional(propagation = Propagation.SUPPORTS, readOnly = true) public String getUserNameById(long id, DsType dsType) { String sql = "select name from t_user where id=?"; List<String> list = this.jdbcTemplate.queryForList(sql, String.class, id); return (list != null && list.size() > 0) ? list.get(0) : null; } //This insert method will take over the main library, and all internal operations will take over the main library @Transactional public void insert(long id, String name) { System.out.println(String.format("insert data", id, name)); this.jdbcTemplate.update("insert into t_user (id,name) values (?,?)", id, name); String userName = this.userService.getUserNameById(id, DsType.SLAVE); System.out.println("Query results:" + userName); } }
4.4. Test cases
package com.javacode2018.readwritesplit.demo1; import com.javacode2018.readwritesplit.base.DsType; import org.junit.Before; import org.junit.Test; import org.springframework.context.annotation.AnnotationConfigApplicationContext; public class Demo1Test { UserService userService; @Before public void before() { AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(); context.register(MainConfig.class); context.refresh(); this.userService = context.getBean(UserService.class); } @Test public void test1() { System.out.println(this.userService.getUserNameById(1, DsType.MASTER)); System.out.println(this.userService.getUserNameById(1, DsType.SLAVE)); } @Test public void test2() { long id = System.currentTimeMillis(); System.out.println(id); this.userService.insert(id, "Zhang San"); } }
The test1 method executes two queries, one for the primary library and one for the secondary library, and outputs:
master library slave library
Whether it's cool or not, it's up to the developer to decide whether to go from one library to another.
The results of test2 execution are as follows, and you can see that the data you just inserted is queried, indicating that all operations in insert are going through the main library.
1604905117467 insert data Query result: Zhang San
5. Case Source
git Address: https://gitee.com/javacode2018/spring-series This case corresponds to the source code: spring-series\lesson-004-readwritesplit
Let's star t. All the code series will be in this one.