Hand-on instructions for DB Read-Write Separation with Spring | 52 Spring Series

1. Background

Most systems read more than write less. To reduce the pressure on the database, you can create multiple slave libraries from the main library, synchronize data from the main library automatically, send write operations to the main library in the program, and send read operations to the slave library to execute.

Today's main goal is to achieve read-write separation through spring.

Read-write separation requires the following two functions:

1. The method of reading, which is controlled by the caller

2. Transactional methods, all internal read and write operations are in the main library

2. Think about three questions

1. The method of reading is controlled by the caller whether it is a slave library or a master library. How can it be implemented?

You can add a parameter to all methods of reading to control whether the read slave library or the main library.

2. How is the data source routed?

An abstract class is provided in the spring-jdbc package: AbstractRoutingDataSource, which implements the javax.sql.DataSource interface. We use this class as a data source class. The key point is that this class can be used as a routing for data sources, multiple real data sources can be configured internally, and the developer decides which data source to use ultimately.

AbstractRoutingDataSource has a map for storing multiple target data sources

private Map<Object, DataSource> resolvedDataSources;

For example, a master-slave library can be stored like this

resolvedDataSources.put("master",Primary Database Data Source);
resolvedDataSources.put("salave",From Library Data Source);

There is also an abstract method, determineCurrentLookupKey, in AbstractRoutingDataSource that uses the return value of this method as the key to find the corresponding data source in the resolvedDataSources above and as the data source for the current operation db

protected abstract Object determineCurrentLookupKey();

3. Where is the read-write separation controlled?

Read-write separation is a common function, which can be achieved by spring's aop. Add an interceptor. Before intercepting the target method, before the target method is executed, get which library you need to go to, store this flag in ThreadLocal, and use this flag as

The return value of the AbstractRoutingDataSource.determineCurrentLookupKey() method, which is cleared from ThreadLocal by the interceptor after the target method has been executed.

3. Code implementation

3.1. Engineering Structure Drawing

3.2,DsType

Represents the type of data source with two values to distinguish between primary and secondary libraries.

package com.javacode2018.readwritesplit.base;

public enum DsType {
    MASTER, SLAVE;
}

3.3,DsTypeHolder

There is a ThreadLocal inside to record whether the current master library or slave library, place this flag in the dsTypeThreadLocal

package com.javacode2018.readwritesplit.base;

public class DsTypeHolder {
    private static ThreadLocal<DsType> dsTypeThreadLocal = new ThreadLocal<>();

    public static void master() {
        dsTypeThreadLocal.set(DsType.MASTER);
    }

    public static void slave() {
        dsTypeThreadLocal.set(DsType.SLAVE);
    }

    public static DsType getDsType() {
        return dsTypeThreadLocal.get();
    }

    public static void clearDsType() {
        dsTypeThreadLocal.remove();
    }
}

3.4, IService interface

This interface acts as a flag, and when a class needs to enable read-write separation, it needs to be implemented, and the classes that implement this interface are intercepted by the read-write separation interceptor.

package com.javacode2018.readwritesplit.base;

//A service that requires read-write separation needs to implement the interface
public interface IService {
}

3.5,ReadWriteDataSource

Read-write detached data source, inherit ReadWriteDataSource, note its internal

The determineCurrentLookupKey method, which gets the flag from the ThreadLocal above whether you currently need to go from the main library or from the library.

package com.javacode2018.readwritesplit.base;

import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
import org.springframework.lang.Nullable;

public class ReadWriteDataSource extends AbstractRoutingDataSource {
    @Nullable
    @Override
    protected Object determineCurrentLookupKey() {
        return DsTypeHolder.getDsType();
    }
}

3.6,ReadWriteInterceptor

Read-write detachment interceptors, which need to be executed in front of transaction interceptors, are set to Integer.MAX_VALUE-2 by @1 code, and later we set the order of transaction interceptors to Integer.MAX_VALUE-1. Transaction interceptors are executed in small order, so ReadWriteInterceptor will execute in transaction interceptorsPreviously executed by org.springframework.transaction.interceptor.TransactionInterceptor.

Since there are calls to each other in business methods, such as service2.m2 in service1.m1 and service2.m3 in service2.m2, we only need to get the specific data source to use before the M1 method can be executed, so the code below will record whether the master library or the slave library is taken when the interceptor first enters.

The following method takes the last parameter of the current target method, which can be of type DsType, through which the developer can control whether the master or slave library is taken.

package com.javacode2018.readwritesplit.base;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

import java.util.Objects;

@Aspect
@Order(Integer.MAX_VALUE - 2) //@1
@Component
public class ReadWriteInterceptor {

    @Pointcut("target(IService)")
    public void pointcut() {
    }

    //Gets the last parameter of the current target method
    private Object getLastArgs(final ProceedingJoinPoint pjp) {
        Object[] args = pjp.getArgs();
        if (Objects.nonNull(args) && args.length > 0) {
            return args[args.length - 1];
        } else {
            return null;
        }
    }

    @Around("pointcut()")
    public Object around(final ProceedingJoinPoint pjp) throws Throwable {
        //Determine if this is the first time in, for handling transaction nesting
        boolean isFirst = false;
        try {
            if (DsTypeHolder.getDsType() == null) {
                isFirst = true;
            }
            if (isFirst) {
                Object lastArgs = getLastArgs(pjp);
                if (DsType.SLAVE.equals(lastArgs)) {
                    DsTypeHolder.slave();
                } else {
                    DsTypeHolder.master();
                }
            }
            return pjp.proceed();
        } finally {
            //On exit, clean up
            if (isFirst) {
                DsTypeHolder.clearDsType();
            }
        }
    }
}

3.7,ReadWriteConfiguration

spring configuration class, role

1, @3: Used to register some classes in the com.javacode2018.readwritesplit.base package into the spring container, such as the interceptor ReadWriteInterceptor above

2, @1: turn on spring aop

3, @2: Turn on spring's ability to automatically manage transactions, @EnableTransactionManagement's order is used to specify transaction interceptors

org.springframework.transaction.interceptor.TransactionInterceptor order, where we set the order to Integer.MAX_VALUE-1 and the order of ReadWriteInterceptor above is Integer.MAX_VALUE-2, so ReadWriteInterceptor executes before the transaction interceptor.

package com.javacode2018.readwritesplit.base;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.transaction.annotation.EnableTransactionManagement;

@Configuration
@EnableAspectJAutoProxy //@1
@EnableTransactionManagement(proxyTargetClass = true, order = Integer.MAX_VALUE - 1) //@2
@ComponentScan(basePackageClasses = IService.class) //@3
public class ReadWriteConfiguration {
}

3.8,@EnableReadWrite

This annotation turns on read-write separation in two ways, @1 imports ReadWriteConfiguration into the spring container through @Import, which automatically enables read-write separation. Read-write separation is required in business, simply by adding the @EnableReadWrite annotation to the spring configuration class.

package com.javacode2018.readwritesplit.base;

import org.springframework.context.annotation.Import;

import java.lang.annotation.*;

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Import(ReadWriteConfiguration.class) //@1
public @interface EnableReadWrite {
}

4. Cases

The key code for read-write separation has been written. Let's go to a case to verify the effect.

4.1. Executing sql scripts

Here are two databases: javacode2018_master and javacode2018_slave.

Both libraries create a t_user table, which inserts a single piece of data that you can use later to verify whether you are going from the primary library to the secondary library.

DROP DATABASE IF EXISTS javacode2018_master;
CREATE DATABASE IF NOT EXISTS javacode2018_master;

USE javacode2018_master;
DROP TABLE IF EXISTS t_user;
CREATE TABLE t_user (
  id   INT PRIMARY KEY       AUTO_INCREMENT,
  name VARCHAR(256) NOT NULL DEFAULT ''
  COMMENT 'Full name'
);

INSERT INTO t_user (name) VALUE ('master library');

DROP DATABASE IF EXISTS javacode2018_slave;
CREATE DATABASE IF NOT EXISTS javacode2018_slave;

USE javacode2018_slave;
DROP TABLE IF EXISTS t_user;
CREATE TABLE t_user (
  id   INT PRIMARY KEY       AUTO_INCREMENT,
  name VARCHAR(256) NOT NULL DEFAULT ''
  COMMENT 'Full name'
);
INSERT INTO t_user (name) VALUE ('slave library');

4.2, spring Configuration Class

@1: Enable read-write separation

masterDs() method: Define the primary database data source

slaveDs() method: Define a slave database data source

dataSource(): Defines read-write split routing data sources

There are two other methods to define JdbcTemplate and Transaction Manager, each of which restricts the injected bean name to dataSource by @Qualifier("dataSource"): i.e., injecting the read-write detached routing data source returned by dataSource() above.

package com.javacode2018.readwritesplit.demo1;

import com.javacode2018.readwritesplit.base.DsType;
import com.javacode2018.readwritesplit.base.EnableReadWrite;
import com.javacode2018.readwritesplit.base.ReadWriteDataSource;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;

import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;

@EnableReadWrite //@1
@Configuration
@ComponentScan
public class MainConfig {
    //Primary Database Data Source
    @Bean
    public DataSource masterDs() {
        org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
        dataSource.setDriverClassName("com.mysql.jdbc.Driver");
        dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_master?characterEncoding=UTF-8");
        dataSource.setUsername("root");
        dataSource.setPassword("root123");
        dataSource.setInitialSize(5);
        return dataSource;
    }

    //From Library Data Source
    @Bean
    public DataSource slaveDs() {
        org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
        dataSource.setDriverClassName("com.mysql.jdbc.Driver");
        dataSource.setUrl("jdbc:mysql://localhost:3306/javacode2018_slave?characterEncoding=UTF-8");
        dataSource.setUsername("root");
        dataSource.setPassword("root123");
        dataSource.setInitialSize(5);
        return dataSource;
    }

    //Read-Write Separated Routing Data Source
    @Bean
    public ReadWriteDataSource dataSource() {
        ReadWriteDataSource dataSource = new ReadWriteDataSource();
        //Set the primary library as the default, which is used when routing without finding a corresponding data source in the map of the datasource
        dataSource.setDefaultTargetDataSource(this.masterDs());
        //Set up multiple target Libraries
        Map<Object, Object> targetDataSources = new HashMap<>();
        targetDataSources.put(DsType.MASTER, this.masterDs());
        targetDataSources.put(DsType.SLAVE, this.slaveDs());
        dataSource.setTargetDataSources(targetDataSources);
        return dataSource;
    }

    //JdbcTemplate, dataSource is the injection read-write detached data source defined above
    @Bean
    public JdbcTemplate jdbcTemplate(@Qualifier("dataSource") DataSource dataSource) {
        return new JdbcTemplate(dataSource);
    }

    //Define the transaction manager, dataSource is the injection read-write detached data source defined above
    @Bean
    public PlatformTransactionManager transactionManager(@Qualifier("dataSource") DataSource dataSource) {
        return new DataSourceTransactionManager(dataSource);
    }
}

4.3,UserService

This class is equivalent to the service we usually write. I use JdbcTemplate to operate the database directly in it for the purpose of method. The real project operation db will be placed in dao.

getUserNameById method: Query name by id.

Insert method: insert data, all internal operations will take the main library, in order to verify whether queries will also take the main library, after inserting data, we will call this.userService.getUserNameById (id, DsType.SLAVE)Method to perform the query operation, the second parameter intentionally uses SLAVE, if the query has results, it means that the main library is going, otherwise it is going from the library, why do you need to call getUserNameById through this.userService?

this.userService is ultimately a proxy object, and access to its internal methods through the proxy object is blocked by read-write detached interceptors.

package com.javacode2018.readwritesplit.demo1;

import com.javacode2018.readwritesplit.base.DsType;
import com.javacode2018.readwritesplit.base.IService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

import java.util.List;

@Component
public class UserService implements IService {

    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Autowired
    private UserService userService;

    @Transactional(propagation = Propagation.SUPPORTS, readOnly = true)
    public String getUserNameById(long id, DsType dsType) {
        String sql = "select name from t_user where id=?";
        List<String> list = this.jdbcTemplate.queryForList(sql, String.class, id);
        return (list != null && list.size() > 0) ? list.get(0) : null;
    }

    //This insert method will take over the main library, and all internal operations will take over the main library
    @Transactional
    public void insert(long id, String name) {
        System.out.println(String.format("insert data{id:%s, name:%s}", id, name));
        this.jdbcTemplate.update("insert into t_user (id,name) values (?,?)", id, name);
        String userName = this.userService.getUserNameById(id, DsType.SLAVE);
        System.out.println("Query results:" + userName);
    }

}

4.4. Test cases

package com.javacode2018.readwritesplit.demo1;

import com.javacode2018.readwritesplit.base.DsType;
import org.junit.Before;
import org.junit.Test;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;

public class Demo1Test {

    UserService userService;

    @Before
    public void before() {
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
        context.register(MainConfig.class);
        context.refresh();
        this.userService = context.getBean(UserService.class);
    }

    @Test
    public void test1() {
        System.out.println(this.userService.getUserNameById(1, DsType.MASTER));
        System.out.println(this.userService.getUserNameById(1, DsType.SLAVE));
    }

    @Test
    public void test2() {
        long id = System.currentTimeMillis();
        System.out.println(id);
        this.userService.insert(id, "Zhang San");
    }
}

The test1 method executes two queries, one for the primary library and one for the secondary library, and outputs:

master library
slave library

Whether it's cool or not, it's up to the developer to decide whether to go from one library to another.

The results of test2 execution are as follows, and you can see that the data you just inserted is queried, indicating that all operations in insert are going through the main library.

1604905117467
 insert data{id:1604905117467, name:Zhang San}
Query result: Zhang San

5. Case Source

git Address:
https://gitee.com/javacode2018/spring-series

This case corresponds to the source code:
    spring-series\lesson-004-readwritesplit

Let's star t. All the code series will be in this one.

Tags: Java Spring

Posted on Thu, 23 Sep 2021 12:25:49 -0400 by Whack