Redis sentry setup and SpringBoot integration

Due to business needs, Redis is used in the company to store tokens for user login authentication and permission verification. The original Redis scheme was a single Redis. There was a risk that the whole set of distributed microservices would not be available after the Redis node was down. Therefore, the technical selection of Redis high availability scheme was carried out. Considering the availability, concurrency and complexity of various schemes, the scheme of using Redis sentinel was finally selected.

1. Redis Sentinel principle

Simply paste a picture and don't talk about it in detail. This article mainly records the application process and doesn't involve too much principle explanation

2. The redis Sentry is built with one master and two slaves

2.1 environmental preparation

Linux server, Download Redis installation package

a. wget http://download.redis.io/releases/redis-5.0.5.tar.gz

b.decompression 
tar -xvf redis-5.0.5.tar.gz
cd redis-5.0.5
make
make install

2.2 Redis service setup

Master redis profile

# redis_master.conf

protected-mode no
port 6380
requirepass "${redis password}"
daemonize yes
# Create the corresponding folder first
dir "/data/redis/redis_slave_one"
logfile "/data/redis/redis_slave_one.log"

masterauth "${redis password}"
client-output-buffer-limit normal 0 0 0
# Generated by CONFIG REWRITE
client-output-buffer-limit replica 512mb 128mb 120
replica-read-only no

Secondary slave configuration file

# redis_slave_one.conf

protected-mode no
port 6381
requirepass ${redis password}
daemonize yes
# Create the corresponding folder first
dir "/data/redis/redis_slave_two"
logfile "/data/redis/redis_slave_two.log"
replicaof ${ip} 6380
masterauth ${redis password}
client-output-buffer-limit replica 512mb 128mb 120
replica-read-only no
--------------------------------------------
# redis_slave_two.conf

protected-mode no
port 6382
requirepass "${redis password}"
daemonize yes
# Create the corresponding folder first
dir "/data/redis/redis_master"
logfile "/data/redis/redis_master.log"
masterauth "${redis password}"
client-output-buffer-limit normal 0 0 0
# Generated by CONFIG REWRITE
replicaof ${ip} 6380
client-output-buffer-limit replica 512mb 128mb 120
replica-read-only no

Start primary redis:
/usr/local/redis-5.0.5/src/redis-server /etc/redis/redis_master.conf

Start two slave:
/usr/local/redis-5.0.5/src/redis-server /etc/redis/redis_slave_one.conf
/usr/local/redis-5.05/src/redis-server /etc/redis/redis_slave_two.conf

2.3 construction of redis Sentinel

Sentinel profile

# master's daemon profile

port 26380
# Create the corresponding folder first
dir "/data/redis/master_sentinel"
# sentinel post startup log file
logfile "/data/redis/sentinel/master_sentinel.log"
# Whether to start sentinel as a background application. The default is no. change it to yes to start in the background
daemonize yes
# Format: sentinel <option_name> <master_name> <option_value>;#The meaning of this line is: the name of the monitored master is mymaster (custom), and the address is 127.0.0.1:6378. The last 1 at the end of the line represents how many sentinels in the sentinel cluster think the master is dead before they can really think that the master is unavailable. This value needs to be less than the current number of sentinels (cardinality), Otherwise, start sentinel and always prompt + sentinel address switch information.
sentinel monitor mymaster ${ip} 6380 2

# The sentinel will send a heartbeat PING to the master to confirm whether the master is alive. If the master does not respond to the PONG within a "certain time range" or replies to an error message, the sentinel will subjectively (unilaterally) think that the master is no longer available (subjectively down, also referred to as SDOWN). The down after milliseconds is used to specify the "certain time range". The unit is milliseconds. The default is 30 seconds.
sentinel down-after-milliseconds mymaster 15000

# The expiration time of the failover. When the failover starts, no failover operation is triggered within this time. Currently, sentinel will consider the failover failed. The default is 180 seconds, or 3 minutes.
sentinel failover-timeout mymaster 120000

# During failover active / standby switching, this option specifies the maximum number of slaves that can synchronize the new master at the same time. The smaller the number, the longer it takes to complete the failover. However, if the number is larger, it means that more slaves are unavailable due to replication. You can set this value to 1 to ensure that only one slave is in a state that cannot process command requests at a time.
sentinel deny-scripts-reconfig yes

sentinel auth-pass mymaster ${redis password}
# slave_one's Guardian profile

port 26381
# Create the corresponding folder first
dir "/data/redis/slave_one_sentinel"
# sentinel post startup log file
logfile "/data/redis/sentinel/slave_one_sentinel.log"
# Whether to start sentinel as a background application. The default is no. change it to yes to start in the background
daemonize yes
# Format: sentinel <option_name> <master_name> <option_value>;#The meaning of this line is: the name of the monitored master is mymaster (custom), and the address is 127.0.0.1:6378. The last 1 at the end of the line represents how many sentinels in the sentinel cluster think the master is dead before they can really think that the master is unavailable. This value needs to be less than the current number of sentinels (cardinality), Otherwise, start sentinel and always prompt + sentinel address switch information.
sentinel monitor mymaster ${ip} 6380 2

# The sentinel will send a heartbeat PING to the master to confirm whether the master is alive. If the master does not respond to the PONG within a "certain time range" or replies to an error message, the sentinel will subjectively (unilaterally) think that the master is no longer available (subjectively down, also referred to as SDOWN). The down after milliseconds is used to specify the "certain time range". The unit is milliseconds. The default is 30 seconds.
sentinel down-after-milliseconds mymaster 15000

# The expiration time of the failover. When the failover starts, no failover operation is triggered within this time. Currently, sentinel will consider the failover failed. The default is 180 seconds, or 3 minutes.
sentinel failover-timeout mymaster 120000

# During failover active / standby switching, this option specifies the maximum number of slaves that can synchronize the new master at the same time. The smaller the number, the longer it takes to complete the failover. However, if the number is larger, it means that more slaves are unavailable due to replication. You can set this value to 1 to ensure that only one slave is in a state that cannot process command requests at a time.
sentinel deny-scripts-reconfig yes

sentinel auth-pass mymaster ${redis password}
# slave_one's Guardian profile

port 26382
# Create the corresponding folder first
dir "/data/redis/slave_two_sentinel"
# sentinel post startup log file
logfile "/data/redis/sentinel/slave_two_sentinel.log"
# Whether to start sentinel as a background application. The default is no. change it to yes to start in the background
daemonize yes
# Format: sentinel <option_name> <master_name> <option_value>;#The meaning of this line is: the name of the monitored master is mymaster (custom), and the address is 127.0.0.1:6378. The last 1 at the end of the line represents how many sentinels in the sentinel cluster think the master is dead before they can really think that the master is unavailable. This value needs to be less than the current number of sentinels (cardinality), Otherwise, start sentinel and always prompt + sentinel address switch information.
sentinel monitor mymaster ${ip} 6380 2

# The sentinel will send a heartbeat PING to the master to confirm whether the master is alive. If the master does not respond to the PONG within a "certain time range" or replies to an error message, the sentinel will subjectively (unilaterally) think that the master is no longer available (subjectively down, also referred to as SDOWN). The down after milliseconds is used to specify the "certain time range". The unit is milliseconds. The default is 30 seconds.
sentinel down-after-milliseconds mymaster 15000

# The expiration time of the failover. When the failover starts, no failover operation is triggered within this time. Currently, sentinel will consider the failover failed. The default is 180 seconds, or 3 minutes.
sentinel failover-timeout mymaster 120000

# During failover active / standby switching, this option specifies the maximum number of slaves that can synchronize the new master at the same time. The smaller the number, the longer it takes to complete the failover. However, if the number is larger, it means that more slaves are unavailable due to replication. You can set this value to 1 to ensure that only one slave is in a state that cannot process command requests at a time.
sentinel deny-scripts-reconfig yes

sentinel auth-pass mymaster ${redis password}

Start the sentry of the main Redis:
/user/local/redis-5.0.5/src/redis-sentinel /etc/redis/redis_master_sentinel.conf
You can also:
/user/local/redis-5.0.5/src/redis-server /etc/redis/redis_master_sentinel.conf --sentinel
Start two sentinels from Redis:
/usr/local/redis-5.0.5/src/redis-sentinel /etc/redis/redis_slave_one_sentinel.conf
/usr/local/redis-5.0.5/src/redis-sentinel /etc/redis/redis_slave_two_sentinel.conf
Look at the results:
The first is whether the service is started normally:

Then there is one master and two slaves:

Finally, whether the sentry has monitored redis:

ok, no problem. Go to the next step.

2.4 test master-slave replication

Add a key to the main library

Check whether there are records from the Library:

ok, no problem.
The redis connection tool used is RedisInsight. If necessary, you can download one from the official website Redis connection tool download

2.5 test automatic switching between master and slave Libraries

After verification, there is no problem. Continue to the next step (remember to restart the 6380 service).

3. SpringBoot integration

3.1 pom dependency

<!--The version number is based on my own springboot If there is a problem, you can see if it does not match your version-->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-redis</artifactId>
			<version>2.2.8.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-pool2</artifactId>
			<version>2.6.1</version>
		</dependency>

3.2 yml file configuration

spring:
  data:
    redis:
      repositories:
        enabled: false
  redis:
    #Cluster mode configuration start
    password: ${password}
    sentinel:
      master: mymaster
      nodes: ${ip}:26382,${ip}:26380,${ip}:26381 # Sentinel's IP:Port list
    #Cluster mode configuration stop
    #Configuration of stand-alone mode start
#    host: ${ip}
#    password: ${password}
#    port: 6379
    #Configuration stop in stand-alone mode
    timeout: 5000
    database: 0
    lettuce:
      pool:
        max-active: 8
        max-wait: -1
        max-idle: 8
        min-idle: 0
    myconfig:
      time-to-live: 86400

3.3 serialized configuration injection

Annotate @ EnableCaching on startup class

@EnableCaching
@SpringBootApplication
public class xxxApplication {

	public static void main(String[] args) {
		SpringApplication.run(xxxApplication.class, args);
	}

}

Write a configuration class

package xx;

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CachingConfigurerSupport;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cache.interceptor.KeyGenerator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.*;
import org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.RedisSerializationContext;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

import javax.annotation.Resource;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.time.Duration;

/**
 * Cache configuration class
 * @author xxx
 * @Date 20xx-xx-xx xx:xx
 **/
@Configuration
@EnableCaching
public class RedisConfig extends CachingConfigurerSupport {
    private static final Logger logger = LoggerFactory.getLogger(RedisConfig.class);
    @Resource
    private RedisConnectionFactory factory;
    /**
     * The default is two hours
     */
    private static final long DURATION_SECOND_7200 = 7200L;
    private static final long DURATION_SECOND_300 = 300L;

    @Override
    @Bean
    public KeyGenerator keyGenerator() {
        return new KeyGenerator() {
            @SuppressWarnings("rawtypes")
            @Override
            public Object generate(Object target, Method method, Object... params) {
                StringBuilder sb = new StringBuilder(RedisAutoCacheValue.AUTO_KEY_PREFIX);
                if(target instanceof Proxy) {
                	//If it is a proxy class
                	Class[] i = target.getClass().getInterfaces();
                	if(i != null  && i.length > 0) {
                		//Just take the first one
                		sb.append(i[0].getName());
                	}else {
                		sb.append(target.getClass().getName());
                	}
                } else if(target instanceof org.springframework.cglib.proxy.Factory){
                    //If it is a cglib agent, you need to manually remove the following $$
                    String className = target.getClass().getName();
                    sb.append(className, 0, className.indexOf("$$"));
                } else {
                	sb.append(target.getClass().getName());
                }
                sb.append(".");
                sb.append(method.getName());
                sb.append("_");
                for (Object obj : params) {
                    if (obj != null) {
                        Class cls = obj.getClass();
                        if (cls.isArray()) {
                            //For basic data processing
                            logger.info("keyGenerator : {}", cls.getComponentType());
                            if (cls.isAssignableFrom(long.class)) {
                                long[] ay = (long[]) obj;
                                for (long o : ay) {
                                    sb.append(o).append("");
                                }
                            } else if (cls.isAssignableFrom(int.class)) {
                                int[] ay = (int[]) obj;
                                for (int o : ay) {
                                    sb.append(o).append("");
                                }
                            } else if (cls.isAssignableFrom(float.class)) {
                                float[] ay = (float[]) obj;
                                for (float o : ay) {
                                    sb.append(o).append("");
                                }
                            } else if (cls.isAssignableFrom(double.class)) {
                                double[] ay = (double[]) obj;
                                for (double o : ay) {
                                    sb.append(o).append("");
                                }
                            } else if (cls.isAssignableFrom(String.class)) {
                                String[] ay = (String[]) obj;
                                for (String o : ay) {
                                    sb.append(o).append("");
                                }
                            } else {
                                sb.append(obj.toString());
                            }
                            //TODO handles other types of arrays
                        } else {
                            sb.append(obj.toString());
                        }
                    } else {
                        sb.append("null");
                    }
                    sb.append("_");
                    //sb.append(obj == null ? "null" : obj.toString());
                }
                sb.delete(sb.length()-1, sb.length());
                return sb.toString();
            }

        };

    }

    /**
     * The default cache management is to store the cache with long aging
     * @param redisTemplate
     * @return
     */
    @SuppressWarnings({"rawtypes", "Duplicates"})
    @Primary
    @Bean
    public CacheManager cacheManager(RedisTemplate redisTemplate) {
        RedisCacheConfiguration config = RedisCacheConfiguration
                .defaultCacheConfig()
                //Expiration time
                .entryTtl(Duration.ofSeconds(DURATION_SECOND_7200))
                //Do not cache null values
                //.disableCachingNullValues()
                //Make it clear that serialization in manager is the same as template to prevent inexplicable problems
                .serializeKeysWith(RedisSerializationContext.SerializationPair.fromSerializer(this.keySerializer()))
                .serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(this.valueSerializer()));

        RedisCacheManager rcm = RedisCacheManager.builder(redisTemplate.getConnectionFactory()).cacheDefaults(config).transactionAware().build();
        return rcm;
    }

    /**
     * Store cache with short aging (5 minutes)
     * @param redisTemplate
     * @return
     */
    @SuppressWarnings({"rawtypes", "Duplicates"})
    @Bean
    public CacheManager cacheManagerIn5Minutes(RedisTemplate redisTemplate) {
        RedisCacheConfiguration config = RedisCacheConfiguration
                .defaultCacheConfig()
                //Expiration time
                .entryTtl(Duration.ofSeconds(DURATION_SECOND_300))
                //Do not cache null values
                //.disableCachingNullValues()
                //Make it clear that serialization in manager is the same as template to prevent inexplicable problems
                .serializeKeysWith(RedisSerializationContext.SerializationPair.fromSerializer(this.keySerializer()))
                .serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(this.valueSerializer()));

        RedisCacheManager rcm = RedisCacheManager.builder(redisTemplate.getConnectionFactory()).cacheDefaults(config).transactionAware().build();
        return rcm;
    }

    /*@SuppressWarnings({"rawtypes", "unchecked"})
    @Bean
    public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
    	//factory = connectionFactory(3,"172.20.11.134",6379,"123456",2000,100,1,1000,2000);
        StringRedisTemplate template = new StringRedisTemplate(factory);

        template.setKeySerializer(keySerializer());
        template.setHashKeySerializer(keySerializer());
        template.setValueSerializer(valueSerializer());
        template.setHashValueSerializer(valueSerializer());
        template.afterPropertiesSet();
        return template;
    }
*/
    private RedisSerializer<String> keySerializer() {
        return new StringRedisSerializer();
    }
    private RedisSerializer<Object> valueSerializer() {
//        Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
//        ObjectMapper om = new ObjectMapper();
//        om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
//        om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
//        //Skip mismatched attributes
//        om.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
//
//        jackson2JsonRedisSerializer.setObjectMapper(om);
//        return jackson2JsonRedisSerializer;

        ObjectMapper om = new ObjectMapper();
        om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        //Skip mismatched attributes
        om.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

        GenericJackson2JsonRedisSerializer genericJackson2JsonRedisSerializer = new GenericJackson2JsonRedisSerializer(om);
        return genericJackson2JsonRedisSerializer;
    }

// ==========Note: the above is used to adapt the configuration of Cacheable cache annotation, and customize the cache type and duration=============================================

// ==========Manual: the following is used to adapt the original Redis configuration and add Redis cache manually. Now Redis is made into a new version of cache configuration==================

    @Bean
    public RedisTemplate<String, Object> redisTemplate() {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashValueSerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(new StringRedisSerializer());
        redisTemplate.setConnectionFactory(factory);
        return redisTemplate;
    }
    @Bean
    public HashOperations<String, String, Object> hashOperations(RedisTemplate<String, Object> redisTemplate) {
        return redisTemplate.opsForHash();
    }

    @Bean
    public ValueOperations<String, Object> valueOperations(RedisTemplate<String, Object> redisTemplate) {
        return redisTemplate.opsForValue();
    }

    @Bean
    public ListOperations<String, Object> listOperations(RedisTemplate<String, Object> redisTemplate) {
        return redisTemplate.opsForList();
    }

    @Bean
    public SetOperations<String, Object> setOperations(RedisTemplate<String, Object> redisTemplate) {
        return redisTemplate.opsForSet();
    }

    @Bean
    public ZSetOperations<String, Object> zSetOperations(RedisTemplate<String, Object> redisTemplate) {
        return redisTemplate.opsForZSet();
    }
}

3.4 testing

Write an interface:

    @GetMapping("/test/redisSentenel")
    @Cacheable(cacheNames = "redisSentenel")
    public ResponseJSON statisticByLevel(@RequestParam(value = "year", required = false) Integer year,
                                         @RequestParam(value = "quarter", required = false) Integer quarter)  {
        int num = Math.random();
        return num;
    }

Then call the interface to see if the data is stored in the redis and call again to see whether the results are consistent before and after. If it is consistent, it will explain the second call to the data that is taken directly from the cache instead of re generating a random number.
Over!!!

Reference articles
Redis annotation usage
Redis start, stop and redis command line operations
Redis tutorial
redis sentinel mode setup
The most complete Redis high availability technology solution in history
Learn how to build a Redis Sentinel cluster
Spring Boot (XIII): integrating Redis sentinel and cluster mode Practice
How can Spring Boot quickly integrate Redis sentinel?

Tags: Operation & Maintenance Database Redis

Posted on Sun, 26 Sep 2021 01:58:43 -0400 by yendor