1. Build master-slave replication
- Environmental Science
Node name IP address master 192.168.3.11 slave1 192.168.3.12 slave2 192.168.3.13
1. Install redis
- All three servers need to be installed
#Turn off security policy systemctl stop firewalld systemctl disable firewalld setenforce 0
- Copy the following script
#!/bin/bash systemctl stop firewalld systemctl disable firewalld setenforce 0 yum -y install gcc gcc-c++ make cd /opt if [ ! -f " redis-5.0.7.tar.gz" ]; then wget http://download.redis.io/releases/redis-5.0.7.tar.gz fi tar zxvf redis-5.0.7.tar.gz -C /opt/ cd redis-5.0.7/ make make PREFIX=/usr/local/redis install yum -y install tcl expect cd /opt/redis-5.0.7/utils/ /usr/bin/expect <<EOF spawn ./install_server.sh expect "instance" {send "\r"} expect "config" {send "\r"} expect "log" {send "\r"} expect "data" {send "\r"} expect "executable" {send "/usr/local/redis/bin/redis-server\r"} expect "abort" {send "\r"} expect eof EOF ln -s /usr/local/redis/bin/* /usr/local/bin/ sed -i '/bind 127.0.0.1/c bind 0.0.0.0' /etc/redis/6379.conf sed -i 's/appendonly no/appendonly yes/' /etc/redis/6379.conf /etc/init.d/redis_6379 restart /etc/init.d/redis_6379 status netstat -natp | grep "redis" pgrep "redis" &> /dev/null if [ $? -eq 0 ];then echo -e "\033[46;37m Redis Service is running normally \033[0m" else echo -e "\033[46;37m Redis Service is running abnormally, please check \033[0m" fi sleep 2 echo ' ' echo -e "\033[46;37m Redis No password set, execute redis-cli Command Logon \033[0m"
2. Modify Redis Profile
- master node
vim /etc/redis/6379.conf bind 0.0.0.0 #70 lines, modified listening address 0.0.0.0 daemonize yes #Line 137, start daemon logfile /var/1og/redis_6379.1og #172 lines, specify log file directory dir /var/lib/redis/6379 #264 lines, specify working directory appendonly yes #700 lines, turn on AOF persistence /etc/init.d/redis_6379 restart #Restart service for configuration to take effect
- Slave1/2 Node
vim /etc/redis/6379.conf bind 0.0.0.0 #70 lines, modified listening address 0.0.0.0 daemonize yes #Line 137, start daemon logfile /var/log/redis_6379.log #172 lines, specify log file directory dir /var/lib/redis/6379 #264 lines, specify working directory replicaof 192.168.3.11 6379 #288 lines, specifying the Master node IP and port to synchronize appendonly yes #700 lines, turn on AOF persistence /etc/init.d/redis_6379 restart #Restart service for configuration to take effect
3. Verify the master-slave effect
- Primary Node View Log
tail -f /var/log/redis_6379.log
- Validation at Primary Node
[root@c7-1 ~]# redis-cli info replication # Replication role:master connected_slaves:2 slave0:ip=192.168.3.13,port=6379,state=online,offset=448,lag=1 slave1:ip=192.168.3.12,port=6379,state=online,offset=448,lag=1 master_replid:a1e500591376020abc8f501177a9d1bf8a506350 //A 40-bit hexadecimal random string generated at master startup to identify the master node master_replid2:0000000000000000000000000000000000000000 //Master node identity changes when switching master slave master_repl_offset:448 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:448
4. Error investigation
① WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128 The current maximum listening queue length per port does not meet this high load environment and needs to be adjusted Solution echo 2048 > /proc/sys/net/core/somaxconn ② WARNING overcommit_memory is set to 0! Background save may fail under low memory condition Memory Excess Warning, current memory set to 0 will cause background save to fail Solution: echo "vm.overcommit_memory=1" > /etc/sysctl.conf #Refresh profile to make it valid sysctl vm.overcommit_memory=1 ③ WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis Large transparent pages are enabled in the kernel ( THP)Support will result in Redis Delay and memory usage issues with Solve: echo never > /sys/kernel/mm/transparent_hugepage/enabled ④ Error condition on socket for SYNC: Connection reset by peer Connection refused because the primary server may be bound to itself IP address Solution Primary Node Profile bind 0.0.0.0
2. Set up Sentinel Mode
- Acts as the primary node automatic failover based on master-slave replication
1. Functions and Principles
1.1 Role of Sentry Mode
- Monitor
Sentinels constantly check whether the primary and secondary nodes are functioning properly - Automatic Failover
When the primary node is not functioning properly, the Sentry will start an automatic failover operation, which will upgrade one of the failed primary nodes to a new primary node and change the other secondary nodes to replicate the new primary node - notice
Sentinels can send failover results to clients
1.2 Principles of Sentry Mode
-
It is a distributed system for monitoring each server in the master-slave architecture, selecting a new Master through a voting mechanism and connecting all slaves to a new Master when a failure occurs
-
The number of entire Sentinel-running clusters must not be less than three nodes
1.3 Structure
-
The sentinel structure consists of two parts, the sentinel node and the data node:
-
Sentinel Nodes
A sentinel system consists of one or more sentinel nodes, which are special redis nodes and do not store data -
Data Node
Both primary and secondary nodes are data nodes
1.4 Work Process
-
Sentinel startup depends on master-slave mode, so you have to install master-slave mode before you go to sentry mode, so you need to deploy sentry mode on your node, which monitors whether all Redis working nodes are working properly
-
When a Master has a problem, because the other nodes are out of contact with the primary node, they vote, voting more than half to think that the Master is indeed a problem, and then notify the sentry that a Sentry will be selected to fail over (which slave will be designated by the sentry to make the new master). Then choose Slaves as the new Master filter by sending messages to each other and voting, with the majority of votes selected
-
It is important to note that an objective offline is a concept only available to the primary node, that is, if a failure occurs from a node and a sentinel node and the sentinel is subjectively offline, there will be no subsequent objective offline and failover operations (and the sentinel mode will only be responsible for aspects of Master, regardless of Slaves)
-
When a sentry finds that the master server has been dropped, it changes the master in SentinelRedistance s in master to SRI_S_DOWN (Subjective Offline) and notify other sentinels that they find the master has dropped the message sent by the other sentinels and attempt to connect to the master. If more than half (set in the configuration file) confirm that the master has been dropped, the master in Sentinel Redistance in the master will be changed to SRI_O_DOWN (objective offline)
2. Environment
Node name IP address master 192.168.3.11 slave1 192.168.3.12 slave2 192.168.3.13
3. Modify Sentry Profile
vim /opt/redis-5.0.7/sentinel.conf #17 lines, turn off protection mode protected-mode no #Line 21, Redis Sentry default listening port port 26379 #26 rows to start daemon daemonize yes #36 lines, specify log storage path logfile "/var/log/sentinel.log" #65 rows, specify the database store path dir "/var/lib/redis/6379" #84 lines, designate sentinel nodes #2 indicates that at least two Sentinel nodes are required to agree to fail over the primary node sentinel monitor mymaster 192.168.3.11 6379 2 #113 lines, determines the time period for the server to down load, defaulting to 30,000 milliseconds (30 seconds) sentinel down-after-milliseconds mymaster 3000 #146 lines with a maximum timeout of 180,000 (180 seconds) for the failing node sentinel failover-timeout mymaster 180000
4. Start Sentry Mode
- Start Primary Node First Start Slave Node
cd /opt/redis-5.0.7/ redis-sentinel sentinel.conf &
- View Sentry Information
[root@master redis-5.0.7]# redis-cli -p 26379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.226.128:6379,slaves=2,sentinels=1
- Analog failure
Simulated failure ( rm -rf /var/run/redis_6379.pid) #View the process number of redis-server [root@c7-1 redis-5.0.7]# netstat -anpt|grep redis tcp 0 0 0.0.0.0:26379 0.0.0.0:* LISTEN 9818/redis-sentinel tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 7616/redis-server 0 kill -9 7616 #Process number to kill redis-server on Master node
- View sentry information again
[root@c7-1 redis-5.0.7]# redis-cli -p 26379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.3.13:6379,slaves=2,sentinels=3 PS:status=odown: o That is objectively ,objective #View master sentry log Verification results tail -f /var/log/sentinel.log redis-cli -p 26379 info Sentinel
3. Setting up Cluster Cluster
- The primary node is responsible for the maintenance of read and write requests and cluster information, and only replicates master node data and status information from the secondary node
1. Role
1.1 Data Partition
- Data partitioning (or data fragmentation) is the core function of a cluster
- Clusters distribute data among multiple nodes. On the one hand, they break the limit of Redis single-machine memory size and greatly increase the storage capacity. On the other hand, each primary node can provide read and write services to the outside world, which greatly improves the responsiveness of the cluster.
- Redis single-machine memory size limitation, mentioned in both persistence and master-slave replication
- For example, if single-machine memory is too large, fork operations of bgsave and bgrewriteaof may cause the master process to block, switch between master and slave environments may cause the slave node to be unable to provide service for a long time, and the replication buffer of the master node may overflow during the full replication phase
1.2 High Availability
- Clusters support master-slave replication and automatic failover of primary nodes (similar to sentinels), which can still serve out when any node sends a failure
1.3 Data fragmentation
- Redis Cluster introduced the concept of hash slots, with 16384 hash slots (numbered 0~16383)
- Each node of the cluster is responsible for a portion of the hash slot, and each Key, after the CRC16 check, redeems 16384 to determine which hash slot to place. With this value, the corresponding node of the slot is found, and then jumps directly to the corresponding node for access.
- Take a three-node cluster as an example:
Node A contains hash slots between 0 and 5469
Node B contains hash slots 5461~10922
Node C contains hash slots from 10923 to 16383
2. Environment
- A redis cluster typically requires six nodes, three masters and three slaves. For convenience, all nodes here are simulated on the same server:
- Differentiate by port number: three primary node port numbers: 6001/6002/6003, corresponding slave node port number: 6004/6005/6006
3. Create Port Working Directory
- Create working directory for redis 6 ports
cd /etc/redis/ mkdir -p redis-cluster/redis600{1..6} #One-click Copy of 6001~6006 Scripts vim /opt/redis.sh #!/bin/bash for i in {1..6} do cp /opt/redis-5.0.7/redis.conf /etc/redis/redis-cluster/redis600$i cp /opt/redis-5.0.7/src/redis-cli /opt/redis-5.0.7/src/redis-server /etc/redis/redis-cluster/redis600$i done sh -x /opt/redis.sh
4. Modify the configuration file
cd /etc/redis/redis-cluster/redis6001 vim redis.conf bind 127.0.0.1 #69 lines, comment out the bind item or do not modify, default listens on all network cards protected-mode no #88 lines, change, turn off protection mode port 6001 #92 lines, modification, redis listening port, daemonize yes #Line 136, start daemon, start as independent process cluster-enabled yes #832 lines, uncomment, cluster on cluster-config-file nodes-6001.conf #840 lines, uncomment, cluster name file settings cluster-node-timeout 15000 #846 lines, uncomment cluster timeout setting appendonly yes #700 lines, modify, turn on AOF persistence The other five profiles have the same changes except for port number and cluster profile name cp redis.conf ../redis6002/ cp redis.conf ../redis6003/ cp redis.conf ../redis6004/ cp redis.conf ../redis6005/ cp redis.conf ../redis6006/
- Write startup scripts
#Start redis based on the corresponding profile vim /opt/redis_start.sh #!/bin/bash for d in {1..6} do cd /etc/redis/redis-cluster/redis600$d redis-server redis.conf done ps -ef | grep redis sh -x /opt/redis_start.sh
5. Join the cluster
redis-cli --cluster create 127.0.0.1:6001 127.0.0.1:6002 127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 127.0.0.1:6006 --cluster-replicas 1 #The six instances are divided into three groups, one master and one slave for each group, the former being the master node, and the latter being the slave node. #You need to enter yes to create the following interaction #-replicas 1 indicates that each primary node has a slave node
6. Test Cluster
redis-cli -p 6001 -c #With the -c parameter, nodes can jump to each other 127.0.0.1:6001> cluster slots #View the hash slot number range of nodes 1) 1) (integer) 5461 2) (integer) 10922 #Hash slot number range 3) 1) "127.0.0.1" 2) (integer) 6003 #Primary node IP and port number 3) " fdca661922216dd69a63a7c9d3c4540cd6baef44" 4) 1) "127.0.0.1" 2) (integer) 6004 #From Node IP and Port Number 3) " a2c0c32aff0f38980accd2b63d6d952812e44740" 2) 1) (integer) 0 2) (integer) 5460 3) 1) "127.0.0.1" 2) (integer) 6001 3) "0e5873747a2e26bdc935bc76c2bafb19d0a54b11" 4) 1) "127.0.0.1" 2) (integer) 6006 3) "8842ef5584a85005e135fd0ee59e5a0d67b0cf8e" 3) 1) (integer) 10923 2) (integer) 16383 3) 1) "127.0.0.1" 2) (integer) 6002 3) "81 6ddaa3d1469540b2ffbcaaf9aa867646846b30" 4) 1) "127.0.0.1" 2) (integer) 6005 3) " f847077bfe6722466e96178ae8cbb09dc8b4d5eb"
- Create data in 6001 and view his slots
127.0.0.1:6001> set name zhangsan -> Redirected to slot [5798] located at 127.0.0.1:6002 //Slice to 6002 for storage OK 127.0.0.1:6002> cluster keyslot name //View the slot number of the name key (integer) 5798 //Slot 5798 belongs to 6002 nodes