1, Concept
Master slave replication refers to copying data from one Redis server to other Redis servers. The former is called master / leader and the latter is called slave / follower; Data replication is unidirectional and can only be from master node to slave node.
Master mainly writes, Slave mainly reads.
By default, each Redis server is the master node;
A master node can have multiple slave nodes (or no slave nodes), but a slave node can only have one master node. ()
The main functions of master-slave replication include:
1. Data redundancy: master-slave replication realizes the hot backup of data, which is a data redundancy method other than persistence.
2. Fault recovery: when the master node has problems, the slave node can provide services to achieve rapid fault recovery; It is actually a kind of service redundancy.
3. Load balancing: on the basis of master-slave replication, combined with read-write separation, the master node can provide write services, and the slave node can provide read services (that is, the application connects to the master node when writing Redis data, and the application connects to the slave node when reading Redis data), sharing the server load; Especially in the scenario of less writing and more reading, the concurrency of Redis server can be greatly improved by sharing the read load among multiple slave nodes.
4. High availability (cluster) cornerstone: in addition to the above functions, master-slave replication is also the basis for sentinel and cluster implementation. Therefore, master-slave replication is the basis for Redis high availability.
Generally speaking, to apply Redis to engineering projects, it is absolutely impossible to use only one Redis (downtime). The reasons are as follows:
1. Structurally, a single Redis server will have a single point of failure, and one server needs to handle all request loads, resulting in great pressure;
2. In terms of capacity, the memory capacity of a single Redis server is limited. Even if the memory capacity of a Redis server is 256G, all memory can not be used as Redis storage memory. Generally speaking, the maximum memory used by a single Redis should not exceed 20G.
Commodities on e-commerce websites are generally uploaded once and browsed countless times. It is said that being professional means "reading more and writing less".
For this scenario, we can make the following architecture:
Master-slave copy, read-write separation! 80% of the cases are read operations! Reduce the pressure on the server! Often used in architecture! One master
Two from!
Master-slave replication must be used in the company, because Redis cannot be used on a single machine in a real project!
2, Environment configuration
2.1 viewing master-slave replication info replication
127.0.0.1:6379> info replication # View information about the current library # Replication role:master # Role master connected_slaves:0 # No slave master_replid:b63c90e6c501143759cb0e7f450bd1eb0c70882a master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
Copy the three configuration files and modify the corresponding information
1. Port
2. pid name
3. log file name
4. dump.rdb name
After the modification, start our three redis servers, which can be viewed through the process information!
2.2 steps for configuring N redis
1. Copy redis.conf
2. Change port number and other configurations
2.1: port 6379 as the master server
Step 1: modify the port
Step 2: open background operation
Step 3: modify pidfile
Step 4: modify the log
Step 5: modify rbd
2.2: other ports are used as slave servers
Step 1: modify the port
Step 2: open background operation
Step 3: modify pidfile
Step 4: modify the log
Step 5: modify rbd
3.1 startup
redis-server src/redis79.conf redis-server src/redis80.conf redis-server src/redis81.conf
3.2 check whether to start
3, Configure master-slave
By default, each Redis server is the master node;
Generally, we only need to configure the slave!
Recognize the boss! One master (79) and two slaves (80, 81)
127.0.0.1:6380> SLAVEOF 127.0.0.1 6379 # SLAVEOF host 6379 who do you want to be your boss! OK 127.0.0.1:6380> info replication # Replication role:slave # The current role is a slave master_host:127.0.0.1 # You can see the host information master_port:6379 master_link_status:up master_last_io_seconds_ago:3 master_sync_in_progress:0 slave_repl_offset:14 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:a81be8dd257636b2d3e7a9f595e69d73ff03774e master_replid2:0000000000000000000000000000000000000000 master_repl_offset:14 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:14 # View in the host! 127.0.0.1:6379> info replication # Replication role:master connected_slaves:1 # More slave configuration slave0:ip=127.0.0.1,port=6380,state=online,offset=42,lag=1 # More slave configuration master_replid:a81be8dd257636b2d3e7a9f595e69d73ff03774e master_replid2:0000000000000000000000000000000000000000 master_repl_offset:42 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:42
File configuration: password can be set
3.1 details
The host can write, the slave can't write, can only read! All information and data in the host will be automatically saved by the slave!
Host write:
3.2 replication principle
Slave Start successfully connected to master A message will be sent after sync Synchronization command Master After receiving the command, start the background save process and collect all received commands for modifying the dataset. After the background process is executed, master Transfer the entire data file to slave，And complete a full synchronization. Full copy: and slave After receiving the database file data, the service saves it and loads it into memory. Incremental replication: Master Continue to pass all new collected modification commands to slave，Complete synchronization, but as long as it is reconnected master，A full synchronization (full copy) will be performed automatically! Our data must be visible in the slave!
3.3 layer by layer link
4, Sentinel mode (automatic election mode)
4.1 General
The method of master-slave switching technology is: when the master server is down, a slave server needs to be manually switched to the master server, which requires manual intervention, which is laborious and laborious, and the service will not be available for a period of time. This is not a recommended way. More often, we give priority to Sentinel mode. Redis has officially provided Sentinel architecture since 2.8 to solve this problem.
The automatic version of Mou Chao's usurpation can monitor whether the host fails in the background. If it fails, it will automatically convert from the library to the main library according to the number of votes.
Sentinel mode is a special mode. Firstly, Redis provides sentinel commands. Sentinel is an independent process. As a process, it will run independently. The principle is that the sentinel sends a command and waits for the response of the Redis server, so as to monitor multiple running Redis instances.
4.2 the sentry here has two functions
Send a command to let Redis server return to monitor its running status, including master server and slave server.
When the sentinel detects that the master is down, it will automatically switch the slave to the master, and then notify other slave servers through publish subscribe mode to modify the configuration file and let them switch hosts.
However, there may be problems when a sentinel process monitors the Redis server. Therefore, we can use multiple sentinels for monitoring.
Each sentinel will also be monitored, which forms a multi sentinel mode.
Assuming that the primary server is down, sentry 1 detects this result first, and the system will not immediately perform the failover process. Sentry 1 subjectively thinks that the primary server is unavailable, which becomes a subjective offline phenomenon.
When the following sentinels also detect that the primary server is unavailable and the number reaches a certain value, a vote will be held between sentinels. The voting result is initiated by one sentinel to perform the "failover" operation.
After the switch is successful, each sentinel will switch its monitored slave server to the host through the publish and subscribe mode. This process is called objective offline.
4.3 configuration test
Our current state is one master and two slaves!
1. Configure sentinel configuration file sentinel.conf
# sentinel monitor monitored name host port 1 sentinel monitor myredis 127.0.0.1 6379 1
The following number 1 means that the host hangs up. slave votes to see who takes over as the host. The one with the most votes will become the host!
2. Start sentinel redis sentinel kconfig / sentinel.conf
[root@kuangshen bin]# redis-sentinel kconfig/sentinel.conf 26607:X 31 Mar 2020 21:13:10.027 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 26607:X 31 Mar 2020 21:13:10.027 # Redis version=5.0.8, bits=64, commit=00000000, modified=0, pid=26607, just started 26607:X 31 Mar 2020 21:13:10.027 # Configuration loaded _._ _.-``__ ''-._ _.-`` `. `_. ''-. _ Redis 5.0.8 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26379 | `-._ ``-._ `-._ `-./ _.-' _.-' PID: 26607 | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' 26607:X 31 Mar 2020 21:13:10.029 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 26607:X 31 Mar 2020 21:13:10.031 # Sentinel ID is 4c780da7e22d2aebe3bc20c333746f202ce72996 26607:X 31 Mar 2020 21:13:10.031 # +monitor master myredis 127.0.0.1 6379 quorum 1 26607:X 31 Mar 2020 21:13:10.031 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @myredis 127.0.0.1 6379 26607:X 31 Mar 2020 21:13:10.033 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @myredis 127.0.0.1 6379
If the Master node is disconnected, a server will be randomly selected from the slave at this time! (there is a voting algorithm in it!)
Sentry log!
4.4 advantages and disadvantages
Sentinel mode
All configuration
# Example sentinel.conf # The port on which the sentinel sentinel instance runs is 26379 by default port 26379 # Sentry sentinel's working directory dir /tmp # ip port of the redis master node monitored by sentinel # The master name can be named by itself. The name of the master node can only be composed of letters A-z, numbers 0-9 and the three characters ". - _. # quorum how many sentinel sentinels are configured to uniformly think that the master master node is lost, so it is objectively considered that the master node is lost # sentinel monitor <master-name> <ip> <redis-port> <quorum> sentinel monitor mymaster 127.0.0.1 6379 2 # When the requirepass foobared authorization password is enabled in the Redis instance, all clients connecting to the Redis instance must provide the password # Set the password of sentinel sentinel connecting master and slave. Note that the same authentication password must be set for master and slave # sentinel auth-pass <master-name> <password> sentinel auth-pass mymaster MySUPER--secret-0123passw0rd # After the specified number of milliseconds, the primary node does not respond to the sentinel sentinel. At this time, the sentinel subjectively thinks that the primary node goes offline for 30 seconds by default # sentinel down-after-milliseconds <master-name> <milliseconds> sentinel down-after-milliseconds mymaster 30000 # This configuration item specifies the maximum number of slaves that can synchronize the new master at the same time when a failover active / standby switch occurs. The smaller the number, the longer it takes to complete the failover. However, if the number is larger, it means that more slaves are unavailable due to replication. You can set this value to 1 to ensure that only one slave is in a state that cannot process command requests at a time. # sentinel parallel-syncs <master-name> <numslaves> sentinel parallel-syncs mymaster 1 # Failover timeout can be used in the following aspects: #1. The interval between two failover of the same sentinel to the same master. #2. When a slave synchronizes data from an incorrect master, the time is calculated. Until the slave is corrected to synchronize data to the correct master. #3. The time required to cancel an ongoing failover. #4. During failover, configure the maximum time required for all slaves to point to the new master. However, even after this timeout, the slave will still be correctly configured to point to the master, but it will not follow the rules configured by parallel syncs # The default is three minutes # sentinel failover-timeout <master-name> <milliseconds> sentinel failover-timeout mymaster 180000 # SCRIPTS EXECUTION #Configure the script to be executed when an event occurs. You can notify the administrator through the script. For example, send an email to notify relevant personnel when the system is not running normally. #There are the following rules for the running results of scripts: #If the script returns 1 after execution, the script will be executed again later. The number of repetitions is currently 10 by default #If the script returns 2 after execution, or a return value higher than 2, the script will not be executed repeatedly. #If the script is terminated due to receiving a system interrupt signal during execution, the behavior is the same as when the return value is 1. #The maximum execution time of a script is 60s. If this time is exceeded, the script will be terminated by a SIGKILL signal and then re executed. #Notification script: when any warning level event occurs in sentinel (such as subjective failure and objective failure of redis instance, etc.), The script will be called. At this time, the script should be sent by email, SMS And other ways to inform the system administrator about the abnormal operation of the system. When calling the script, two parameters will be passed to the script, one is the type of event and the other is the description of event. If sentinel.conf If the script path is configured in the configuration file, you must ensure that the script exists in the path and is executable. Otherwise sentinel Unable to start normally, successfully. #Notification script # shell programming # sentinel notification-script <master-name> <script-path> sentinel notification-script mymaster /var/redis/notify.sh # Client reconfiguration master node parameter script # When a master changes due to failover, this script will be called to notify the relevant clients of the change of the master address. # The following parameters will be passed to the script when calling the script: # <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port> # At present, < state > is always "failover", # < role > is one of "leader" or "observer". # The parameters from IP, from port, to IP and to port are used to communicate with the old master and the new master (i.e. the old slave) # This script should be generic and can be called multiple times, not targeted. # sentinel client-reconfig-script <master-name> <script-path> sentinel client-reconfig-script mymaster /var/redis/reconfig.sh # It is generally configured by Yun Wei!