Linux Enterprise Operations and Maintenance - Redis installation and deployment, master-slave replication, master auto-switching

1. Configure sentinel file

Configure sentinel in server1 host and copy to server2 and server3

cd redis-6.2.4/ cp sentinel.conf /etc/redis/ cd /etc/redis/ vim sentinel.conf /// sentinel monitor mymaster 172.25.7.1 6379 2 ##master is server1, 2 means two votes are required and the host is identified as down sentinel down-after-milliseconds mymaster 10000 ##Connection timeout is 10s /// scp sentinel.conf server2:/etc/redis/ scp sentinel.conf server3:/etc/redis/ redis-sentinel /etc/redis/sentinel.conf ##You can see two slave ends

Change file sentinel.conf

sentinel file configuration completed!!!

2.master auto-switch

Monitor information in server1, master opens sentry and puts monitor in the background
redis-sentinel /etc/redis/sentinel.conf &

server2, server3 are slave s at this time
Also turn on Sentry mode

redis-sentinel /etc/redis/sentinel.conf &

Open a new shell panel and connect to server1

Turn off redis in server1
Viewing changes in another server1 shell panel will show the process of closing and running for master for server2, server3

master became server2

Serr1 is now off, and when server1 redis is turned on, server1 automatically switches to slave and empties all previous data

Clustering principles

1. All redis nodes are interconnected (PING-PONG mechanism), and binary protocols are used internally to optimize transmission speed and bandwidth.
2. Node fail is only valid when more than half of the nodes in the cluster detect failures.
3. Clients connect directly to the redis node without the intermediate proxy layer. Clients do not need to connect to all nodes in the cluster, but to any available node in the cluster.
4.redis-cluster maps all physical nodes to [0-16383] slots, and cluster maintains node<->slot<->value

There are 16384 hash slots built into the Redis cluster. When a key-value needs to be placed in the Redis cluster, redis first calculates a result using the crc16 algorithm for the key, then calculates the remainder for 16384, so that each key corresponds to a hash slot numbered between 0 and 16383, and redis maps the hash slot to different nodes based on roughly equal number of nodes

1. Cluster building

Build clusters manually

Serr1 host is master, restart redis, open AOF mode, set up medis directory, cluster 7001-7006 directory, configure configuration files under directory, start service

In server1

/etc/init.d/redis_6379 restart vim /etc/redis/6379.conf \\\ appendonly yes ##Turn on AOF mode \\\ cd /usr/local mkdir redis #Create directory cd redis/ mkdir 700 #Create a directory of six nodes cd 7001 vim redis.conf #Edit Profile /// port 7001 cluster-enabled yes ##Open Cluster cluster-config-file nodes.conf ##Cluster Profile cluster-node-timeout 5000 ##Node Timeout appendonly yes ##Turn on AOF mode daemonize yes ##Start with a daemon thread /// redis-server redis.conf ##Start Services ps ax ##redis-server *:7001 [cluster] cp redis.conf ../7002/ ##Copy profile to 7002-7006 cp redis.conf ../7003/ cp redis.conf ../7004/ cp redis.conf ../7005/ cp redis.conf ../7006/ Enter each node directory to modify the configuration file separately: port 7001 For example: cd 7002/ vim redis.conf /// port 7002 cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000 appendonly yes daemonize yes /// redis-server redis.conf #Start Services

Turn on AOF mode in /etc/redis/6379.conf file

After creating the redis directory, edit the configuration file redis.conf (files in the 7001 directory) in each node directory

Ultimately, you can use ps to view when the services for all nodes are started

ps ax ##View process

Use redis-cli for cluster interaction, clients connect to any node, use -c to sign in as a cluster, and -p to specify a port. info view, currently master, no slave end.

Create a cluster master-slave node, with the cluster-replicas parameter as a number, 1 indicating that each master node requires a slave node. Then check the cluster to see the cluster information.

redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1 #Create Cluster redis-cli --cluster check 127.0.0.1:7001 ##Check Cluster redis-cli --cluster info 127.0.0.1:7001 ##Viewing Cluster Information

Create six nodes, 1, 2, 3 as master, one master, one slave

Check the cluster and see that there is one master and one slave

Viewing Cluster Information

Script Build Cluster

cd cd redis-6.2.4/ cd utils/create-cluster/ ./create-cluster start ##Open instances Starting 30001~30006 ps ax ./create-cluster stop ##Stop all instances and use the manually built cluster for the next experiment ps ax #See

2. Node master-slave switching

View node information before we go to node master-slave information

As shown in the figure, 7004 is a slave node of 7003. When port 7003 is connected and the current node is turned off, 7004 automatically switches to the primary node, but not from the node. Turn on server2, then check the cluster for the slave node, and you can see that 7004 has a slave node of 7003.

redis-cli -c -p 7003 > SHUTDOWN ps ax ##7003 disappeared redis-cli --cluster check 127.0.0.1:7001 ##7004 0 slaves cd .. cd 7003 redis-server redis.conf ##Start the service again ps ax ##redis-server *:7003 [cluster] redis-cli --cluster check 127.0.0.1:7001 ##7004 1 slaves is 7003

Close 7003

ps ax view found 7003 disappeared

redis-cli --cluster check 127.0.0.1:7001
Checking all node information found 7004 to be master and not from a node

Turn on 7003 again, look at the node information again and find that 7003 has become a slave node of 7004, that is, switch master-slave node automatically

3. Add Nodes and Slices

Create directories and configuration files for 7007 and 7008 nodes

mkdir 700 cd 7007 cp ../7001/redis.conf . vim redis.conf /// port 7007 cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000 appendonly yes daemonize yes /// redis-server redis.conf 7008 identical ps ax

View the processes of two nodes

The 7007 nodes were added to the cluster, but slots were not allocated, so this node did not really start sharing the cluster work, so it had to be partitioned. Re-partitioning basically means moving hash slots from one set of nodes to another, just like cluster creation.

redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001 #Adding 7007 nodes to the cluster redis-cli --cluster check 127.0.0.1:7001 #7007 0 slaves slots: (0 slots) master

The node joined the cluster but did not allocate slot s, so the node did not really start sharing the cluster work

Check cluster information, 7007 node is master

Allocate slot s

redis-cli --cluster reshard 127.0.0.1:7001 #Re-slicing Fragmentation operation How many slots do you want to move (from 1 to 16384)? 1000 #Number of moving slots What is the receiving node ##ID of 7007 node (ID of receiving node) Source node #1: all #From which nodes do you get these keys, input all gets these hash slots from all other primary nodes Do you want to proceed with the proposed reshard plan (yes/no)? yes #Determine if you want to continue repartitioning, enter yes redis-cli --cluster check 127.0.0.1:7001 #Check cluster information

Check cluster information again

At this point, 7007 nodes have 1000 hash slots and no slave nodes, so 7008 nodes are added to the 7001-node cluster as slave nodes of 8a0bcd08256e3f0790097ee26a62450440fe3a41. If not specified--cluster-master-id is randomly assigned to any of the primary nodes.

redis-cli --cluster check 127.0.0.1:7001 redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7001 --cluster-slave --cluster-master-id 8a0bcd08256e3f0790097ee26a62450440fe3a41 #The 7008 node is added to the 7001 node cluster and acts as the slave node ID with node_id 7007. If the primary node is not specified ID --cluster-master-id Randomly assigned to any primary node redis-cli --cluster check 127.0.0.1:7001 #Check Cluster redis-cli --cluster info 127.0.0.1:7001 #View cluster information

Add 7008 to Cluster

Check Cluster
redis-cli --cluster check 127.0.0.1:7001

Show cluster information, 7007 now has a slave node
redis-cli --cluster info 127.0.0.1:7001

Redis as the cache server for mysql

The Psmisc package contains three programs to help manage/proc directories

fuser Displays processes that use the specified file or file system PID. killall Kill a process by name that signals all processes running the specified command. pstree Tree displays currently running processes.

Kill all is used to kill processes by name. We can use the kill command to kill processes with the specified process PID. If we want to find processes that we need to kill, we need to use ps and other commands before we can use grep to find processes. Kill all combines these two processes into one
Format: killall [command-name]
fuser

The fuser command is used to display all process information that is using the specified file, file system, or sockets.

Common Options
-a Displays all the files specified on the command line so that the accessed files are displayed by default.
-c, like-m, is used for POSIX compatibility.
-k Kills the process that accesses the file. If -signal is not specified, a SIGKILL signal is sent.
-i Ask the user before killing the process, if no-k option is ignored.
-l Lists all known signal names.
-m name specifies a file mounted on the file system or a mounted block device (name). All processes accessing this file or file system will be listed. If a directory is specified, it will be automatically converted to "name/" and all file systems mounted under that directory will be used.
-n space specifies a different name space. Different space files (filename, default here), TCP (local TCP port), udp (local udp port) are supported. For ports, you can specify the port number or name, and if this does not cause ambiguity, you can use simple representations such as name/space (i.e., 80/tcp).
-s silent mode, when-u,-v is ignored. -a cannot be used with-s.
-signal uses the specified signal, not the SIGKILL, to kill the process. The signal can be represented by a name or number (e.g. -HUP,-1). This option should be used with-k, otherwise it will be ignored.
-u Add the user name of the process owner after each PID.
-v verbose mode. Output like ps command contains many domains such as PID,USER,COMMAND, etc. If accessed by kernel then PID is kernel. -V Output version number.
-4 uses IPV4 sockets and cannot be used with-6. Naming of-n tcp and udp is not ignored only if they exist.
-6 uses IPV6 sockets and cannot be used with-4. Naming of-n tcp and udp is not ignored.

• Reset all options and set the signal to SIGKILL.

In server1 host (nginx+php)

Install process management tools to facilitate killall commands and shut down all previous redis processes

yum install -y psmisc ##Easy to use killall command killall redis-server ps ax ##No redis and normal nginx and php

Install packages php, gearmand, libevent-devel, libgearman related packages

Local Host Get Package lftp 172.25.254.250 cd /pub/docs/redis mirror rhel7/ exit cd rhel7/ yum install -y *.rpm #Install All systemctl daemon-reload systemctl start php-fpm #Open php

Packages in rhel7 directory

Modify the test page test.php content, modify master-slave ip

mv test.php /usr/local/nginx/html/ vim test.php /// 3 $redis->connect('172.25.7.2',6379) 10 $connect = mysql_connect('172.25.7.3','redis','westos'); ///

Test Page test.php Source

<?php $redis = new Redis(); $redis->connect('127.0.0.1',6379) or die ("could net connect redis server"); # $query = "select * from test limit 9"; $query = "select * from test"; for ($key = 1; $key < 10; $key++) { if (!$redis->get($key)) { $connect = mysql_connect('127.0.0.1','redis','westos'); mysql_select_db(test); $result = mysql_query($query); //Cache the results of the query sql to redis if $key is not found while ($row = mysql_fetch_assoc($result)) { $redis->set($row['id'],$row['name']); } $myserver = 'mysql'; break; } else { $myserver = "redis"; $data[$key] = $redis->get($key); } } echo $myserver; echo "<br>"; for ($key = 1; $key < 10; $key++) { echo "number is <b><font color=#FF0000>$key</font></b>"; echo "<br>"; echo "name is <b><font color=#FF0000>$data[$key]</font></b>"; echo "<br>"; } ?>

In server2 host (redis)
Turn on redis, info view information, currently server1's slave. Modify the configuration file/etc/redis/redis.conf, delete the settings about the master and restart redis. info view server2 is now the master host

In server3 host (mariadb database)

On the server3 host, stop rdis and cancel boot-up. Remove redis environment variable and take effect

/etc/init.d/redis_6379 stop chkconfig redis_6379 off ##Cancel boot-up self-start vim .bash_profile /// PATH=$PATH:$HOME/bin /// source .bash_profile

Install mariadb to restore mysql's configuration file, delete all mysql's data files, and open mariadb

yum install -y mariadb-server.x86_64 vim /etc/my.cnf /// [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock [mysqld_safe] log-error=/var/lib/mysql/mysqld.log pid-file=/var/lib/mysql/mysqld.pid /// systemctl start mariadb

Enter the database, you will see the test library, import the local file test.sql file, is the contents of the test library. Enter the database, you will see the imported data, and authorize redis users

lftp 172.25.254.250 > cd pub/docs/redis/ > get test.sql mysql > show databases; ##test mysql < test.sql #Import test library contents mysql > use test > show tables; > select * from test; ##test1~test9 > grant all on test.* to redis@'%' identified by 'westos'; #To grant authorization

Before importing data

View database after importing test.sql data

Restart the database after importing data
systemctl restart mariadb.service

test.sql source

use test; CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8; INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9'); #DELIMITER $$ #CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN # SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); # END$$ #DELIMITER ;

Access 172.25.7.1/test.php in browser
This data can only be copied synchronously, not asynchronously. It can only be overwritten and read on redis. The mysql side changes will not take effect

We can change the data in server2 redis
Figure:

Verify data is synchronized in browser

Read and write data on mysql side

In servre3

server3(mariadb) lftp 172.25.254.250 cd /pub/docs/redis/ get lib_mysqludf_json-master.zip exit yum install -y unzip unzip lib_mysqludf_json-master.zip cd lib_mysqludf_json-master/ yum install -y gcc gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c cp lib_mysqludf_json.so /usr/lib64/mysql/plugin/ Get into mysql See mysql mysql> show global variables like 'plugin_dir';

mysql> CREATE FUNCTION json_object RETURNS STRING SONAME 'lib_mysqludf_json.so'; mysql> select * from mysql.func;

tar zxf gearman-mysql-udf-0.6.tar.gz cd gearman-mysql-udf-0.6/ ./configure --libdir=/usr/lib64/mysql/plugin/ yum install -y libgearman* libevent* libevent-devel-2.0.21-4.el7.x86_64.rpm libgearman-1.1.12-18.el7.x86_64.rpm libgearman-devel-1.1.12-18.el7.x86_64.rpm ./configure --libdir=/usr/lib64/mysql/plugin/ #Re-detect make #Compile make install #install cd /usr/local/lib (Directory is empty) mysql mysql> CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so'; mysql> CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so'; mysql> select * from mysql.func;

Install Package

CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so';
CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so';
select * from mysql.func;

vim test.sql

Add new comment

0 comments