Docker Redis 5.0 cluster building

1, Prepare tools

Install docker (from official website)

(1) Install required packages

$ sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2

(2) Use the following command to set up a stable repository

$ sudo yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo

(3) Install docker ce(docker Community Edition)

$ sudo yum install docker-ce

(4) Start docker

$ sudo systemctl start docker

Obtain redis image in docker Library (as of March 27, 2019, the latest version is 5.0.4)

$ docker pull redis

At this point, all tools of redis cluster on docker are ready. Enter docker images on the command line to view the installed images

2, Cluster building

Create Redis container

(1) Create a redis configuration file (redis-cluster.tmpl)

I create a folder redis cluster under the path / home, a file redis-cluster.tmpl under the path / home / redis cluster, and copy the following contents. (Note: the path can be customized. I use / home / redis cluster)

# Node port
port ${PORT}
# Turn on cluster mode
protected-mode no
# cluster mode
cluster-enabled yes
# Cluster configuration name
cluster-config-file nodes.conf
# Timeout
cluster-node-timeout 5000
# In fact, ip assigned to each node's network card is replaced by Internet off ip
cluster-announce-ip 192.168.1.225
# Node mapping port
cluster-announce-port ${PORT}
# Node bus port
cluster-announce-bus-port 1${PORT}
# Persistent mode
appendonly yes

Command to view

[root@node-01 redis-cluster]# cat /home/redis-cluster/redis-cluster.tmpl
port ${PORT}
# Turn on cluster mode
protected-mode no
# cluster mode
cluster-enabled yes
# Cluster configuration name
cluster-config-file nodes.conf
# Timeout
cluster-node-timeout 5000
# In fact, ip assigned to each node's network card is replaced by Internet off ip
cluster-announce-ip 192.168.1.225
# Node mapping port
cluster-announce-port ${PORT}
# Node bus port
cluster-announce-bus-port 1${PORT}
# Persistent mode
appendonly yes

>Note: This template file is a general file for cluster nodes. If ${PORT} will read the command line variable ip, it will be replaced according to the ip assigned by the network card to ensure that the node configuration file is consistent except for the PORT and ip.

(2) Create custom network

$ docker network create redis-net

docker network ls view network card information

[root@node-01 conf]# docker network ls 
NETWORK ID NAME DRIVER SCOPE 7f804fa5fbd2 redis-net bridge local

>Note: the purpose of creating a redis net virtual network card is to enable the docker container to bridge the network with the host (centos7) and connect with the outside world indirectly

(3) View the redis net virtual network card gateway ip

[root@node-01 conf]# docker network inspect redis-net | grep "Gateway" | grep --color=auto -P '(\d{1,3}.){3}\d{1,3}' -o 172.21.0.1

>Note: docker network inspect network name displays the corresponding configuration information of network name (gerp filters the configuration line of gateway and filters ip) manually

(4) Generate conf and data targets under / home / redis cluster, and generate configuration information

 $ for port in `seq 7000 7005`; do \
  mkdir -p ./${port}/conf \
  && PORT=${port} envsubst < ./redis-cluster.tmpl > ./${port}/conf/redis.conf \
  && mkdir -p ./${port}/data; \
  done

>Six folders are generated, from 7000 to 7005. Each folder contains data and conf folders, and the redis.conf configuration file is in conf

(5) Create 6 redis containers

$ for port in `seq 7000 7005`; do \
docker run -d -ti -p ${port}:${port} -p 1${port}:1${port} \
-v /home/redis-cluster/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
-v /home/redis-cluster/${port}/data:/data \
--restart always --name redis-${port} --net redis-net \
--sysctl net.core.somaxconn=1024 redis redis-server /usr/local/etc/redis/redis.conf; \
done
Note: the command translates into a loop 7000-7005 to run the redis container
 Docker run
 -d daemons mode
 --restart always keep container started
 --name redis-700 * container name
 --Net redis net container uses virtual network card
 -p specifies the host machine and container port mapping 700 *: 700*
-P specifies that the host and container redis bus port map 1700 *: 1700*
--privileged=true -v /home/redis-cluster/700*/conf/redis.conf:/usr/local/etc/redis/redis.conf
 Pay authority to mount the host 700 * node file to the container / usr/local/etc/redis/redis.conf file
--privileged=true -v /home/redis-cluster/${port}/data:/data \
Pay permission to mount the host 700*/data directory to the container / data directory
--sysctl net.core.somaxconn=1024 redis redis-server /usr/local/etc/redis/redis.conf;
The container starts the redis server according to the attached configuration file

(6) With the command docker ps, you can view the six container information just generated

[root@node-01 redis-cluster]# docker ps
CONTAINER ID        IMAGE                                         COMMAND                  CREATED             STATUS                PORTS                                                              NAMES
15c479074b87        redis                                         "docker-entrypoint..."   10 seconds ago      Up 9 seconds          0.0.0.0:7005->7005/tcp, 6379/tcp, 0.0.0.0:17005->17005/tcp         redis-7005
45bec33c2c35        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7004->7004/tcp, 6379/tcp, 0.0.0.0:17004->17004/tcp         redis-7004
45482d9e5bb8        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7003->7003/tcp, 6379/tcp, 0.0.0.0:17003->17003/tcp         redis-7003
f633d5c767c9        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7002->7002/tcp, 6379/tcp, 0.0.0.0:17002->17002/tcp         redis-7002
eefc0d49fedf        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7001->7001/tcp, 6379/tcp, 0.0.0.0:17001->17001/tcp         redis-7001
58b311e5dbcb        redis                                         "docker-entrypoint..."   12 seconds ago      Up 11 seconds         0.0.0.0:7000->7000/tcp, 6379/tcp, 0.0.0.0:17000->17000/tcp         redis-7000

(7) View container assignment ip

[root@node-01 conf]# docker network inspect redis-net
[
{
    "Name": "redis-net",
    "Id": "7f804fa5fbd2473f1e6e9783ac52aa980c355e3253b15a28e10f8793c46fc39b",
    "Created": "2019-03-27T12:59:14.45569686+08:00",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
        "Driver": "default",
        "Options": {},
        "Config": [
            {
                "Subnet": "172.21.0.0/16",
                "Gateway": "172.21.0.1"
            }
        ]
    },
    "Internal": false,
    "Attachable": false,
    "Containers": {
        "449c38916b4ef919d1b1828da3f69e0f5256eb2023204f25f69fbb8c449f2163": {
            "Name": "redis-7005",
            "EndpointID": "5c89e6fdd0e983d6d2b950c9bc41616920265dda93f2852650ca5e9da1da22b1",
            "MacAddress": "02:42:ac:15:00:07",
            "IPv4Address": "172.21.0.7/16",
            "IPv6Address": ""
        },
        "600aa2f84dfbe72a19caf5dc684cb7691e89a43753f70d74355468b7659caa23": {
            "Name": "redis-7000",
            "EndpointID": "75873081222c7a2eedc5f3ee639df5cc7645790012dd72e34c97fd97dcb5cf47",
            "MacAddress": "02:42:ac:15:00:02",
            "IPv4Address": "172.21.0.2/16",
            "IPv6Address": ""
        },
        "78aeae8b2d381d3789848f45f766dd01d7c0d5443505920decf41fce751f765d": {
            "Name": "redis-7002",
            "EndpointID": "a70d816cd9e3650b1d10507a85fa85c0c2e143ba99c0974fb000ec87ad5b173d",
            "MacAddress": "02:42:ac:15:00:04",
            "IPv4Address": "172.21.0.4/16",
            "IPv6Address": ""
        },
        "7ecd2e1372a2bc15cb89fad856ab410757b49936f662ce170990c04e1861037d": {
            "Name": "redis-7004",
            "EndpointID": "f8cdb522601e9667ec06dc0dc0d39b46331b3d24cbea1155b00b73e600eca8c7",
            "MacAddress": "02:42:ac:15:00:06",
            "IPv4Address": "172.21.0.6/16",
            "IPv6Address": ""
        },
        "bd8254045c66bc6569c9a585b58cb028e864b6c7f16d836009debd3a3350913c": {
            "Name": "redis-7003",
            "EndpointID": "6a25ff0046e6d6575579cdf08bccbb2a62b29460e3932de3f8e476bb908fdc9c",
            "MacAddress": "02:42:ac:15:00:05",
            "IPv4Address": "172.21.0.5/16",
            "IPv6Address": ""
        },
        "cfef6ca3aeea9d3b4fa527538fbe2a2f79aac25d52921b4edc31839b6f330887": {
            "Name": "redis-7001",
            "EndpointID": "98a36b61bc6f50f78444a6cefa5e9999f5366e72425602511bc95f602ef0dc93",
            "MacAddress": "02:42:ac:15:00:03",
            "IPv4Address": "172.21.0.3/16",
            "IPv6Address": ""
        }
    },
    "Options": {},
    "Labels": {}
}
]

>Note: if ip is assigned to each container, you need to modify the node configuration file redis.conf and restart the container.

(8) Modify host mount directory profile

vim /home/redis-cluster/700*/conf/redis.conf ##Edit the ip assigned by ip for the query and exit esc +: wq

port 7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.21.0.2
cluster-announce-port 7000
cluster-announce-bus-port 17000
appendonly yes

Start cluster

Enter a node

docker exec -it redis-7000 bash

To create a cluster using Redis 5, simply type:

redis-cli --cluster create 172.21.0.2:7000 172.21.0.3:7001 172.21.0.4:7002 172.21.0.5:7003 172.21.0.6:7004 172.21.0.7:7005 --cluster-replicas 1

At this point, the cluster has been created.

[root@node-01 data]# docker exec -it redis-7000 bash
root@600aa2f84dfb:/data# redis-cli --cluster create 172.21.0.2:7000  172.21.0.3:7001  172.21.0.4:7002  172.21.0.5:7003  172.21.0.6:7004  172.21.0.7:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.21.0.6:7004 to 172.21.0.2:7000
Adding replica 172.21.0.7:7005 to 172.21.0.3:7001
Adding replica 172.21.0.5:7003 to 172.21.0.4:7002
M: 0d403d866c4bdfa90f3a1e21b2a33356474014ee 172.21.0.2:7000
   slots:[0-5460] (5461 slots) master
M: ab4bf4833926dd7ffca5f2d34ef754466730e7d7 172.21.0.3:7001
   slots:[5461-10922] (5462 slots) master
M: 846f087ee5885f17d5a5c05d2eb2ba470441f700 172.21.0.4:7002
   slots:[10923-16383] (5461 slots) master
S: e44c2b96f1446b1860fe582547634b12a461407d 172.21.0.5:7003
   replicates 846f087ee5885f17d5a5c05d2eb2ba470441f700
S: 4efa0a9b7b4b159807900efa5f86410e55c1d2b3 172.21.0.6:7004
   replicates 0d403d866c4bdfa90f3a1e21b2a33356474014ee
S: 8a616ee30a7104bcd341d57af825c6b5b30176ff 172.21.0.7:7005
   replicates ab4bf4833926dd7ffca5f2d34ef754466730e7d7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 172.21.0.2:7000)
M: 0d403d866c4bdfa90f3a1e21b2a33356474014ee 172.21.0.2:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: e44c2b96f1446b1860fe582547634b12a461407d 172.21.0.5:7003
   slots: (0 slots) slave
   replicates 846f087ee5885f17d5a5c05d2eb2ba470441f700
M: ab4bf4833926dd7ffca5f2d34ef754466730e7d7 172.21.0.3:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 8a616ee30a7104bcd341d57af825c6b5b30176ff 172.21.0.7:7005
   slots: (0 slots) slave
   replicates ab4bf4833926dd7ffca5f2d34ef754466730e7d7
M: 846f087ee5885f17d5a5c05d2eb2ba470441f700 172.21.0.4:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 4efa0a9b7b4b159807900efa5f86410e55c1d2b3 172.21.0.6:7004
   slots: (0 slots) slave
   replicates 0d403d866c4bdfa90f3a1e21b2a33356474014ee
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

Connect the client and view the master-slave information

root@600aa2f84dfb:/data# redis-cli -c -p 7000     
127.0.0.1:7000> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.21.0.6,port=7004,state=online,offset=3556,lag=1
master_replid:ef97265d34769b68096786c3e802fb3a13968fe9
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:3556
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:3556

Other precautions

Host host open ip

for port in `seq 7000 7005`; do \  
firewall-cmd --zone=public --add-port=${port}/tcp --permanent  
done
#Reload
firewall-cmd --reload
You need to pause the container and delete it to repeat step 6
for port in `seq 7000 7005`; do \
  docker stop redis-${port};
  docker rm redis-${port};
done

Tags: Database Redis Docker network sudo

Posted on Mon, 11 May 2020 00:14:10 -0400 by ThYGrEaTCoDeR201