LVS Load Balancing Cluster
Understanding the principles of load balancing clustering
Master LVS-NAT deployment
Overview of Enterprise Cluster Application
The meaning of clustering:
1.Cluster, Cluster, Cluster
2. Composed of multiple hosts, but external only as a whole
In Internet applications, with the site's increasing requirements for hardware performance, response speed, service stability, data reliability, etc., a single server is not satisfied
Solution:
1. Use expensive small and mainframe computers
2. Build a service cluster using a regular server
Enterprise Cluster Classification
There are three types of clusters based on the target differences they target:
1. Load Balanced Clusters (Polling, Minimum Connection Weight)
2. Highly available clusters (access speed, reliability)
3. High Performance Operational Clustering (Concurrent Processing Tasks)
Load Balance Cluster:
1. Achieve high concurrency, load (LB) and overall performance with the goal of improving the responsiveness of the application system, processing as many access requests as possible, and reducing latencies
2.LB's load allocation depends on the shunt algorithm of the primary node
High Availability Cluster:
1. To improve the reliability of the application system and minimize the interruption time, ensure the continuity of service and achieve high availability (HA) fault tolerance.
2.HA works in duplex and master-slave modes
High Performance Computer Cluster:
1. Achieve high performance computing (HPC) capabilities comparable to large, supercomputers, with the goal of improving CPU speed, expanding hardware resources and analytical capabilities of application systems.
2. The high performance of high performance computing clusters relies on "distributed computing", "parallel computing". By integrating CPU, memory and other resources from multiple servers with dedicated hardware and software, only large and supercomputers can achieve computing power.
Load Balancing Cluster Work Mode Analysis
Load Balancing Cluster is the cluster type most used by enterprises at present
Cluster load scheduling technology works in three modes:
1. Address Translation
2.IP Tunnel
3. Direct Routing (DR)
NAT mode
Network Address Translation:
1. NAT mode for short, similar to the private network structure of firewalls, the load dispatcher acts as gateway to all server nodes, that is, as access entry to clients and as access exit for nodes to respond to clients.
2. Server nodes use private IP addresses, which are on the same physical network as the load dispatcher, and are more secure than the other two methods
TUN mode
IP Tunnel:
1. TUN mode, which uses an open network structure, has a load dispatcher as the access entry point for clients only. Nodes respond directly to clients through their respective Internet connections, instead of passing through the load dispatcher.
2. Server nodes are scattered across different locations in the Internet, have separate public network IP addresses, and communicate with each other through dedicated IP tunnels and load dispatchers
DR mode
Direct Routing:
1. DR mode, a semi-open network structure, is similar to TUN mode in structure, but the nodes are not scattered around, but are in the same physical network as the scheduler
2. Load dispatcher connects with each node server via local network, no dedicated IP tunneling is required
Load Balancing Cluster Architecture
Structure of load balancing:
1. Layer 1, Load Scheduler (Load Balancer or Director)
2.Layer 2, Server Pool
3. Layer 3, Share Storage
About LVS Virtual Server
Linux Virtual Server:
1. Load balancing solutions for the Linux kernel
2.In May 1998, created by Dr. Zhang Wensong of China
3. Official website: http://www.linuxvirtualserver.org/
Load Scheduling Algorithm for LVS
1. Round Robin:
1. Assign incoming access requests to nodes in the cluster in turn (real servers)
(2) Treat each server equally, regardless of the actual number of connections and system load on the server
2. Weighted Round Robin:
1. The dispatcher can automatically query each node's load and dynamically adjust its weight based on the access requests received by the real server in turn according to its processing power.
(2) Ensure that servers with strong processing power bear more access traffic
3. Least Connections
(1) Allocate access requests received to nodes with the least number of connections based on the number of connections established by the real server
4. Weighted Least Connections
1. Automatically adjust weights for real servers when the performance of server nodes differs greatly
(2) Nodes with higher weights will bear a larger proportion of active connection load
Using the ipvsadm tool
VS Cluster Creation and Management
NFS Shared Storage Service
Network File System, Network File System
1. Dependent on RPC (Remote Procedure Call)
2. nfs-utils, rpcbind packages need to be installed
3. System Services: nfs, rpcbind
4. Sharing profile: /etc/exports
Accessing NFS Shared Resources on Clients
1. Install the rpcbind package and start the rpcbind service
2. Manually mount NFS shared directories
3.fstab automount settings
Demo:LVS Load Balancing Cluster
Environmental preparation:
CentOS 7-1: Scheduler, gateway (requires two network cards): 12.0.0.1 In: 192.168.200.1
CentOS 7-2: Apache 192.168.200.110
CentOS 7-3: Apache 192.168.200.120
CentOS 7-4: Provide shared storage 192.168.200.130
win7-1: Client 12.0.0.12
yum Online Installation Pre-operation:
1. Shared storage server CentOS 7-4 operations:
[root@localhost ~]# rpm -q nfs-utils nfs-utils-1.3.0-0.48.el7.x86_64 [root@localhost ~]# rpm -q rpcbind rpcbind-0.2.0-42.el7.x86_64
2. Node Server CentOS 7-2 and 7-3 Node Servers:
[root@localhost ~]# yum install httpd -y
3. Scheduler Gateway CentOS 7-1 Operation:
//First add a network adapter to become two network cards [root@localhost ~]# yum install ipvsadm -y
Shared Storage Server CentOS 7-4:
//Change network card to host-only mode first [root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 BOOTPROTO="static" //Change dhcp to static DEVICE="ens33" ONBOOT="yes" IPADDR=192.168.200.130 //Insert under last line: IP, subnet, gateway NETMASK=255.255.255.0 GATEWAY=192.168.200.1 //Input after modification: wq save exit [root@localhost ~]# service network restart Restarting network (via systemctl): [ Determine ] [root@localhost ~]# systemctl stop firewalld.service [root@localhost ~]# setenforce 0 [root@localhost ~]# systemctl start nfs.service [root@localhost ~]# systemctl status nfs.service ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled) Active: active (exited) since 2 2019-11-26 17:42:05 CST; 11s ago ......Omit multiple lines, status is Active Explain normal [root@localhost ~]# systemctl start rpcbind.service [root@localhost ~]# systemctl status rpcbind.service ● rpcbind.service - RPC bind service Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; indirect; vendor preset: enabled) Active: active (running) since 2 2019-11-26 17:40:23 CST; 4min 26s ago ......Omit multiple lines, status is Active Explain normal [root@localhost ~]# vim /etc/exports /usr/share *(ro,sync) /opt/accp 192.168.200.0/24(rw,sync) /opt/benet 192.168.200.0/24(rw,sync) //After adding the above, enter: wq Save Exit [root@localhost ~]# cd /opt/ [root@localhost opt]# mkdir benet accp [root@localhost opt]# ls -l //Total dosage 0 drwxr-xr-x. 2 root root 6 11 February 2617:50 accp drwxr-xr-x. 2 root root 6 11 February 2617:50 benet drwxr-xr-x. 2 root root 6 3 February 26, 2015 rh [root@localhost opt]# chmod 777 accp/ benet/ //elevate permissions [root@localhost opt]# ls -l //Total dosage 0 drwxrwxrwx. 2 root root 6 11 February 2617:50 accp drwxrwxrwx. 2 root root 6 11 February 2617:50 benet drwxr-xr-x. 2 root root 6 3 February 26, 2015 rh [root@localhost opt]# Exportfs-rv //Publish exporting 192.168.200.0/24:/opt/benet exporting 192.168.200.0/24:/opt/accp exporting *:/usr/share
Operations on the node server (CentOS 7-2):
[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 BOOTPROTO="static" //Change dhcp to static DEVICE="ens33" ONBOOT="yes" IPADDR=192.168.200.110 //Insert under last line: IP, subnet, gateway NETMASK=255.255.255.0 GATEWAY=192.168.200.1 //Input after modification: wq save exit [root@localhost ~]# service network restart Restarting network (via systemctl): [ Determine ] [root@localhost ~]# systemctl stop firewalld.service [root@localhost ~]# setenforce 0 [root@localhost ~]# systemctl start httpd.service [root@localhost ~]# netstat -ntap | grep 80 tcp6 0 0 :::80 :::* LISTEN 7315/httpd //Set Network Adapter to Host Only [root@localhost ~]# ping 192.168.200.130 PING 192.168.200.130 (192.168.200.130) 56(84) bytes of data. 64 bytes from 192.168.200.130: icmp_seq=1 ttl=64 time=0.754 ms 64 bytes from 192.168.200.130: icmp_seq=2 ttl=64 time=0.368 ms 64 bytes from 192.168.200.130: icmp_seq=3 ttl=64 time=0.398 ms [root@localhost ~]# showmount -e 192.168.200.130 Export list for 192.168.200.130: /usr/share * /opt/benet 192.168.200.0/24 /opt/accp 192.168.200.0/24 [root@localhost ~]# mount.nfs 192.168.200.130:/opt/accp /var/www/html/ [root@localhost ~]# df -h //File System Capacity Used Available%Mountpoint /dev/sda2 20G 4.3G 16G 22% / devtmpfs 898M 0 898M 0% /dev tmpfs 912M 0 912M 0% /dev/shm tmpfs 912M 9.0M 903M 1% /run tmpfs 912M 0 912M 0% /sys/fs/cgroup /dev/sda1 6.0G 174M 5.9G 3% /boot /dev/sda5 10G 54M 10G 1% /home tmpfs 183M 4.0K 183M 1% /run/user/42 tmpfs 183M 20K 183M 1% /run/user/0 /dev/sr0 4.3G 4.3G 0 100% /run/media/root/CentOS 7 x86_64 192.168.200.130:/opt/accp 20G 3.4G 17G 17% /var/www/html [root@localhost ~]# cd /var/www/html/ [root@localhost html]# echo "this is accp web" > index.html [root@localhost html]# ls index.html
Now go back to the storage server CentOS 7-4 to see if this file is available:
[root@localhost ~]# cd /opt/ [root@localhost opt]# ls accp benet rh [root@localhost opt]# cd accp/ [root@localhost accp]# ls index.html [root@localhost accp]# cat index.html this is accp web //Here's our new index.html file
Verify: Use Firefox browser input on CentOS 7-2 node servers: 127.0.0.1 to see if we can display the web page content we write:
Operations on the node server (CentOS 7-3):
[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 BOOTPROTO="static" //Change dhcp to static DEVICE="ens33" ONBOOT="yes" IPADDR=192.168.200.120 //Insert under last line: IP, subnet, gateway NETMASK=255.255.255.0 GATEWAY=192.168.200.1 //Input after modification: wq save exit [root@localhost ~]# service network restart Restarting network (via systemctl): [ Determine ] [root@localhost ~]# systemctl stop firewalld.service [root@localhost ~]# setenforce 0 [root@localhost ~]# systemctl start httpd.service [root@localhost ~]# netstat -ntap | grep 80 tcp6 0 0 :::80 :::* LISTEN 7315/httpd //Set Network Adapter to Host Only [root@localhost ~]# ping 192.168.200.130 PING 192.168.200.130 (192.168.200.130) 56(84) bytes of data. 64 bytes from 192.168.200.130: icmp_seq=1 ttl=64 time=0.532 ms 64 bytes from 192.168.200.130: icmp_seq=2 ttl=64 time=1.01 ms 64 bytes from 192.168.200.130: icmp_seq=3 ttl=64 time=0.940 ms [root@localhost ~]# showmount -e 192.168.200.130 Export list for 192.168.200.130: /usr/share * /opt/benet 192.168.200.0/24 /opt/accp 192.168.200.0/24 [root@localhost ~]# mount.nfs 192.168.200.130:/opt/benet /var/www/html/ [root@localhost ~]# df -h //File System Capacity Used Available%Mountpoint /dev/sda2 20G 3.4G 17G 17% / devtmpfs 898M 0 898M 0% /dev tmpfs 912M 0 912M 0% /dev/shm tmpfs 912M 9.0M 903M 1% /run tmpfs 912M 0 912M 0% /sys/fs/cgroup /dev/sda1 6.0G 174M 5.9G 3% /boot /dev/sda5 10G 54M 10G 1% /home tmpfs 183M 4.0K 183M 1% /run/user/42 tmpfs 183M 20K 183M 1% /run/user/0 /dev/sr0 4.3G 4.3G 0 100% /run/media/root/CentOS 7 x86_64 192.168.200.130:/opt/benet 20G 3.4G 17G 17% /var/www/html [root@localhost ~]# cd /var/www/html/ [root@localhost html]# echo "this is benet web" > index.html [root@localhost html]# ls index.html
Now go back to the storage server CentOS 7-4 to see if this file is available:
[root@localhost ~]# cd /opt/ [root@localhost opt]# ls accp benet rh [root@localhost opt]# cd accp/ [root@localhost accp]# ls index.html [root@localhost accp]# cat index.html this is benet web //Here's our new index.html file
Verify: Use Firefox browser input on CentOS 7-3 node servers: 127.0.0.1 to see if we can display the content of the page we write:
Scheduling, operation of Gateway Server CentOS 7-1:
[root@localhost ~]# cd /etc/sysconfig/network-scripts/ [root@localhost network-scripts]# ls ifcfg-ens33 ifdown-ppp ifup-ib ifup-Team ifcfg-lo ifdown-routes ifup-ippp ifup-TeamPort [root@localhost network-scripts]# cp ifcfg-ens33 ifcfg-ens36 [root@localhost network-scripts]# vim ifcfg-ens36 BOOTPROTO="static" //Change dhcp to static NAME="ens36" //Change name to ens36 UUID Number Delete DEVICE="ens36" //Change name to ens36 ONBOOT="yes" IPADDR=12.0.0.1 //Insert under last line: IP, subnet NETMASK=255.255.255.0 //Input after modification: wq save exit [root@localhost network-scripts]# vim ifcfg-ens33 BOOTPROTO="static" //Change dhcp to static DEVICE="ens33" ONBOOT="yes" //Insert under last line: IP, subnet IPADDR=192.168.200.1 NETMASK=255.255.255.0 //Input after modification: wq save exit [root@localhost network-scripts]# service network restart Restarting network (via systemctl): [ Determine ] [root@localhost network-scripts]# vim /etc/sysctl.conf //Insert in the last line by o to go down net.ipv4.ip_forward=1 //Input after modification: wq save exit [root@localhost network-scripts]# sysctl -p net.ipv4.ip_forward = 1 [root@localhost network-scripts]# iptables -t nat -F [root@localhost network-scripts]# iptables -F [root@localhost network-scripts]# iptables -t nat -A POSTROUTING -o ens36 -s 192.168.200.0/24 -j SNAT --to-source 12.0.0.1
Verify forwarding in win7-1:
First set the network card to host-only mode, and then configure the network card to a static address, while closing the firewall:
The internal node server ping:12.0.0.12 is used to test Ping connectivity:
[root@localhost html]# ping 12.0.0.12 PING 12.0.0.12 (12.0.0.12) 56(84) bytes of data. 64 bytes from 12.0.0.12: icmp_seq=1 ttl=127 time=1.14 ms 64 bytes from 12.0.0.12: icmp_seq=2 ttl=127 time=1.78 ms 64 bytes from 12.0.0.12: icmp_seq=3 ttl=127 time=1.02 ms //You can ping at this point to indicate that there is no problem with the response
Load the LVS kernel module:
[root@localhost network-scripts]# modprobe ip_vs [root@localhost network-scripts]# cat /proc/net/ip_vs IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn
Start the service: (Note: In CentOS 7, you must save the startup service first, otherwise you will get an error!!)
[root@localhost network-scripts]# ipvsadm --save > /etc/sysconfig/ipvsadm [root@localhost network-scripts]# systemctl start ipvsadm.service
Write configuration rules:
[root@localhost network-scripts]# cd /opt/ [root@localhost opt]# vim nat.sh #!/bin/bash ipvsadm -C //Clear all records in the kernel virtual server table ipvsadm -A -t 12.0.0.1:80 -s rr //Add a new virtual server ipvsadm -a -t 12.0.0.1:80 -r 192.168.200.110:80 -m ipvsadm -a -t 12.0.0.1:80 -r 192.168.200.120:80 -m ipvsadm //Enter when input is complete: wq save exit [root@localhost opt]# source nat.sh IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP localhost.localdomain:http rr -> 129.168.200.110:http Masq 1 0 0 -> 129.168.200.120:http Masq 1 0 0
Verify: Use the win7-1 client terminal to access 12.0.0.1 to see if it is successful:
