catalogue
1, rsync remote synchronization
5. Operation (downlink replication)
1, rsync remote synchronization
1. Introduction to rsync
rsync is an open source, fast, multi-functional tool that can realize full and incremental local or remote data synchronous backup. And the attribute information of the original data can not be changed to realize the backup and migration characteristics of the data
rsync software is suitable for unix/linux/windows and other operating system platforms
rsync is a fast and very similar file copying tool. It can copy by instinct, remote, or remote daemon. It provides a large number of parameters to control all aspects of its behavior, and allows a very flexible way to realize file transmission and replication
It is famous for its delta transfer algorithm
rsync listening port: 873
rsync operation mode: C/S
2. Synchronization mode
Full backup:
All the original phases are transmitted
The original document and the new submission are transmitted together, which can be copied in full, and the efficiency is low
Incremental backup
Before data transmission, compare the data you have with the data I have through some algorithms, and transfer different data through the network for incremental replication, with high efficiency
3. rsync command
rsync [option] Original position target position
| Common options | explain |
| -r | Reverse mode, including all files in the directory and subdirectory |
| -l | For symbolic link files, they are still copied as symbolic link files |
| -v | Displays details of the synchronization process |
| -Z | Compress when transferring files |
| -a | Archive mode, recursion and retention of object attributes, equivalent to - rlptgoD |
| -p | Keep the permission mark of the file |
| -t | Time stamp of retention file |
| -g | Keep the belonging group flag of the file (for super users only) |
| -o | Keep the master tag of the file (for super users only) |
| -H | Keep hard linked files |
| -A | Retain ACL national information |
| -D | Keep equipment documents and other special documents |
| --delete | Do not delete files that exist in the target location but not in the original location |
| --checksum | Decide whether to skip the file based on the checksum of the object |
4. Configuration mode
Format I: user name@Host address::Shared module name Format 2: rsync://User name @ host address / shared module name
5. Operation (downlink replication)
Server: 192.168.255.142 #Basic environment configuration systemctl stop firewalld.service setenforce 0 #Configure rsync source server rpm -q rsync yum -y install rsync #Create the / etc/rsyncd.conf configuration file vim /etc/rsyncd.conf #Add the following configuration uid = nobody #root gid = nobody #root use chroot = yes #Locked in the source directory address = 192.168.255.142 #Listening address port 873 #Listen on port tcp/udp 873. You can view it through cat /etc/services | grep rsync log file = /var/log/rsyncd.log #Log file location pid file = /var/run/rsyncd.pid #The file location where the process ID is stored hosts allow = 192.168.255.0/24 #Allowed client addresses [wwwroot] path = /var/www/html #The actual path to the source directory comment = Document Root of www.ljm.com read only = yes #Is it read-only dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z #File types that are no longer compressed during synchronization auth users = backuper #Authorized accounts. Multiple accounts are separated by spaces secrets file = /etc/rsyncd_users.db #Data file for storing account information -------------------------------------------- uid = nobody gid = nobody use chroot = yes address = 192.168.255.142 port 873 log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid hosts allow = 192.168.255.0/24 [wwwroot] path = /var/www/html comment = Document Root of www.ly.com read only = yes dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z auth users = backuper secrets file = /etc/user.db #If anonymity is adopted, just remove the "auth users" and "secrets file" configuration items #Create data files for backup accounts vim /etc/user.db backuper:123456 #There is no need to establish a system user with the same name chmod 600 /etc/user.db #Ensure that all users have read permission to the source directory / var/www/html mkdir -p /var/www/html chmod +r /var/www/html/ ls -ld /var/www/html/ #Start rsync service program rsync --daemon netstat -natp | grep rsync #Turn off rsync service kill $(cat /var/run/rsyncd.pid) rm -rf /var/run/rsyncd.pid ---------------------------------------------- #Server: cd /var/www/html echo "hello world" >> 1.txt
#client: mkdir /abc chmod 777 /abc rsync -avz backuper@192.168.255.142::wwwroot /abc/ #View synchronization: cd /abc
#Client: interactive free format configuration echo "123456" > /etc/server.pass chmod 600 /etc/server.pass rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.255.142::wwwroot /abc/
2, inotify
1. Introduction
You can monitor file system changes and respond to notifications
Tuning inotify kernel parameters (optimization)
/Etc / sysctl.conf (kernel parameter configuration file)
inotifywait: used for continuous monitoring and real-time output of results
inotifywatch: used for short-term monitoring and output results after the task is completed
max_queue_events #Monitor event queue size max_user_instances #Maximum number of monitoring instances max_user_watches #Maximum number of monitoring files per instance
2,inotifywait
Format: inotifywait [parameter]
| Common parameters | explain |
| -m | Continuous monitoring |
| -r | Recursively monitor all child objects |
| -q | Simplified output information |
| -e | Specify which event types to monitor |
3. Operation
#Server: modify rsync configuration file
vim /etc/rsyncd.conf
uid = root
gid = root
use chroot = yes
address = 192.168.255.142
port 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.255.0/24
[wwwroot]
path = /var/www/html
comment = Document Root of www.ly.com
read only = no #close
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z
auth users = backuper
secrets file = /etc/user.db
kill `cat /var/run/rsyncd.pid`
netstat -natp | grep rsync
rsync --daemon
netstat -natp | grep rsync
#Client: inotify kernel parameters
cat /proc/sys/fs/inotify/max_queued_events
cat /proc/sys/fs/inotify/max_user_instances
cat /proc/sys/fs/inotify/max_user_watches
vim /etc/sysctl.conf ((added at the end)
fs.inotify.max_queued_events = 32768 #Monitoring time queue, the default is 16384
fs.inotify.max_user_instances = 1024 #The maximum number of monitoring instances is 128 by default
fs.inotify.max_user_watches = 1048576 #The maximum number of monitoring files per instance is 8192 by default
#When the amount of directory and file data to be monitored is large or changes frequently, it is recommended to increase the parameter value
sysctl -p
#Client installation inotify tools
yum -y install gcc gcc-c++
tar zxvf inotify-tools-3.14.tar.gz -C /opt
cd /opt/inotify-tools-3.14/
./configure
make && make install
#Execute the "inotifywait" command, then add and move files in the client / abc directory, and track the screen output results
inotifywait -mrq -e modify,create,move,delete /abc
#Client write trigger synchronization script
vim /opt/inotify.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc/"
RSYNC_CMD="rsync --apzH --delete --password-file=/etc/server.pass /abc/ backuper@192.168.255.142::wwwroot "
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
do
if [ $(pgrep rsync | wc -l) -le 0 ] ; then
$RSYNC_CMD
fi
done
chmod +x /opt/inotify.sh
chmod +x /etc/rc.d/rc.local
echo "/opt/inotify.sh" >> /etc/rc.d/rc.local
test
