catalogue
1, rsync remote synchronization
5. Operation (downlink replication)
1, rsync remote synchronization
1. Introduction to rsync
rsync is an open source, fast, multi-functional tool that can realize full and incremental local or remote data synchronous backup. And the attribute information of the original data can not be changed to realize the backup and migration characteristics of the data
rsync software is suitable for unix/linux/windows and other operating system platforms
rsync is a fast and very similar file copying tool. It can copy by instinct, remote, or remote daemon. It provides a large number of parameters to control all aspects of its behavior, and allows a very flexible way to realize file transmission and replication
It is famous for its delta transfer algorithm
rsync listening port: 873
rsync operation mode: C/S
2. Synchronization mode
Full backup:
All the original phases are transmitted
The original document and the new submission are transmitted together, which can be copied in full, and the efficiency is low
Incremental backup
Before data transmission, compare the data you have with the data I have through some algorithms, and transfer different data through the network for incremental replication, with high efficiency
3. rsync command
rsync [option] Original position target position
Common options | explain |
-r | Reverse mode, including all files in the directory and subdirectory |
-l | For symbolic link files, they are still copied as symbolic link files |
-v | Displays details of the synchronization process |
-Z | Compress when transferring files |
-a | Archive mode, recursion and retention of object attributes, equivalent to - rlptgoD |
-p | Keep the permission mark of the file |
-t | Time stamp of retention file |
-g | Keep the belonging group flag of the file (for super users only) |
-o | Keep the master tag of the file (for super users only) |
-H | Keep hard linked files |
-A | Retain ACL national information |
-D | Keep equipment documents and other special documents |
--delete | Do not delete files that exist in the target location but not in the original location |
--checksum | Decide whether to skip the file based on the checksum of the object |
4. Configuration mode
Format I: user name@Host address::Shared module name Format 2: rsync://User name @ host address / shared module name
5. Operation (downlink replication)
Server: 192.168.255.142 #Basic environment configuration systemctl stop firewalld.service setenforce 0 #Configure rsync source server rpm -q rsync yum -y install rsync #Create the / etc/rsyncd.conf configuration file vim /etc/rsyncd.conf #Add the following configuration uid = nobody #root gid = nobody #root use chroot = yes #Locked in the source directory address = 192.168.255.142 #Listening address port 873 #Listen on port tcp/udp 873. You can view it through cat /etc/services | grep rsync log file = /var/log/rsyncd.log #Log file location pid file = /var/run/rsyncd.pid #The file location where the process ID is stored hosts allow = 192.168.255.0/24 #Allowed client addresses [wwwroot] path = /var/www/html #The actual path to the source directory comment = Document Root of www.ljm.com read only = yes #Is it read-only dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z #File types that are no longer compressed during synchronization auth users = backuper #Authorized accounts. Multiple accounts are separated by spaces secrets file = /etc/rsyncd_users.db #Data file for storing account information -------------------------------------------- uid = nobody gid = nobody use chroot = yes address = 192.168.255.142 port 873 log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid hosts allow = 192.168.255.0/24 [wwwroot] path = /var/www/html comment = Document Root of www.ly.com read only = yes dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z auth users = backuper secrets file = /etc/user.db #If anonymity is adopted, just remove the "auth users" and "secrets file" configuration items #Create data files for backup accounts vim /etc/user.db backuper:123456 #There is no need to establish a system user with the same name chmod 600 /etc/user.db #Ensure that all users have read permission to the source directory / var/www/html mkdir -p /var/www/html chmod +r /var/www/html/ ls -ld /var/www/html/ #Start rsync service program rsync --daemon netstat -natp | grep rsync #Turn off rsync service kill $(cat /var/run/rsyncd.pid) rm -rf /var/run/rsyncd.pid ---------------------------------------------- #Server: cd /var/www/html echo "hello world" >> 1.txt
#client: mkdir /abc chmod 777 /abc rsync -avz backuper@192.168.255.142::wwwroot /abc/ #View synchronization: cd /abc
#Client: interactive free format configuration echo "123456" > /etc/server.pass chmod 600 /etc/server.pass rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.255.142::wwwroot /abc/
2, inotify
1. Introduction
You can monitor file system changes and respond to notifications
Tuning inotify kernel parameters (optimization)
/Etc / sysctl.conf (kernel parameter configuration file)
inotifywait: used for continuous monitoring and real-time output of results
inotifywatch: used for short-term monitoring and output results after the task is completed
max_queue_events #Monitor event queue size max_user_instances #Maximum number of monitoring instances max_user_watches #Maximum number of monitoring files per instance
2,inotifywait
Format: inotifywait [parameter]
Common parameters | explain |
-m | Continuous monitoring |
-r | Recursively monitor all child objects |
-q | Simplified output information |
-e | Specify which event types to monitor |
3. Operation
#Server: modify rsync configuration file vim /etc/rsyncd.conf uid = root gid = root use chroot = yes address = 192.168.255.142 port 873 log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid hosts allow = 192.168.255.0/24 [wwwroot] path = /var/www/html comment = Document Root of www.ly.com read only = no #close dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z auth users = backuper secrets file = /etc/user.db kill `cat /var/run/rsyncd.pid` netstat -natp | grep rsync rsync --daemon netstat -natp | grep rsync #Client: inotify kernel parameters cat /proc/sys/fs/inotify/max_queued_events cat /proc/sys/fs/inotify/max_user_instances cat /proc/sys/fs/inotify/max_user_watches vim /etc/sysctl.conf ((added at the end) fs.inotify.max_queued_events = 32768 #Monitoring time queue, the default is 16384 fs.inotify.max_user_instances = 1024 #The maximum number of monitoring instances is 128 by default fs.inotify.max_user_watches = 1048576 #The maximum number of monitoring files per instance is 8192 by default #When the amount of directory and file data to be monitored is large or changes frequently, it is recommended to increase the parameter value sysctl -p #Client installation inotify tools yum -y install gcc gcc-c++ tar zxvf inotify-tools-3.14.tar.gz -C /opt cd /opt/inotify-tools-3.14/ ./configure make && make install #Execute the "inotifywait" command, then add and move files in the client / abc directory, and track the screen output results inotifywait -mrq -e modify,create,move,delete /abc #Client write trigger synchronization script vim /opt/inotify.sh #!/bin/bash INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc/" RSYNC_CMD="rsync --apzH --delete --password-file=/etc/server.pass /abc/ backuper@192.168.255.142::wwwroot " $INOTIFY_CMD | while read DIRECTORY EVENT FILE do if [ $(pgrep rsync | wc -l) -le 0 ] ; then $RSYNC_CMD fi done chmod +x /opt/inotify.sh chmod +x /etc/rc.d/rc.local echo "/opt/inotify.sh" >> /etc/rc.d/rc.local