rsync remote synchronization and inotify monitoring

catalogue

1, rsync remote synchronization

1. Introduction to rsync

2. Synchronization mode

3. rsync command

4. Configuration mode

5. Operation (downlink replication)

2, inotify

1. Introduction

2,inotifywait

3. Operation

test

1, rsync remote synchronization

1. Introduction to rsync

rsync is an open source, fast, multi-functional tool that can realize full and incremental local or remote data synchronous backup. And the attribute information of the original data can not be changed to realize the backup and migration characteristics of the data

rsync software is suitable for unix/linux/windows and other operating system platforms

rsync is a fast and very similar file copying tool. It can copy by instinct, remote, or remote daemon. It provides a large number of parameters to control all aspects of its behavior, and allows a very flexible way to realize file transmission and replication

It is famous for its delta transfer algorithm
rsync listening port: 873
rsync operation mode: C/S

2. Synchronization mode

Full backup:

All the original phases are transmitted
The original document and the new submission are transmitted together, which can be copied in full, and the efficiency is low

Incremental backup

Before data transmission, compare the data you have with the data I have through some algorithms, and transfer different data through the network for incremental replication, with high efficiency

3. rsync command

rsync [option]  Original position target position
Common optionsexplain
-rReverse mode, including all files in the directory and subdirectory
-lFor symbolic link files, they are still copied as symbolic link files
-vDisplays details of the synchronization process
-ZCompress when transferring files
-aArchive mode, recursion and retention of object attributes, equivalent to - rlptgoD
-pKeep the permission mark of the file
-tTime stamp of retention file
-gKeep the belonging group flag of the file (for super users only)
-oKeep the master tag of the file (for super users only)
-HKeep hard linked files
-ARetain ACL national information
-DKeep equipment documents and other special documents
--deleteDo not delete files that exist in the target location but not in the original location
--checksumDecide whether to skip the file based on the checksum of the object

4. Configuration mode

Format I:
user name@Host address::Shared module name

Format 2:
rsync://User name @ host address / shared module name

5. Operation (downlink replication)

Server: 192.168.255.142

#Basic environment configuration
systemctl stop firewalld.service 
setenforce 0

#Configure rsync source server
rpm -q rsync
yum -y install rsync

#Create the / etc/rsyncd.conf configuration file
vim /etc/rsyncd.conf      		#Add the following configuration

uid = nobody					#root
gid = nobody					#root
use chroot = yes				#Locked in the source directory                                                
address = 192.168.255.142		#Listening address
port 873                     	#Listen on port tcp/udp 873. You can view it through cat /etc/services | grep rsync                                           
log file = /var/log/rsyncd.log	#Log file location                  
pid file = /var/run/rsyncd.pid	#The file location where the process ID is stored                  
hosts allow = 192.168.255.0/24	#Allowed client addresses
[wwwroot]                                                               
path = /var/www/html     		#The actual path to the source directory                               
comment = Document Root of www.ljm.com	
read only = yes             	#Is it read-only                                     
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z  	#File types that are no longer compressed during synchronization      
auth users = backuper           #Authorized accounts. Multiple accounts are separated by spaces                                 
secrets file = /etc/rsyncd_users.db #Data file for storing account information      
--------------------------------------------
uid = nobody
gid = nobody
use chroot = yes                                                
address = 192.168.255.142
port 873                                           
log file = /var/log/rsyncd.log                  
pid file = /var/run/rsyncd.pid                  
hosts allow = 192.168.255.0/24
[wwwroot]                                                               
path = /var/www/html                               
comment = Document Root of www.ly.com	
read only = yes                                     
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z     
auth users = backuper                                
secrets file = /etc/user.db

#If anonymity is adopted, just remove the "auth users" and "secrets file" configuration items

#Create data files for backup accounts
vim /etc/user.db
backuper:123456			#There is no need to establish a system user with the same name
chmod 600 /etc/user.db

#Ensure that all users have read permission to the source directory / var/www/html
mkdir -p /var/www/html
chmod +r /var/www/html/
ls -ld /var/www/html/

#Start rsync service program
rsync --daemon
netstat -natp | grep rsync

#Turn off rsync service
kill $(cat /var/run/rsyncd.pid)
rm -rf /var/run/rsyncd.pid
----------------------------------------------


#Server:
cd /var/www/html
echo "hello world" >> 1.txt

#client:
mkdir /abc
chmod 777 /abc
rsync -avz backuper@192.168.255.142::wwwroot /abc/

#View synchronization:
cd /abc

#Client: interactive free format configuration
echo "123456" > /etc/server.pass
chmod 600 /etc/server.pass

rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.255.142::wwwroot /abc/

2, inotify

1. Introduction

You can monitor file system changes and respond to notifications

Tuning inotify kernel parameters (optimization)

/Etc / sysctl.conf (kernel parameter configuration file)

inotifywait: used for continuous monitoring and real-time output of results

inotifywatch: used for short-term monitoring and output results after the task is completed

max_queue_events    	#Monitor event queue size
max_user_instances  	#Maximum number of monitoring instances
max_user_watches    	#Maximum number of monitoring files per instance  

2,inotifywait

Format: inotifywait [parameter]
Common parametersexplain
-mContinuous monitoring
-rRecursively monitor all child objects
-qSimplified output information
-eSpecify which event types to monitor

3. Operation

#Server: modify rsync configuration file
vim /etc/rsyncd.conf
uid = root
gid = root
use chroot = yes
address = 192.168.255.142
port 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.255.0/24
[wwwroot]
path = /var/www/html
comment = Document Root of www.ly.com 
read only = no                                    #close
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z
auth users = backuper
secrets file = /etc/user.db


kill `cat /var/run/rsyncd.pid`
netstat -natp | grep rsync
rsync --daemon
netstat -natp | grep rsync

#Client: inotify kernel parameters
cat /proc/sys/fs/inotify/max_queued_events
cat /proc/sys/fs/inotify/max_user_instances 
cat /proc/sys/fs/inotify/max_user_watches 

vim /etc/sysctl.conf ((added at the end)
fs.inotify.max_queued_events = 32768      #Monitoring time queue, the default is 16384
fs.inotify.max_user_instances = 1024	  #The maximum number of monitoring instances is 128 by default
fs.inotify.max_user_watches = 1048576	  #The maximum number of monitoring files per instance is 8192 by default
#When the amount of directory and file data to be monitored is large or changes frequently, it is recommended to increase the parameter value

sysctl -p

#Client installation inotify tools
yum -y install gcc gcc-c++ 
tar zxvf inotify-tools-3.14.tar.gz -C /opt

cd /opt/inotify-tools-3.14/
./configure
make && make install

#Execute the "inotifywait" command, then add and move files in the client / abc directory, and track the screen output results
inotifywait -mrq -e modify,create,move,delete /abc

#Client write trigger synchronization script
vim /opt/inotify.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc/"
RSYNC_CMD="rsync --apzH --delete --password-file=/etc/server.pass /abc/ backuper@192.168.255.142::wwwroot "
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
do
    if [ $(pgrep rsync | wc -l) -le 0 ] ; then
        $RSYNC_CMD
	fi
done

chmod +x /opt/inotify.sh
chmod +x /etc/rc.d/rc.local
echo "/opt/inotify.sh" >> /etc/rc.d/rc.local

test

Tags: Linux network server

Posted on Sun, 21 Nov 2021 05:23:48 -0500 by b