MFS (MooseFS Distributed File System)

Author: Zhang Shoufu
 Time: 2020-05-22
w x: y18163201

MooseFS[MFS] is a fault-tolerant network distributed file system.It distributes data across multiple physical servers and presents users with a unified resource.
Official address: http://www.moosefs.com/

Introduction to MFS

MooseFS is a fault-tolerant, highly available, high-performance, scalable network distributed file system that distributes data across multiple storage servers that are virtual disks to users and POSIX-compliant and supported like any other UNIX file system:

  • Hierarchy: Files and Folders
  • File Properties
  • Special Files:
  • Hard and Soft Links
  • Security Control and ACL

MooseFS pro Enterprise Architecture Diagram, the normal version we use, does not have the following slave servers and cannot use its own high availability directly. We will introduce another high availability method, keep alive later
<img src="/Users/zhumengyuan/Library/Application Support/typora-user-images/image-20200522104005363.png" alt="image-20200522104005363" style="zoom:189%;" />

Advantages of MooseFS:

  • High availability: No single point of failure for the storage node (Chunk server), metadata for the file system is saved as two or more copies on the physical redundancy server, and user data redundancy is distributed on the storage server in the system
  • Low cost data security: MooseFS allows users to save a lot of hard disk space and maintain the same level of data redundancy.
  • High performance: Supports high performance I/O operations, allowing user data to be read/written simultaneously on rainy storage nodes.

Four roles for the MooseFS file system

  • Management server master:
    Responsible for management of individual data storage servers (chunk servers), file read and write scheduling, file space recovery and recovery, multi-node copy

  • Metalogger server Metalogger:
    Responsible for backing up master server change log file, file type changelog_ml.*.mfs to replace master server when it has a problem

  • Data Storage Server chunk server:
    Listens to the management server scheduling, provides storage space, and does not provide data transmission from the client.A server that actually stores user data.When storing files, you first divide the files into blocks, which are then copied between data server chunkserver s (how many copies can be manually specified, with a recommended setting of 3).Data servers can be multiple, and the more they are, the greater the "disk space" available and the more reliable they are

  • Client mounting uses client computer:
    Mount the shared storage and use by the process MFS server.Mount the hard disks shared by the data storage servers managed on the process management server through the fuse kernel interface.Shared file systems are used similarly to nfs.Hosts that use the MFS file system to store and access are called MFS clients. After successfully hooking up the MFS file system, you can share this virtual storage just like you used NFS before.

Workflow of MooseFS File System

Process for reading files


1, the client asks the master server where the resource is located
2, then the master server returns the location of the resource on the chunk server
3, the client takes the returned address to find the resource, and chunk server returns the actual resource

Procedures for storing data


1, the client tells the master server that he wants to store data
2, and then the master server side to find the free location over the chunk server
3, if there is still a free location on the chunk server, it will be returned to the master server location
4, then master server returns to the client, and the client takes this location to store the data
5. The chunk server stores the data in multiple ways and returns it to the client when it is finished.
6, then disconnect

Install Build MooseFS

Host Planning

host name role IP Address
master_11 Managing server 10.0.0.11/24
node_1_12 Metalogger server 10.0.0.12/24
node_2_13 chunk server 1 10.0.0.13/24
node_3_14 chunk server 2 10.0.0.14/24
Node_4_15 client computers 10.0.0.15/24

Environmental preparation

The host needed for the experiment must support the FUSE kernel module, because the mfs client program, that is, the command to load the mfs disk system, uses fuse, so as long as you want to hang on the mfs server, the necessary prerequisite is to install fuse first, so that the mfs can be compiled through

FUSE: Filesystem in Userspace is a concept in the operating system that refers to a file system implemented entirely in the user state. Note: FUSE comes with more than 2.6 cores

  • All servers use: centos 7.5
  • Turn off selinux and IPtables

Start building

1) Add yum source to all servers

curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo
sed -i "/^gpgcheck/c gpgcheck=0" /etc/yum.repos.d/MooseFS.repo
yum clean  all 
yum list

2) Install on the primary server

[root@master_11 /]# yum install -y moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli
[root@master_11 /]# rpm -ql moosefs-master
/etc/mfs/mfsexports.cfg.sample
/etc/mfs/mfsmaster.cfg.sample
/etc/mfs/mfstopology.cfg.sample

3) On the metaloggers server

[root@node_1_12 /]# yum install moosefs-metalogger -y
[root@node_1_12 /]# rpm -ql moosefs-metalogger
/etc/mfs/mfsmetalogger.cfg.sample

4) chunk server on data node

[root@node_2_13 monitor]# yum install moosefs-chunkserver  -y
[root@node_3_14 monitor]# yum install moosefs-chunkserver  -y
[root@node_2_13 monitor]# rpm -ql moosefs-chunkserver
/etc/mfs/mfschunkserver.cfg.sample
/etc/mfs/mfshdd.cfg.sample

5) Install on Client Node

[root@node_4_15 ~]# yum -y  install moosefs-client
[root@node_4_15 ~]# rpm -ql moosefs-client
/etc/mfs/mfsmount.cfg.sample

6) Modify shared directories and permissions on the master side

[root@master_11 ~]# echo '10.0.0.0/24 /mfs_test rw,admin,maproot=0:0' >> /etc/mfs/mfsexports.cfg
[root@master_11 /]# systemctl start moosefs-master.service
[root@master_11 /]# systemctl enable moosefs-master.service
[root@master_11 /]# netstat -ntalp | grep mfsmaster
tcp        0      0 0.0.0.0:9419            0.0.0.0:*               LISTEN      3969/mfsmaster      
tcp        0      0 0.0.0.0:9420            0.0.0.0:*               LISTEN      3969/mfsmaster      
tcp        0      0 0.0.0.0:9421            0.0.0.0:*               LISTEN      3969/mfsmaster

7) Configure the address of the manage ment side on the log server

[root@node_1_12 mfs]# vim /etc/mfs/mfsmetalogger.cfg
MASTER_HOST = 10.0.0.11
[root@node_1_12 mfs]# systemctl start moosefs-metalogger.service
[root@node_1_12 mfs]# systemctl enable moosefs-metalogger.service
[root@node_1_12 mfs]# netstat -ntalp | grep mfs
tcp        0      0 10.0.0.12:45152         10.0.0.11:9419          ESTABLISHED 2837/mfsmetalogger
// You can see that the log and master sides have successfully established a connection

8) Modify the chunk server side and start

[root@node_2_13 mfs]# vim mfschunkserver.cfg
MASTER_HOST = 10.0.0.11
[root@node_2_13 mfs]# vim mfshdd.cfg
/tmp    //Where is the data stored? A separate partition will be configured for general work
[root@node_2_13 mfs]# systemctl start moosefs-chunkserver.service
[root@node_2_13 mfs]# systemctl enable moosefs-chunkserver.service
[root@node_2_13 mfs]# netstat -ntalp | grep mfs
tcp        0      0 0.0.0.0:9422            0.0.0.0:*               LISTEN      1387/mfschunkserver 
tcp        0      0 10.0.0.13:45530         10.0.0.11:9420          ESTABLISHED 1387/mfschunkserver 
// You can see that he's connected to the master side, and another chunk server is configured the same way

[root@node_2_13 tmp]# ls 
00  22  44  66  88  AA                            C8  EA
01  23  45  67  89  AB                            C9  EB
02  24  46  68  8A  AC                            CA  EC
03  25  47  69  8B  AD                            CB  ED
04  26  48  6A  8C  AE                            CC  EE
// Blocked storage, not directly recognizable by humans

9) Client mount

[root@node_4_15 mfs]# vim mfsmount.cfg
mfsmaster=10.0.0.11
[root@node_4_15 mfs]# mfsmount /tmp/ -o nonempty -H 10.0.0.11:/mfs_test -p
mfsmaster accepted connection with parameters: read-write,restricted_ip,admin ; root mapped to root:root
 The mounted directory is the master management sideMfshdd.cfgShared directories and permissions inside, if I do not specify a password for the connection, I need to use the -o nonempty parameter to trust the password

10) Unmount

[root@node_4_15 ~]# umount -t fuse.mfs /tmp/

11) Open web interface

Execute on master server

[root@master_11 ~]# mfscgiserv 
lockfile created and locked
starting simple cvimgi server (host: any , port: 9425 , rootpath: /usr/share/mfscgi)

Then browser access IP:9425 port

Optimize operation

Set the number of copies of file copy (mfssetgoal)

Execute commands on the client side

[root@node_4_15 tmp]# mfssetgoal -r 1 /tmp/
-r Recursion 1 represents one copy and is then implemented on which mount directory

View how many copies of a file have been copied (mfscheckfile\mfsfileinfo)

Execute commands on the client side

mfscheckfile /tmp/sshd /tmp/1.txt 
/tmp/sshd:
 chunks with 1 copy:             1
/tmp/1.txt:
chunks with 1 copy:              1
// Chuks with 1 copy represents the current copy

Test whether copy number setting succeeded

[root@node_4_15 tmp]# mfssetgoal -r 2 /tmp/
/tmp/:
 inodes with goal changed:                      47
 inodes with goal not changed:                   0
 inodes with permission denied:                  0
[root@node_4_15 tmp]# mfscheckfile /tmp/sshd /tmp/1.txt 
/tmp/sshd:
 chunks with 2 copies:            1

mfsfileinfo View File Properties

Parameters for mfsfileinfo:

  • -q: concise mode, showing only the number of copies, the number of copies
  • -c: Receive block checksums from chunkserver
  • -s: Compute file signatures (using checksums)
[root@node_4_15 tmp]# mfsfileinfo /tmp/sshd 
/tmp/sshd:
        chunk 0: 0000000000000038_00000001 / (id:56 ver:1)
                copy 1: 10.0.0.13:9422 (status:VALID)
                copy 2: 10.0.0.14:9422 (status:VALID)
// From the above, you can see where two copies exist

mfsgetclass shows the number of copies of a file and the number of copies of a directory

[root@node_4_15 tmp]# mfsgetsclass -r /tmp/
/tmp/:
 files with goal                2 :         46
 directories with goal          2 :          2

mfsdirinfo displays the status of the directory

Usage: mfsdirinfo [-nhHkmg] [-idfclsr] [-p] name [name...]

  • -n - Display numbers in a simple format
  • -h-Human readable number with cardinality 2 prefix (IEC 60027)
  • -H - Readable numbers using cardinality 10 prefix (SI)
  • -k - Display pure numbers in thousands (binary thousands - 1024)
  • -m - Display pure numbers in mebis (binary mega - 1024 ^ 2)
  • -g - Show pure numbers in deficit (binary gigabytes - 1024 ^ 3)

'Display'switch:

  • -i - Display the number of index nodes
  • -d - Display the number of directories
  • -f - Display the number of files
  • -c - Number of display blocks
  • -l - Display length
  • -s - Display size
  • -r - Show realsize
    'Mode'switch:
  • -p - Precise mode
[root@node_4_15 tmp]# mfsdirinfo -h /tmp/
/tmp/:
 inodes:           48           //Currently used inode number
  directories:      2           //Two directories
  files:           46           //46 files
 chunks:           33           //33 blocks occupied
 length:        28KiB           //The total file size is 28K
 size:         2.3MiB           //The sum of block lengths is 2.3M
 realsize:     4.6MiB           //Disk space used because we copied two copies, so.

 [root@node_4_15 tmp]# mfsdirinfo -h -p /tmp/
/tmp/:
 inodes:           48
  directories:      2
  files:           46
 chunks:           33
 length:        28KiB
 size:         2.3MiB
 realsize:     4.6MiB
/tmp/ (precise data):
 inodes:           48
  directories:      2
  files:           46
 chunks:           33
 length:        28KiB
 size:         2.3MiB
 realsize:     4.6MiB

mfssetgoal and mfsgetgoal commands

mfssetgoal sets the number of copies of the target copy

mfssetgoal 3 /tmp  //3 copies in tmp directory, but two copies are backed up in tmp directory
mfssetgoal -r 3 /tmp  //copy 3 copies of all the files in the tmp directory, including the subdirectories inside

mfsgetgoal gets the number of copies of the target copy

[root@node_4_15 tmp]# mfsgetgoal /tmp/
/tmp/: 2
[root@node_4_15 tmp]# mfsgetgoal /tmp/sshd
/tmp/sshd: 2

mfsgetquota Gets the quota given by the execution directory

[root@node_4_15 tmp]# mfsgetquota -h /tmp/
/tmp/: (current values | soft quota | hard quota) ; soft quota grace period: default
 inodes   |     47 |      - |      - |
 length   |  28KiB |      - |      - |
 //Current value soft quota hard quota

 [root@node_4_15 tmp]# mfsgetquota -h /tmp/123/
/tmp/123/: (current values | soft quota | hard quota) ; soft quota grace period: default
 inodes   |      0 |      - |      - |
 length   |     0B |      - |      - |

mfsrsettrashtime and mfsgettrashtime command dustbin storage time

Deleted files are stored in the trash bin at quarantine time, which can be verified with mfsgettrashtime or set with mfssettrashtime.Set time is calculated in hours, set in seconds, less than an hour is calculated in hours.

Introduction to mfs web presentation interface

  • 1. Ip address of current master
  • 2. Current mfs version
  • 3. Current memory usage
  • 4. cpu usage
  • 5. Total disk size
  • 6. Available disk size

Profile Details

Mfsmaster.cfgMaster Master Profile Details

#######Run Options######################
# WORKING_USER = mfs   #User running master daemon, default is mfs
# WORKING_GROUP = mfs  #User group running master daemon, default is mfs
# SYSLOG_IDENT = mfsmaster #The process name to place in the syslog message (default is mfsmaster), the message is logged in/var/log/messages by default
# LOCK_MEMORY = 0       #Whether to execute mlockall() to avoid mfsmaster process overflow (default 0); 
# LIMIT_GLIBC_MALLOC_ARENAS = 4  #Linux only: Limit malloc zones to a given value - Prevent servers from using large amounts of virtual memory (default 4)
# DISABLE_OOM_KILLER = 1 #Linux only: Disable memory killer (default 1)
# NICE_LEVEL = -19       #Good level of running the daemon (default is -19; note: the process must start as root to increase priority, and if priority settings fail, the process retains a good level of start)
# FILE_UMASK = 027       #Set default umask for groups and others (user always 0, default 027, block writes to groups and blocks all others)
# DATA_PATH = /usr/local/mfs-3.0.94/var/mfs #Where to store the daemon lock file (default is/usr/local/mfs-3.0.94/var/mfs)
# EXPORTS_FILENAME = /usr/local/mfs-3.0.94/etc/mfs/mfsexports.cfg #Mfsexports.cfgLocation of file (default is/usr/local/mfs-3.0.94/etc/mfs/mfsexports.cfg)
# TOPOLOGY_FILENAME = /usr/local/mfs-3.0.94/etc/mfs/mfstopology.cfg #Mfstopology.cfgLocation of file (default is/usr/local/mfs-3.0.94/etc/mfs/mfstopology.cfg)
# BACK_LOGS = 50         #The number of metadata change log files (the default is 50), resulting in changelog.0.mfs-changelog.49.mfs, which hold metadata change records.
# METADATA_SAVE_FREQ = 1 #How often does the master save metadata, defaulting to 1 hour.
# BACK_META_KEEP_PREVIOUS = 1  #The number of previous metadata files to keep (default is 1),metadata.mfs.back.1 Leave a backup file like this.
# CHANGELOG_PRESERVE_SECONDS = 1800 #How many seconds of change logs must be kept in memory (default is 1800; this sets the minimum value, since logs are stored in 5k blocks, the actual number may be larger; zero disables additional log storage)
# MISSING_LOG_CAPACITY = 100000     #How many missing blocks will be stored in the host (up to 100*MISSING_can be allocated)LOG_CAPACITY Byte Memory)
#######Command Connection Options#####################
# MATOML_LISTEN_HOST = *      #The IP address to listen for, for the connection between metalogger and masters, * for any.
# MATOML_LISTEN_PORT = 9419   #The listening port defaults to 9419, which is used for connections between metalogger and masters
#######CHUNKSERVER Connection Options###############
# MATOCS_LISTEN_HOST = *      #IP address to listen for chunkservers connections, defaults to all
# MATOCS_LISTEN_PORT = 9420   #Listening port defaults to 9420
# MATOCS_TIMEOUT = 10               #Default timeout for master-chunkserver connections (default is 10 seconds)
# AUTH_CODE = mfspassword           #Optional authentication string.When Defined - Only Has Same AUTH_CODE's block server allows connections to the primary station.When undefined (default) - then all chunkservers are allowed.If you want to turn on chunkserver authentication,
#Define AUTH_in all chunkservers firstCODE (and reload/restart them), then uncomment this option and reload/restart master to reload the current connection chunkservers without disconnecting.When chunkservers will make a new connection, the new AUTH_will be usedCODE.
# REMAP_BITS = 24                   #Optional IP class remap.
# REMAP_SOURCE_IP_CLASS = 192.168.1.0
# REMAP_DESTINATION_IP_CLASS = 10.0.0.0
#######CHUNKSERVER Work Options##############
#REPLICATIONS_DELAY_INIT = 300           #Initial delay before starting replication (seconds) (default 300)
# CHUNKS_LOOP_MAX_CPS = 100000           #Chunks loops should not check more blocks per second (default is 100000)
# CHUNKS_LOOP_MIN_TIME = 300             #Chunks loop should not be completed in less than a given number of seconds (default is 300)
# CHUNKS_SOFT_DEL_LIMIT = 10             #Maximum number of blocks deleted on a chunk server (default is 10)
# CHUNKS_HARD_DEL_LIMIT = 25             #Maximum number of blocks deleted on a chunkserver (default is 25)
# CHUNKS_WRITE_REP_LIMIT = 2,1,1,4       #The maximum number of blocks is copied to a chunkserver, with the first restriction being more dangerous blocks (blocks with only one copy); the second restriction being undergoal blocks (blocks with fewer copies than the specified target);
#The third limit is the arithmetic average between the balanced and surrounding space usage servers; the fourth limit is the balance between other servers (very low or very high space usage); here the official mfs recommendation is that the first number is greater than or equal to the second, the second is greater than or equal to the third, and the fourth is greater than or equal to the third (the first limit > =the second limit > =The third restriction < =the fourth restriction).
# CHUNKS_READ_REP_LIMIT = 10,5,2,5       #Maximum number of blocks copied from a chunkserver (default is 10,5,2,5) A number equals four identical numbers, separated by a colon, the restriction group and the write limit are the same, and the relationship between the numbers should be the same in the write limit (1st> = 2nd> = 3rd <= 4th)
# CS_HEAVY_LOAD_THRESHOLD = 100          #Chukserver load value defaults to 100
# CS_HEAVY_LOAD_RATIO_THRESHOLD = 5.0    #Chukserver load threshold rate (default is 5.0, whenever the chunkserver load is higher than the previously specified threshold and this ratio is higher than the average load, then chunkserver switches to grace mode)
# CS_HEAVY_LOAD_GRACE_PERIOD = 900       #Defines how long chunkserver will stay in "grace" mode (in seconds)
# ACCEPTABLE_PERCENTAGE_DIFFERENCE = 1.0 #Percentage of maximum difference in chunkserver's space usage (default 1.0; valid 0.1 - 10.0)
# PRIORITY_QUEUES_LENGTH = 1000000       #Length of the priority queue (endangered, undergoal, etc. - blocks should be processed first, default 1000000).
# CS_MAINTENANCE_MODE_TIMEOUT = 0        #Maximum number of seconds the server can be in maintenance mode (default is 0 - which means "always").
# CS_TEMP_MAINTENANCE_MODE_TIMEOUT = 1800 #Maximum number of seconds the server can be in "temporary" maintenance mode (after reconnecting the server to normal mode automatically, the server switches to this mode, default is 1800)
###########Customer Connection Options########################
# MATOCL_LISTEN_HOST = *                 #IP address used to listen for client (installation) connections (* denotes any)
# MATOCL_LISTEN_PORT = 9421              #Port Listening Client (mount) Connection (default 9421)
###########Customer Job Selection########################
# SESSION_SUSTAIN_TIME = 86400           #Time to maintain a disconnected client session (in seconds; default is 86400 = 1 day)
# QUOTA_DEFAULT_GRACE_PERIOD = 604800    #Default grace period (seconds) for soft quotas (604800 - 7 days by default) is not recommended: QUOTA TIME_LIMIT
# ATIME_MODE = 0                         
#Modify mode (default 0 - Always modify atime during access)
#All modes:
#0 - Always modify the atime of files, folders, and symbolic links.
#1 - Always modify the atime, but only in the case of files (do not modify the atime in the case of folders and symbolic links).
#2 - Modify atime only when ctime or mtime is lower than current time is higher than ctime or mtime, and when current atime is earlier than 24 hours.Operate on all objects during access (such as the "relatime" option on Linux).
#3 - Same as above but only in the case of files.If folders and symbolic links cannot modify atime.
#4 - Do not modify atime (such as the "noatime" option) during access.

Mfsexports.cfgProfile Details

# cat/usr/local/mfs/etc/mfs/Mfsexports.cfg#Similar to the NFS/etc/exports file, which clients are allowed to mount

The format of a line: [ip range] [path] [options]

ip range:

*: Represents any ip (same as 0.0.0/0)

A.B.C.D: Indicates a single IP

A.B.C.D-E.F.G.H: IP Address Range

A.B.C.D/XX:A.B.C.D Network and Subnet Masks

A.B.C.D/E.F.G.H:A.B.C.D network and E.F.G.H network mask
path:

.: Special "path", meaning "meta"

/...: Path in mfs structure

options:

ro/rw/readonly/readwrite: literally read-only/read-write

alldirs: any subdirectory can be loaded as root

dynamicip:ip is tested only during the first authentication, so the client can use the same session ID from any IP

ignoregid: Group ID (gid)

admin: administrative rights-current: allow manipulation of quota values

maproot=UID[:GID]: Treat all root (uid zero) operations as actions performed by the user, where uid equals UID and GID equals GID (default GID for the user if GID is not specified)

mapall=UID[:GID}: For all operations (for both options, UID and/or GID can be specified as the user or group name that exists on the host) as described above

password=TEXT: Force authentication to use the given password

md5pass=MD5: As mentioned above, but the password is defined as an MD5 hash (MD5 specifies 128 hexadecimal digits)

minversion=VER: Only clients with version number equal to or greater than VER are allowed (VER can be specified as X or X.Y or X.Y.Z)

mingoal=N: Target below N is not allowed (N should be a number from'1'to'9')

maxgoal=N: Target setting above N is not allowed

mintrashtime=TIMEDURATION: Do not set garbage time under TIME DURATION (time DURATION can specify seconds or combinations of elements W, D, H, M, S in the set order)

maxtrashtime=TIMEDURATION: Time beyond TIMEDURATION is not allowed (TIMEDURATION can be specified above)

Default value:

readonly,maproot=999:999,mingoal=1,maxgoal=9,mintrashtime=0,maxtrashtime=4294967295

Mfsmetalogger.cfgProfile Details

###########Run Options############################
# WORKING_USER = mfs
# WORKING_GROUP = mfs
# SYSLOG_IDENT = mfsmetalogger    #The name of the process to place in the syslog message (default is mfsmetalogger)
# LOCK_MEMORY = 0                 
# LIMIT_GLIBC_MALLOC_ARENAS = 4
# DISABLE_OOM_KILLER = 1
# NICE_LEVEL = -19
# FILE_UMASK = 027
# DATA_PATH = /usr/local/mfs-3.0.94/var/mfs
# BACK_LOGS = 50
# BACK_META_KEEP_PREVIOUS = 3      #The number of previous metadata files to keep (default is 3), metadata_Ml.mfs.back.1-metadata_Ml.mfs.back3 Leave three such backup files.
# META_DOWNLOAD_FREQ = 24          #Metadata download frequency is in hours (default is 24, should be at least BACK_LOGS/2).Note that this is generally changed to 1.
META_DOWNLOAD_FREQ = 1
##########MASTER Connection Options######################
# MASTER_RECONNECTION_DELAY = 5    #If not, delay a second (default is 5) before the next attempt to reconnect to the host
# BIND_HOST = *                    #Local address used to connect to the host (default is *, default local address)
# MASTER_HOST = mfsmaster          #The MooseFS host, which only allows IP to be used in a single-host installation (default is mfsmaster), is also where modifications are required.
# MASTER_PORT = 9419               #master's port
# MASTER_TIMEOUT = 10              #Timeout for connection to master, default is 10 seconds

Mfschunkserver.cfgProfile Details

##########Run Options###########################
# WORKING_USER = mfs
# WORKING_GROUP = mfs
# SYSLOG_IDENT = mfschunkserver  #Process name placed in syslog message (default is mfschunkserver)
# LOCK_MEMORY = 0
# LIMIT_GLIBC_MALLOC_ARENAS = 4
# DISABLE_OOM_KILLER = 1
# NICE_LEVEL = -19
# FILE_UMASK = 027
# DATA_PATH = /usr/local/mfs-3.0.94/var/mfs  #Where to store the daemon lock file (default is/usr/local/mfs-3.0.94/var/mfs)
# HDD_CONF_FILENAME = /usr/local/mfs-3.0.94/etc/mfs/mfshdd.cfg  #Mfshdd.cfgFile location, location of profile allocated to MFS disk space
# HDD_TEST_FREQ = 10  #Block test cycle (default is 10 seconds)
# HDD_LEAVE_SPACE_DEFAULT = 256MiB       #How much space should be left on each hard drive (default: 256MiB)
# HDD_REBALANCE_UTILIZATION = 20         #Percentage of total working time allowed by the bulk server for hdd space rebalancing
# HDD_ERROR_TOLERANCE_COUNT = 2          #How many I/O errors (COUNTs) are allowed on a single hard drive in a given number of seconds (PERIOD); if the number of errors exceeds this setting, harmful hard drives will be marked as damaged
# HDD_ERROR_TOLERANCE_PERIOD = 600
# HDD_FSYNC_BEFORE_CLOSE = 0             #Enable/disable fsync before chunk closes
# HDD_SPARSIFY_ON_WRITE = 1              #Enable/disable sparsification during writing (skip zero)
# WORKERS_MAX = 250                      #Maximum number of active sessions
# WORKERS_MAX_IDLE = 40                  #Maximum number of idle sessions
###########and master Connection Options#################
# LABELS =                               #Label string (empty - no label by default)
# BIND_HOST = *                          #Local address for master connection (default is *, default local address)
# MASTER_HOST = mfsmaster                #The MooseFS host, which only allows IP to be used in a single-host installation (default is mfsmaster).This is where you want to modify it.
# MASTER_PORT = 9420                     #Ports connected to master
# MASTER_TIMEOUT = 10                    #Timeout for connection to master (default is 10 seconds)
# MASTER_RECONNECTION_DELAY = 5          #If not, delay a second (default is 5) before the next attempt to reconnect to the host
# AUTH_CODE = mfspassword                #authentication string (used only if the master needs authorization)
##########Client Connection Options####################
# CSSERV_LISTEN_HOST = *                 #IP address to listen for client (mount) connections (*means anything)
# CSSERV_LISTEN_PORT = 9422              #Port Listening Client (mount) Connection (default 9422)

Fsmshdd.cfgProfile Details

This file retains the definition of the installation point (path) of the hard drive for use with the block server.

A path can start with more than one character, which can switch other options:

*: Indicates that this hard drive is marked for deletion and that all data will be copied to other hard drives (usually on other chunkserver s)

<: Indicates that all data from this hard drive should be moved to another hard drive

>: Indicates that all data from other hard drives should be moved to that hard drive

: Significant changes to the total count will not mark this drive as damaged

If you use both'<'and'>' drives, you can only move data between them

Optional space limits (after each installation point) can be specified in two ways:

Set the space to unused on the hard drive (this overrides theMfschunkserver.cfgDefault settings for)

Limit the space used on your hard disk, such as: /mnt/hd4-5GiB or/mnt/hd4 1.5TiB

Mfsmount.cfgMount Files

#OptionalMfsmount.cfgThe file can be used to specify a default value for mfsmount.Default installation options can be specified on one or more lines separated by commas.For example (as you can see below, it cannot be explained):
# nosuid,nodev
# mfsmaster=mfsmaster    
# mfspassword=secret    #Mainly here, if master sets the mount password, you will have to type it manually if you don't set it here, and then you will need to set the password here if the client wants to start the automount.
#The default installation point can also be set.The default installation point must start with'/'and be a fully qualified path.For example:
# /mnt/mfs

data recovery

Error Delete Data Recovery from Trash

1) client We have mounted the directory, delete a file directly inside the file and restore it to him

> [root@node_5_16 tmp]# echo "this is test del file" >> test.png
[root@node_5_16 tmp]# ls 
test_file.txt

2) Mount garbage bin

First check when deleted files are stored in the dustbin
[root@node_5_16 tmp]# mfsgettrashtime /tmp/
/tmp/: 3600

//Mount dustbin
[root@node_5_16 tmp]# mfsmount -m /mnt/ -H 10.0.0.11:/mfs_test
mfsmaster accepted connection with parameters: read-write,restricted_ip

3) Delete the data and find the location of the file in the dustbin

[root@node_5_16 tmp]# rm -f test.png 
[root@node_5_16 trash]# find /mnt/ -name *test.txt
/mnt/trash/0F9/000000F9|test.txt

Deleted file names can actually be found in the dustbin. File names consist of an 8-bit hexadecimal i-node and deleted file names.Instead of'/'between the file name and i-node, use'|'.If the length of a file name exceeds the operating system limit (usually 255 characters), the excess will be deleted.File names deleted from all paths from the mount point can still be read and written.It is important to note that deleted files are using file names (note that the file names are in two parts) and must be enclosed in single quotes.

4) Restore data

[root@node_5_16 tmp]# cd /mnt/trash/0F9/
[root@node_5_16 0F9]# ls 
000000F9|1.txt  undel
[root@node_5_16 0F9]# mv 000000F9\|1.txt undel/
[root@node_5_16 0F9]# cd /tmp/
[root@node_5_16 tmp]# ls
1.txt
[root@node_5_16 tmp]# cat test.txt 
this is test del file

5) Note:
The garbage collection bin doesn't have to be mounted all the time. You can wait until you want to recover the data from the garbage bin.

Tags: Linux yum network RPM vim

Posted on Sun, 24 May 2020 21:12:59 -0400 by m@ndio