MFS: distributed storage

1, Introduction
MooseFS is a distributed network file system with redundant error tolerance function. It stores data on multiple physical servers or separate disks or partitions to ensure that one data has multiple backup copies. The whole distributed network file system cluster looks like a resource to clients or users. From its operation on files, MooseFS is equivalent to a UNIX like file system:
1. mfs is a hierarchical directory tree structure
2. Store POSIX standard supported file attributes (permissions, last access and modification time)
3. Support special files, such as block devices, character devices, pipes and sockets, link files (symbolic links and hard links)
4. Support file system access based on IP address and password
2, Functional function
·MooseFS is a fault-tolerant network distributed file system.
·It stores data on multiple physical servers, and presents users with a unified resource.

3, Component
·Metadata server (Master): responsible for managing the file system and maintaining metadata in the whole system.
·Metadata log server (MetaLogger): backs up the change log files of the Master server. The file type is changelog_ml.*.mfs. When the Master server data is lost or damaged, files can be obtained from the log server for recovery.
·Chunk Server: the server that actually stores data. When storing files, the files will be saved in blocks and copied between data servers. The more data servers, the greater the "capacity" that can be used, the higher the reliability and the better the performance.
·Client: the MFS file system can be mounted like NFS, and the operation is the same.
(metadata is defined as data describing data and descriptive information about data and information resources.)
4, Processing of MFS reading data
·The client sends a read request to the metadata server.
·The metadata server informs the client of the location where the required data is stored (the IP address and the Chunk number of the Chunk Server).
·The client requests to send data to a known Chunk Server.
·The Chunk Server sends data to the client.
5, MFS write data processing
·The client sends a write request to the metadata server.
·The metadata server interacts with the Chunk Server (only when the required Chunks exist), but the metadata server only creates new Chunks on some servers. After the creation is successful, the Chunk Servers inform the metadata server that the operation is successful.
·The metadata server tells the client which Chunks of which Chunk Server can write data.
·The client writes data to the specified Chunks Server.
·The Chunk Server synchronizes data with other chunk servers. After the synchronization is successful, the Chunk Server informs the client that the data is written successfully.
·The client informs the metadata server that this write is complete.
6, Case implementation

IPhost name
192.168.94.139master
192.168.94.140Metalogger server
192.168.94.141Chunk server1
192.168.94.142Chunk server2
192.168.94.143client

Package download link: https://pan.baidu.com/s/1QqREtYpjnEuAAnweiRNwAQ (1.6.27)
Extraction code: o15d
Link: https://pan.baidu.com/s/18VGAhdU9_X8KC0_xmPz7_Q (3.0.84)
Extraction code: 1czb
Upload the software package to the / root directory of each node
Ensure that all virtual machine firewalls are turned off
1, master server node:
1. Environmental preparation

[root@master ~]# Yum - y install zlib devel GCC gcc-c + + (all nodes)

2. Create user

[root@master ~]# useradd -s /sbin/nologin -M mfs (all nodes)

3. Unzip the software package

[root@master ~]# tar zxf mfs-1.6.27-5.tar.gz -C /usr/src
[root@master ~]# cd /usr/src/mfs-1.6.27
[root@master mfs-1.6.27]# ./configure --prefix=/usr/local/mfs \
>--with-default-user=mfs --with-default-group=mfs \
>--disable-mfschunkserver --disable-mfsmount
[root@master mfs-1.6.27]# make && make install

4. Adjust the configuration file

[root@master ~]# cd /usr/local/mfs/etc/mfs
[root@master mfs]# cp mfsmaster.cfg.dist mfsmaster.cfg
[root@master mfs]# cp mfsexports.cfg.dist mfsexports.cfg 
[root@master mfs]# cp mfstopology.cfg.dist mfstopology.cfg
[root@master mfs]# cd /usr/local/mfs/var/mfs/
[root@master mfs]# cp metadata.mfs.empty metadata.mfs

5. Start the master server:

[root@master ~]# ln -s /usr/local/mfs/sbin/* /usr/local/sbin/
[root@master ~]# mfsmaster start
[root@master ~]# netstat -anput | grep mfs
tcp      0      0 0.0.0.0:9419      0.0.0.0:*   LISTEN      9222/mfsmaster      
tcp      0      0 0.0.0.0:9420      0.0.0.0:*   LISTEN      9222/mfsmaster      
tcp      0      0 0.0.0.0:9421      0.0.0.0:*   LISTEN      9222/mfsmaster

You can see that the mfsmaster port is started.
2, metalogger server node:
1. Unzip the package:

[root@metalogger ~]# tar zxf mfs-1.6.27-5.tar.gz -C /usr/src
[root@metalogger ~]# cd /usr/src/mfs-1.6.27
[root@metalogger mfs-1.6.27]# ./configure --prefix=/usr/local/mfs \
--with-default-user=mfs --with-default-group=mfs \
--disable-mfschunkserver --disable-mfsmount
[root@metalogger mfs-1.6.27]# make && make install

2. Adjust the configuration file

[root@metalogger ~]# cd /usr/local/mfs/etc/mfs
[root@metalogger mfs]# cp mfsmetalogger.cfg.dist mfsmetalogger.cfg
[root@metalogger mfs]# vim mfsmetalogger.cfg
# Change to master ip
MASTER_HOST = 192.168.94.139
MASTER_PORT = 9419

3. Start metalogger:

[root@metalogger ~]# ln -s /usr/local/mfs/sbin/* /usr/local/sbin/
[root@metalogger ~]# mfsmetalogger start
[root@metalogger ~]# netstat -anput | grep mfs
tcp    0   0 192.168.140:54654    192.168.94.139:9419   ESTABLISHED 72820/mfsmetalogger

The virtual machine was found to have contacted the master node.
3, Set up chunk server1 and chunk server2 (the two nodes operate in the same way)
1. Unzip the package:

[root@chunk server1 ~]# tar zxf mfs-1.6.27-5.tar.gz -C /usr/src
[root@chunk server1 ~]# cd /usr/src/mfs-1.6.27
[root@chunk server1 mfs-1.6.27]# ./configure --prefix=/usr/local/mfs \
--with-default-user=mfs --with-default-group=mfs \
--disable-mfsmaster --disable-mfsmount
[root@chunk server1 mfs-1.6.27]# make && make install

2. Adjust the configuration file

[root@chunk server1 ~]# cd /usr/local/mfs/etc/mfs
[root@chunk server1 mfs]# cp mfschunkserver.cfg.dist mfschunkserver.cfg
[root@chunk server1 mfs]# cp mfshdd.cfg.dist mfshdd.cfg
[root@chunk server1 mfs]# vim mfschunkserver.cfg
# Change to master ip
MASTER_HOST = 192.168.94.139
MASTER_PORT = 9419(There is no need to modify here. Remove the previous#(OK)
[root@chunk server1 mfs]# vim mfshdd.cfg
# Add at the end
/data
# Here / data is a partition for MFS
[root@chunk server1 mfs]# mkdir /data
[root@chunk server1 mfs]# chown -R mfs:mfs /data

3. Start chunk server:

[root@chunk server1 ~]# ln -s /usr/local/mfs/sbin/* /usr/local/sbin/
[root@chunk server1 ~]# mfschunkserver start
[root@chunk server1 mfs]#  netstat -anput | grep mfs
tcp   0   0 0.0.0.0:9422         0.0.0.0:*           LISTEN      16869/mfschunkserve 
tcp   0   0 192.168.94.141:37532   192.168.94.139:9420   ESTABLISHED 16869/mfschunkserve

4, client node:
1. Install fuse:

[root@client ~]# ln -s /usr/local/mfs/sbin/* /usr/local/sbin/
[root@client ~]# mfschunkserver start
[root@client mfs]#  netstat -anput | grep mfs
tcp   0   0 0.0.0.0:9422         0.0.0.0:*           LISTEN      16869/mfschunkserve 
tcp   0   0 192.168.94.143:37532   192.168.94.139:9420   ESTABLISHED 16869/mfschunkserve

2. Adjust environmental variables:

[root@client ~]# vim /etc/profile
Add at the end
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
[root@client ~]# source /etc/profile

3. Install MFS client:

[root@master ~]# tar zxf mfs-1.6.27-5.tar.gz -C /usr/src
[root@client ~]# cd /usr/src/mfs-1.6.27
[root@client mfs-1.6.27]# ./configure --prefix=/usr/local/mfs --with-default-user=mfs \
--with-default-group=mfs --disable-mfsmaster --disable-mfschunkserver --enable-mfsmount
[root@client mfs-1.6.27]# make && make install

4. Create a mount point and mount:

[root@client ~]# mkdir /opt/mfs
[root@client ~]# modprobe fuse  # Load fuse into kernel
[root@client ~]# ln -s /usr/local/mfs/sbin/* /usr/local/sbin/
[root@client ~]# ln -s /usr/local/mfs/bin/* /usr/local/bin/
Mount mount point to master Host, back mapped to root
[root@client ~]# mfsmount /opt/mfs -H 192.168.94.139
[root@client ~]# df -hT | grep mfs
192.168.94.139:9421       fuse.mfs   83G     0   83G   0% /opt/mfs

5, Start the monitor on the master node:

[root@master ~]# mfscgiserv

Visit 192.168.94.139:9425 on the browser

Note: in version 3.0.84, the host name may not be found in the web page, so you need to add the following in the / etc/hosts file of the master node:

192.168.94.139 master mfsmaster

Posted on Tue, 30 Nov 2021 20:30:46 -0500 by josephman1988