Use of docker storage volume

Docker storage volume of docker

COW mechanism

Docker image is composed of multiple read-only layers. When the container is started, docker will load the read-only image layer and add a read-write layer at the top of the image stack.

If the running container modifies an existing file, the file will be copied from the read-only layer below the read-write layer to the read-write layer. The read-only version of the file still exists, but it has been hidden by the copy of the file in the read-write layer. This is the "copy on write (COW)" mechanism.

In this way, when we access a file, modify and delete, the efficiency will be very low, because there are many layers of images.

To bypass this limitation, we can use the mechanism of storage volume.

What is a storage volume

Storage volume is to directly bind a directory existing in the local file system of the host to a directory on the file system inside the container. This means that when we write data in this directory in the container, the container will write its contents directly to the directory on the host that has established a binding relationship with this container.

The directory on the host that forms a binding relationship with the container is called a storage volume.

Benefits of using storage volumes

If all the valid data of the processes running in the container are saved in the storage volume and separated from the container's own file system, the benefit is that when the container is closed or even deleted, we don't have to worry about data loss as long as the storage directory bound to the container on the host computer is not deleted. Therefore, data persistence can be realized, which is separated from the life cycle of the container.

By managing containers in this way, we can break away from the restrictions of hosts, run containers on any host with docker deployed, and its data can be placed on a shared storage file system, such as nfs.

By default, the storage volume of Docker uses the local file system directory on the host, that is, the host has its own hard disk, which is not shared with other Docker hosts, and the storage volume used by the container started on this host is associated with a directory on the hard disk of this host.

This means that the container stops running on this host or is deleted and rebuilt. As long as it is associated with this directory on the hard disk, its data still exists. But if you start a new container on another host, the data is gone. If we manually mount the container data to an nfs server when creating the container, this problem is no longer a problem.

Why use storage volumes

If the container is closed and restarted, its data will not be affected, but if the Docker container is deleted, all its changes will be lost.
Therefore, Docker has the following problems:

It is stored in the jointly mounted file system and is not easy to access by the host
Inconvenient data sharing between containers
Delete the container and its data will be lost
To solve these problems, the solution is to use storage volumes.

Storage volume management mode

The Data Volume is automatically created when the container is initialized, and the data in the volume provided by base image completes replication during this period.

The original intention of Volume is to realize data persistence independent of the life cycle of the container. Therefore, when deleting the container, it will neither delete the Volume nor garbage collect the unreferenced Volume.

Storage volume provides Docker with a container independent data management mechanism. We can imagine the image as a static file, such as "program", and compare the volume to dynamic content, such as "data". So mirrors can be reused and volumes can be shared.

Volume realizes the separation of "program (image)" and "data (volume)", as well as the separation of "program (image)" and "host making image". When making image, users do not need to consider the environment of the host where the container running image is located.

Classification of storage volumes

Docker has two types of volumes. Each type has a mount point in the container, but its location on the host is different:

Bind mount volume
Points to a volume at a user specified location on the host file system
Docker managed volumes
The Docker daemon creates managed volumes in a portion of the host file system that is owned by Docker

The docker managed volume of the current version of docker exists in / var / lib / docker / overlay 2

Container data management

When using Docker, users often need to be able to view the data generated by applications in the container, or need to back up the data in the container, or even share the data among multiple containers, which inevitably involves the data management operation of the container.

There are two main ways to manage data in containers:

Data Volumes place data on a host or a directory of shared storage
Data Volumes Containers place data in containers
Container Volume usage syntax:

//Docker-managed volume
docker run -it --name CONTAINER_NAME -v VOLUMEDIR IMAGE_NAME
// Using data volumes in containers
[root@localhost ~]# docker run -it --rm -v /data busybox

[root@localhost ~]# docker inspect b5b07426b0e4 / / use this command to view the directory shared by the container and the host machine 
Omitted here N that 's ok
"Mounts": [
            {
                "Type": "volume",
                "Name": "2bedfe4fa7cae1f5adfd26d71b74e7e75614a8654501ccf3d2fd6e18d1400329",
                "Source": "/var/lib/docker/volumes/2bedfe4fa7cae1f5adfd26d71b74e7e75614a8654501ccf3d2fd6e18d1400329/_data",  //This directory is the directory shared by the host and the container

// Create a file in the container for testing
/ # cd data/  
/data # echo 'xxhh' > file
/data # cat file 
xxhh

// View to host
[root@localhost _data]# ls
file
[root@localhost _data]# cat file 
xxhh

// When the container is deleted, the created data directory disappears
[root@localhost _data]# ls
[root@localhost _data]#
//Bind mount volume

// format
docker run -it --name CONTAINER_NAME -v HOSTDIR:VOLUMEDIR IMAGE_NAME

// Mount a local host file as a data volume
[root@localhost ~]# docker run -it --rm -v /opt/data:/data busybox / / the / opt/data directory of the host machine is bound to the / data directory of the container
/ # cd data/
/data # touch 123

[root@localhost data]# pwd
/opt/data
[root@localhost data]# ls
123

// The deleted container directory still exists. Because -- rm is added, exit the container and the container will be deleted automatically
[root@localhost data]# ls
123

Leverage NFS as a shared storage volume

// First install the nfs service
[root@localhost ~]# yum -y install nfs-utils
[root@client ~]# yum -y install nfs-utils
// Authorize an object for its use
[root@client ~]#mkdir /nfs / / because there is no such directory, it needs to be created
[root@client ~]# cat /etc/exports
/nfs 192.168.71.138(rw)

[root@client ~]#systemctl enable --now nfs-server.service 
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-server.service → /usr/lib/systemd/system/nfs-server.service.
// View the shared directory of all outputs on the specified NFS server
[root@localhost ~]# showmount -e 192.168.71.140
Export list for 192.168.71.140
/nfs 192.168.71.138
// First associate and then mount

// relation
[root@localhost ~]# docker run -it --rm -v /opt/data:/data busybox
/ # cd data/
/data # ls
/data # mkdir abc

[root@localhost ~]# cd /opt/data/
[root@localhost data]# 
[root@localhost data]# ls
abc

// mount 
[root@localhost ~]# mount -t nfs 192.168.71.140:/nfs /opt/data/
[root@localhost data]#
[root@localhost data]# df -h
Filesystem            Size  Used Avail Use% Mounted on
devtmpfs              876M     0  876M   0% /dev
tmpfs                 895M     0  895M   0% /dev/shm
tmpfs                 895M  8.9M  886M   1% /run
tmpfs                 895M     0  895M   0% /sys/fs/cgroup
/dev/mapper/cs-root    66G   13G   53G  20% /
/dev/mapper/cs-home    32G  260M   32G   1% /home
/dev/sr0              9.0G  9.0G     0 100% /mnt/cdrom
/dev/sda1            1014M  195M  820M  20% /boot
tmpfs                 179M     0  179M   0% /run/user/0
overlay                66G   13G   53G  20% /var/lib/docker/overlay2/1d71253990775778a734de93ced30222d7b24f53c0fa6a59f5498c71ae2b1a68/merged
overlay                66G   13G   53G  20% /var/lib/docker/overlay2/de07492b31d8a0414669e88dff3744c1277b261fae80062c05a01de9d110a386/merged
192.168.182.151:/nfs   66G  2.2G   63G   4% /opt/data

// Then create a directory in the container to see the effect
/data # mkdir 123
/data # ls
123  abc
[root@localhost data]# ls / / not found
[root@localhost data]# 
// When the mount is cancelled, the directory will appear
[root@localhost ~]# umount /opt/data
[root@localhost ~]# cd /opt/data/
[root@localhost data]# ls / / the directory is hidden instead of disappearing
123  abc
// Therefore, you cannot associate first and then mount
// Mount first and then associate

// mount 
[root@localhost ~]# mount -t nfs 192.168.182.151:/nfs /opt/data / / mount the client's nfs directory to the local / opt/data directory
[root@localhost ~]# df -h
192.168.71.140:/nfs   66G  2.2G   63G   4% /opt/data

// relation
[root@localhost ~]# docker run -it --rm -v /opt/data/:/data busybox
/ # cd data/
/data # touch 123 / / display permission is denied
touch: 123: Permission denied

[root@localhost data]# touch 123
touch: cannot touch '123': Permission denied

// The reason is that the / nfs directory does not have permissions. The / nfs directory uses the nobody user by default, and the centso7 or redhat7 directory uses the nfsnobody user by default
[root@client ~]# setfacl -m u:nobody:rwx /nfs/
[root@client ~]# getfacl /nfs/
getfacl: Removing leading '/' from absolute path names
# file: nfs/
# owner: root
# group: root
user::rwx
user:nobody:rwx
group::r-x
mask::rwx
other::r-x

/data # touch 123
[root@localhost data]# ls
123
// If the server is restarted, can the data be synchronized

[root@localhost ~]# mount -t nfs 192.168.71.140:/nfs /opt/data/

[root@localhost data]# docker run -d --name test -v /opt/data/:/data httpd
ba29a7dcc0931db13b80ea75ca4da7319d12543bb6178ce342d6775026913fc6

[root@localhost ~]# docker start test 
test

[root@localhost ~]# docker exec -it test /bin/bash
root@ba29a7dcc093:/data# ls / / no content
root@ba29a7dcc093:/data#

[root@localhost data]# mount -t nfs 192.168.182.151:/nfs /opt/data/

You need to stop the container and then mount it again
[root@localhost ~]# docker stop test 
test

[root@localhost ~]# docker start test 
test
[root@localhost ~]# docker exec -it test /bin/bash

root@ba29a7dcc093:~# cd /data/
root@ba29a7dcc093:/data# ls
123

// To prevent this from happening again, you need to mount permanently
[root@localhost data]# vim /etc/fstab

[root@localhost data]# tail -1 /etc/fstab 
192.168.71.140:/nfs /opt/data nfs defaults,_netdev 0 0
[root@localhost data]# mount -a
// The default permission for Docker to mount data volumes is read-write (rw), and users can also specify it as read-only through (ro):

[root@localhost ~]# docker run -it --rm -v /opt/data/:/data:ro busybox
/data # ls
123

/data # rm -f 123 
rm: can't remove '123': Read-only file system

Yes:ro After that, the data of the data volume mounted in the container cannot be modified.

//Mount a local host file as a data volume
-The v option can also mount a single file from the host into a container as a data volume:

[root@localhost ~]# docker run -it --rm -v /opt/data/123:/data/123 busybox

/ # cd /data/
/data # ls
123
/data # cat 123 
/data # echo 'hello world' > 123 
/data # cat 123 
hello world

[root@localhost data]# cat 123 
hello world

In this way, you can record the command history entered in the container

Tags: Operation & Maintenance Docker Container

Posted on Mon, 06 Dec 2021 22:45:55 -0500 by The Bat