Dockerfile image hierarchy

1, Federated file system (UnionFS)

UnionFS (Federated file system) is a layered, lightweight and high-performance file system. It supports the superposition of file system modifications as a commit operation. At the same time, different directories can be mounted under the same virtual file system. AUFS, OverlayFS and Devicemapper are all UnionFS.
  UnionFS is the foundation of Docker image. Overlay 2 is the default storage driver of Docker at present, and aufs was used before. Images can be integrated through layering. Based on the basic image (the basic image has no parent image), various specific application images can be made.

Features: multiple file systems are loaded at the same time, but from the outside, only one file system can be seen. Joint loading will superimpose all layers of file systems, so that the final file system will contain all underlying files and directories.
When we download, we see layers of Federated file systems. The federated file systems supported by Docker include overlayfs, aufs, Btrfs, VFS, ZFS, and device mapper

overlay Storage engine architecture (different from mirror tiering):
#Overlay FS has only two layers on the linux host. One directory is on the lower layer to store the docker, and the other directory is on the upper layer to store the container information
1,rootfs	base image 
2,lower	Lower layer information (mirror layer, readable)
3,upper	Upper level directory (container information),(writable)
4,worker	Working directory of the run( copy-on-write Copy on write->Prepare container (environment)
5,merged	"View layer"(Container view)

docker Mirror hierarchy:
1,base image: base image 
2,image: It solidifies a standard operating environment and the function of the image itself-Encapsulate a group of functional files and provide them in a unified way (read-only)
3,container: Container layer (read / write)
4,docker-server end
5,Present to docker-client(View)

2, Docker image hierarchy

The Dockerfile is composed of multiple instructions. Each instruction in the Dockerfile corresponds to each layer in the Docker image

  • Images are generated in a hierarchical manner during generation:
  1. Each instruction in Dockerfile will create a new image layer (a temporary container that will no longer exist after execution, and then re create and operate later)
  2. The image layer will be cached and reused (the subsequent image layers will be based on the previous layer, and each layer will have the cache of the next several layers)
  3. When the Dockerfile instruction is modified, the copied file changes, or the specified variables are different when building the image (subsequent operations will inevitably change the previous image layer), the corresponding image layer cache will become invalid (it will be destroyed automatically)
  4. After the image cache of a layer fails, the image cache behind it will fail (if the first layer fails, the second layer will succeed again, which is equivalent to the foundation)
  5. The modification of the container will not affect the image. If you add a file in one layer and delete it in the next layer, the file will still be included in the image

3, Dockerfile overview

1.Dockerfile definition

  • Docker image is a special file system. In addition to providing the program, library, resource, configuration and other files required for container operation, it also contains some configuration parameters prepared for operation (such as anonymous volume, environment variable, user, etc.). The image does not contain any dynamic data, and its content will not be changed after construction.
  • The creation of image is actually to customize the configuration and files added by each layer. If we can write the commands of modification, installation, construction and operation of each layer into a script and use this script to build and customize the image, the problems of transparency and volume of image construction will be solved. This script is Dockerfile.
  • Dockerfile is a text file containing instructions. Each Instruction builds a layer. Therefore, the content of each execution is to describe how the layer should be built. With dockerfile, when we need to customize our own additional requirements, we just need to add or modify instructions on dockerfile and regenerate the image, eliminating the trouble of typing commands.
  • Each specification in Dockerfile corresponds to a command in Linux. Docker program will read the instructions in Dockerfile and generate the specified image.

2. Layering of docker image structure

The image is not a single file, but has multiple layers. The container actually adds a read-write layer to the top of the image. Any file changes made in the running container will be written to this read-write layer. If you delete a container, you delete its top read-write layer, and file changes are lost. Docker uses the storage driver to manage the container layer that mirrors the content of each layer and the read-write layer.

  • Each instruction in Dockerfile creates a new mirror layer
  • The image layer will be cached and reused (the cache after the execution of the next layer will be used by the upper layer)
  • When the Dockerfile instruction is modified, the copied file changes, or the specified variables are different when building the image, the corresponding image layer cache will become invalid
  • If a layer command is modified, the current cache layer will become invalid, and the upper layer referencing the current cache layer will also become invalid. The image needs to be reconstructed. It is often used for version upgrade (function modification and addition)
  • The image layer is immutable. If you add a file in one layer and delete it in the next layer, the file will still be included in the image, but the file is not visible in the Docker container
  • Each layer of Docker image has a unique number. You can view which layers an image consists of through docker history

docker Mirror tiering (based on AUFS (build):
Docker Mirror in bootfs above
 The next layer of each mirror becomes the parent mirror
 Layer 1 image becomes base image(Operating system environment (mirror)
Container layer (readable and writable), at the top( writeout-able)
Below the container floor readonly

PS:
LXC is a container technology in the kernel. In the early days, when docker did not container resources, it relied on LXC in the kernel to complete container virtualization. Now docker has its own docker libcontainer library file, which can container resources, so its dependence on LXC is greatly reduced.

4, Dockerfile operation instruction

instructionsmeaning
FROM [mirror]Specify the image on which the new image is based. The first instruction must be a FROM instruction. Each time you create an image, you need a FROM instruction
MAINTAINER [first name]Describe the maintainer information of the new image
RUN [command]Execute the command on the based image and commit to the new image
CMD ["program to run", "parameter 1", "parameter 2"]The command or script to run when instructing to start the container. Dockerfile can only have one CMD command. If multiple commands are specified, only the last command can be executed
Export [port number]Specify the port to open when the new image is loaded into Docker
ENV [environment variable] [variable value]Setting the value of an environment variable will be used by the following RUN
ADD [source file / directory] [destination file / directory]Copy the source file to the target file. The source file should be located in the same directory as the Dockerfile or a URL. If the source file is a compressed package, it will be decompressed
COPY [source file / directory] [destination file / directory]Copy the file / directory on the local host to the destination. The source file / directory should be in the same directory as the Dockerfile
VOLUME [directory]Create a mount point in the container
USER [USER name / UID]Specifies the user who runs the container
WORKDIR [path]Specifying the working directory for subsequent RUN, CMD and ENTRYPOINT is equivalent to a temporary "CD", otherwise the absolute path needs to be used
ONBUILD [command]Specifies the command to run when the generated image is used as a base image (an optimization)
HEALTHCHECKhealth examination

Build mirror command (you can specify resource limits when building a mirror) example:

docker build -t nginx:test .

-t: tag Label
-f: appoint dockerfile catalogue
.: Refers to the environment (current) directory used when building the image and the context environment used when building the image

Format to follow:

  • The first line must use FROM to specify the name of the image on which the new image is based
  • Then, the MAINTAINER instruction is used to explain the user information maintaining the image
  • Then there are instructions related to the mirror operation, such as the RUN instruction. Each time an instruction is RUN, a new layer is added to the underlying image
  • Finally, use the CMD instruction to specify the command operation to run when starting the container

Docker executes Dockerfile process:

  1. docker runs a container from the underlying image
  2. Execute an instruction and make changes to the container
  3. Perform an operation similar to docker commit to submit a new image layer
  4. docker then runs a new container based on the image just submitted
  5. Execute the next instruction in the dockerfile until all instructions are executed
    Example of build image command: docker build -t image_name. (don't ignore this point)
    Example of using the image command: docker run -d -P image_name
    Finally, use docker ps -a to check the running status of the container. If it is up, you can mirror the test for verification

1. Difference between add and COPY:

COPY can only be used for copying. ADD can also decompress if the copied object is a compressed package. However, COPY saves more resources than ADD.

2. Difference between CMD and ENTRYPOINT:

cmd sets the command and its parameters to be executed by default after the container is started, but cmd can be replaced (overwritten) by the command line parameters following docker run. If docker run does not specify any execution command or there is no entry point in dockerfile, cmd will be used
The specified default execution command is executed. Only the last CMD command will take effect.

ENTRYPOINT specifies the command to be executed when the container is started. You can append the command. The command specified by ENTRYPOINT needs to run with docker
Start the container for matching. Take the content following the docker run instruction as a parameter as the parameter of the running command specified by the ENTRYPOINT instruction. Generally, the linux command specified by the ENTRYPOINT will not be overwritten. The ENTRYPOINT instruction is not required because it increases complexity.

5, Dockerfile various image cases

1.HTTPD

[root@c7-1 ~]#mkdir /opt/apache
[root@c7-1 ~]#cd /opt/apache/
[root@c7-1 /opt/apache]#vim Dockerfile

#centos:7 based basic image
FROM centos:7
#Define the user information of the mirror
MAINTAINER this is apache image <test>
#Install apache software according to the image operation instructions
RUN yum -y update;yum -y install httpd
#Open port 80
EXPOSE 80 
#Copy website home page file
ADD index.html /var/www/index.html

##Foreground startup method I (script execution needs to be prepared)
#Copy the execution script to the mirror
ADD run.sh /run.sh
RUN chmod 755 /run.sh
#Execute script when container starts
CMD ["/run.sh"]

##Foreground startup method 2 (no other script is required)
ENTRYPOINT ["/usr/sbin/apachectl"]
#The foreground starts apache, and CMD passes parameters for ENTRYPOINT
CMD ["-D","FOREGROUND"]

Prepare to execute script (foreground startup method I)

[root@c7-1 /opt/apache]#vim run.sh

#!/bin/bash
rm -rf /run/httpd/*
#Clean up httpd cache
/usr/sbin/apachectl -D FOREGROUND
#Designated as foreground run

The Docker container will only remain running when its process 1 (PID 1) is. If process 1 exits, the Docker container will also exit.

Prepare web page

[root@c7-1 /opt/apache]#echo "this is test web" > index.html
[root@c7-1 /opt/apache]#ls
Dockerfile  index.html  run.sh

Generate image
First, make sure you have centos:7 image. Here, use the first dockerfile

#Speed is related to network and host performance
[root@c7-1 /opt/apache]#docker build -t http:test1 .
......
Successfully built fba8c0058eaf
Successfully tagged http:test1
[root@c7-1 /opt/apache]#docker images	#You can see that the image has been built, but compared with the image downloaded from the official website, it is not optimized
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
http         test1     fba8c0058eaf   52 seconds ago   512MB
centos       7         eeb6ee3f44bd   2 months ago     204MB

Run the container and test

[root@c7-1 /opt/apache]#docker run -itd --name http_test -p 11111:80 http:test1
38a79b3a18411a3c4e79046a5d576a7ce63d47422f22e19353b06817e26e733b
[root@c7-1 /opt/apache]#docker ps -a
CONTAINER ID   IMAGE        COMMAND     CREATED         STATUS         PORTS                                     NAMES
38a79b3a1841   http:test1   "/run.sh"   4 seconds ago   Up 3 seconds   0.0.0.0:11111->80/tcp, :::11111->80/tcp   http_test
[root@c7-1 /opt/apache]#curl http://192.168.10.20:11111
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html><head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
		<title>Apache HTTP Server Test Page powered by CentOS</title>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
......

2 NGINX

Write Dockerfile

FROM centos:7
MAINTAINER GONGBOYI
RUN yum -y update && \
    yum -y install pcre-devel zlib-devel gcc gcc-c++ make && \
    useradd -M -s /sbin/nologin nginx
ADD nginx-1.15.9.tar.gz /usr/local/src
WORKDIR /usr/local/src/nginx-1.15.9
RUN ./configure \
    --prefix=/usr/local/nginx \
    --user=nginx \
    --group=nginx \
    --with-http_stub_status_module && make && make install
ENV PATH /usr/local/nginx/sbin:$PATH
VOLUME ["/usr/local/nginx/html"]
EXPOSE 80
CMD ["nginx","-g","daemon off;"]
#RUN echo "daemon off;" >> /usr/local/nginx/conf/nginx.conf
#CMD nginx

Run test

docker run -itd --name nginx-v1 -P nginx:1.12.0		#The image we write has its own environment. Do not specify bash when running
#The dockerfile starts nginx as a daemon
curl IP:PORT
-------------------------------------------------------
[root@c7-1 ~]#docker build -t nginx:1.15.9 .
......
Successfully built 944d66226a64
Successfully tagged nginx:1.15.9
[root@c7-1 ~]#docker images
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
nginx        1.15.9    944d66226a64   2 minutes ago   574MB
centos       7         eeb6ee3f44bd   2 months ago    204MB
[root@c7-1 ~]#docker run -itd --name nginx-test -P nginx:1.15.9
849a45b6835fa39f59b3e0da4b9d3758f1aef901f8da82eedba91d6acbfc7ce7
[root@c7-1 ~]#docker ps -a
CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                                     NAMES
849a45b6835f   nginx:1.15.9   "nginx -g 'daemon of..."   17 seconds ago   Up 16 seconds   0.0.0.0:49163->80/tcp, :::49163->80/tcp   nginx-test
[root@c7-1 ~]#curl 192.168.10.20:49163
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

During image construction:
① Each layer of mirroring temporarily generates a new layer of mirroring and runs as a temporary container
② Each layer of temporary containers will run based on the cache layer (container) mirrored by the previous layer
③ If the temporary container of the image layer reports an error during operation, exited (non-0 value) will exit, and it will be saved in docker ps -a, and the docker build process will be aborted
④ When modifying based on the same dockerfile, the image image cache corresponding to the modified instruction will become invalid, but the cache of the image layer before the image layer will be retained

3 SSH

FROM centos:7
MAINTAINER this is ssh image <test>
RUN yum -y update;\
yum install -y openssh* net-tools lsof telnet passwd;\
echo '123456' | passwd --stdin root;\
sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config;\
#PAM certification is not used
sed -ri '/^session\s+required\s+pam_loginuid.so/ s/^/#/' /etc/pam.d/sshd;\
#Cancel pam restriction
ssh-keygen -t rsa -A;\
#Generate key authentication file
mkdir -p /root/.ssh;\
chown root.root /root;\
chmod 700 /root/.ssh
EXPOSE 22
CMD ["/usr/sbin/sshd","-D"]
#/usr/sbin/sshd -D is used to start the sshd service in the foreground

------------------------------------
docker build -t sshd:test .
docker images
#Start the container and change the root password
docker run -itd --name sshd -P sshd:test
docker ps -a
ssh localhost -p 49154
echo '654321' | passwd --stdin root		#In container

4 systemctl image

FROM sshd:test
MAINTAINER this is systemctl iamge <test>
ENV container docker
#Delete all files except systemd-tmpfiles-setup.service
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done);\
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /lib/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*;\
rm -f /lib/systemd/system/sockets.target.wants/*udev*;\
rm -f /lib/systemd/system/sockets.target.wants/*initctl*;\
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD [ "/usr/sbin/init" ]

----------------------------------------------------
#Start the container, mount the host directory into the container and initialize it
#--privileged: make the root in the container have real root permission. Otherwise, the root in the container is only an external ordinary user permission.
docker run -d --privileged --name systemctl -P -v /sys/fs/cgroup:/sys/fs/cgroup:ro  systemctl:test
docker exec -it systemctl bash
systemctl status sshd

5 Tomcat image

FROM centos:7
MAINTAINER this is tomcat image <test>
ADD jdk-8u91-linux-x64.tar.gz /usr/local/
WORKDIR /usr/local/
RUN mv jdk1.8.0_91 /usr/local/java
ENV JAVA_HOME /usr/local/java
ENV JRE_HOME ${JAVA_HOME}/jre
ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib
ENV PATH $JAVA_HOME/bin:$PATH
ADD apache-tomcat-8.5.16.tar.gz /usr/local/
WORKDIR /usr/local/
RUN mv apache-tomcat-8.5.16 /usr/local/tomcat
EXPOSE 8080
ENTRYPOINT [ "/usr/local/tomcat/bin/catalina.sh","run" ]

-----------------------------------------------------------
docker build -t tomcat:test .
docker run -d --name tomcat -P tomcat:test
docker ps -a
curl IP:PORT

6, Dockerfile optimization

emsp; when we build our own images, we will find that the images on and off the official website are much smaller than those built by ourselves. Why? The images on and off the official website have been optimized. The common optimization methods are as follows:

  • Instructions that do not need to be output are dropped into / dev/null
  • Reduce RUN build
  • Multi phase build (use the FROM command to generate multiple images, and build the specified image as the basic image environment of other images)
  • Use a lighter linux distribution (Ubuntu, debian, alpine...)
  • Optimize network requests (when using some image sources or URLs on the Internet in dockerfile, use some open source sites with good network, which can save time and reduce failure rate)

1. Throw the instructions that do not need to be output into / dev/null

FROM centos:7
RUN yum install -y gcc pcre pcre-devel devel zlib-devel make &> /dev/null && yum clean all
ADD nginx-1.12.2.tar.gz /mnt
WORKDIR /mnt/nginx-1.12.2
#Close the debug log
RUN sed -i 's/CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' auto/cc/gcc
RUN ./configure --prefix=/usr/local/nginx &> /dev/null
RUN make &> /dev/null
RUN make install &> /dev/null
RUN rm -rf /mnt/nginx-1.12.2
EXPOSE 80
VOLUME ["/usr/local/nginx/html"]
CMD ["/usr/local/nginx/sbin/nginx""-g","daemon off;"]

2. Reduce RUN build

FROM centos:7
ADD nginx-1.12.2.tar.gz /mnt 
WORKDIR /mnt/nginx-1.12.2
RUN yum install -y gcc pcre pcre-devel devel zlib-devel make &> /dev/null && \
 yum clean all && \
 sed -i 's/CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' auto/cc/gcc && \
 ./configure --prefix=/usr/local/nginx &> /dev/null && \
 make &> /dev/null && make install &> /dev/null &&\
 rm -rf /mnt/nginx-1.12.2
EXPOSE 80
VOLUME ["/usr/local/nginx/html"]		##Mount; if the mount point is not specified, the default is / var/lib/docker/volumes / container id/_data
CMD ["/usr/local/nginx/sbin/nginx","-g","daemon off;"]

3. Multi-stage construction

FROM centos:7 as build 
ADD nginx-1.12.2.tar.gz /mnt 
WORKDIR /mnt/nginx-1.12.2
RUN yum install -y gcc pcre pcre-devel devel zlib-devel make &> /dev/null && \
 yum clean all && \
 sed -i 's/CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' auto/cc/gcc && \
 ./configure --prefix=/usr/local/nginx &> /dev/null && \
 make &>/dev/null && \
 make install &>/dev/null && \
 rm -rf /mnt/nginx-1.12.2 
 
FROM centos:7 
EXPOSE 80
VOLUME ["/usr/local/nginx/html"]
COPY --from=build /usr/local/nginx /usr/local/nginx
CMD ["/usr/local/nginx/sbin/nginx","-g","daemon off;"]

---------------------------------------------------------------

docker build -t nginx:test .

4. Use a lighter linux distribution

debian
alpine
apt add

Tags: Operation & Maintenance Docker Container

Posted on Thu, 02 Dec 2021 17:47:54 -0500 by kel