docker container Harbor warehouse


1, Harbor

1. Harbor introduction

2. Characteristics of Harbor

3. Harbor composition

2, Deploy

1. Deploy docker compose service

2. Client upload image

3, Maintenance management Harbor

1. Create a project from Harbor Web

2. Create Harbor user


There are rolling, blue-green, Canary / grayscale methods for online publishing. At the same time, we should be ready to roll back the scheme at any time. To briefly talk about various online publishing, blue-green means that we basically divide the business flow into two parts, half run the new system and half the old system, and finally complete the online upgrade; Scrolling is to start a new version, then stop an old version, then start a new version, and then stop an old version until the upgrade is completed; Grayscale and Canary probably start a new version of the application first, but do not directly cut the traffic. Instead, testers test the new version online, start the new version of the application, and then divide a small part of the traffic. After it is stable, they are more cautious

1, Harbor

1. Harbor introduction

Harbor is an open source enterprise Docker Registry project of VMware. Its goal is to help users quickly build an enterprise Docker Registry service.
Based on Docker's open source Registry, Harbor provides functions required by enterprise users such as graphical management UI, role-based access control, AD/LDAP integration, and audit logging. At the same time, it supports Chinese.
Each component of Harbor is built in the form of Docker container, and Docker compose is used to deploy it. The Docker compose template for deploying Harbor is located at Harbor / Docker compose.yml

2. Characteristics of Harbor

Role based control: users and warehouses are organized based on projects, and users can have different permissions in projects.
Image based replication strategy: images can be replicated (synchronized) between multiple Harbor instances.
Support LDAP/AD: Harbor can integrate the existing AD/LDAP (a table similar to the database) in the enterprise for authentication and management of existing users.
Image deletion and garbage collection: the image can be deleted or the space occupied by the image can be recycled.
Graphical user interface: users can browse, search the image warehouse and manage the project through the browser.
Audit management: all operations on the image warehouse can be recorded and traced for audit management.
Support for RESTful API: RESTful API provides administrators with more control over Harbor, making it easier to integrate with other management software.
Relationship between harbor and docker registry: Harbor essentially encapsulates docker registry and extends its own business template

3. Harbor composition

In terms of architecture, Harbor mainly includes six components: Proxy, Registry, Core services, Database (Harbor dB), Log collector (Harbor log) and Job services

1.All requests or actions considered will be handed over to the first proxy(Reverse proxy)

2.proxy The request will be forwarded to the back end first Core services,Core services Contains UI,token(Authentication service) webhook(Some service functions of the website)

3.Forward to registry(Image storage). If you need to download images and other permission operations, you need to go through Core services Medium token The authentication service for the token is OK

4.Each download and upload generates an operation record, generates a log, and saves it to database in

5.database Record and save the meta information of the image and the identity information of users and groups. Relevant operations can be allowed only after authentication and authorization

2, Deploy

Harbor server192.168.10.20docker-ce,docker-compose,harbor-offline-v1.2.2
client server192.168.10.30


1. Deploy docker compose service

#wget -P /usr/local/bin
[root@c7-1 ~]#curl -L`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
[root@c7-1 ~]#chmod +x /usr/local/bin/docker-compose 
[root@c7-1 ~]#docker-compose -v
docker-compose version 1.21.1, build 5a3f1a3

Download or upload Harbor installer

#wget -P /opt

[root@c7-1 ~]#cd /opt
[root@c7-1 /opt]#rz -E
#Import the harbor installation package harbor-offline-installer-v1.2.2.tgz
[root@c7-1 /opt]#tar zxvf harbor-offline-installer-v1.2.2.tgz -C /usr/local/

Modify the configuration file of harbor installation

[root@c7-1 /opt]#vim /usr/local/harbor/harbor.cfg 
#Line 5, modify the IP address or domain name set as the Harbor server
hostname =
#Line 59, specify the initial password of the administrator. The default user name / password is admin/Harbor12345
harbor_admin_password = Harbor12345

Two types of parameters in harbor.cfg configuration file

about harbor.cfg There are two types of parameters in the configuration file: required parameters and optional parameters

hostname: Used to access the user interface and register Service. It should be the target machine IP Address or fully qualified domain name( FQDN),For example or Do not use localhost Or Is the host name.

ui_url_protocol: http or https(Default to http),For access UI And token/Agreement of the notification service. If notarization is enabled, this parameter must be https. 

max_job_workers: Mirror copy job thread.

db_password: be used for db_auth of MySQL database root User's password.

customize_crt: This property can be set to on or off, which is on by default. When this property is on, the preparation script creates the private key and root certificate for generation/Validate registry token. Set this property to when the key and root certificate are provided by an external source off. 

ssl_cert: SSL Path to the certificate, only if the protocol is set to https Apply only when.

secretkey_path: Used to encrypt or decrypt remote in replication policy register The key path of the password.

Optional parameters

These parameters are optional for updates, that is, the user can leave them as default values and update them at startup Harbor After Web UI Update on. If you enter Harbor.cfg,It will only start the first time Harbor Takes effect when, and then updates these parameters, Harbor.cfg Will be ignored.
  If you choose to pass UI To set these parameters, make sure you start Harbor Do this immediately after. Specifically, you must register or Harbor Set the required before creating any new users in auth_mode. When there are users in the system (except the default admin User), auth_mode Cannot be modified. The specific parameters are as follows:

Email: Harbor This parameter is required to send password reset e-mail to users and is enabled only when this feature is required. Note that by default SSL Not enabled when connecting SMTP Server needs SSL,But not supported STARTTLS,Then it should be enabled by setting SSL email_ssl = TRUE. 
harbour_admin_password: The initial password of the administrator, only in Harbour Takes effect the first time it starts. After that, this setting will be ignored and should be UI Set the administrator's password in. Note that the default user name is/The password is admin/Harbor12345. 
auth_mode: The authentication type used. By default, it is db_auth,That is, the credentials are stored in the database LDAP Authentication, please set it to ldap_auth. 
self_registration: Enable/Disable user registration. When disabled, new users can only be registered by Admin Created by user, only administrator users can Harbour Create a new user in. Note: when auth_mode Set to ldap_auth The self registration function is always disabled and this flag is ignored.
Token_expiration: The expiration time (minutes) of the token created by the token service. The default is 30 minutes.
project_creation_restriction: Flag that controls which users have permission to create an item. By default, everyone can create an item. If its value is set to“ adminonly",So only admin You can create projects.
verify_remote_cert: On or off, on by default. This flag determines when Harbor With remote register Verify when communicating with instances SSL/TLS Certificate. Set this property to off Will bypass SSL/TLS Authentication, which is often used when the remote instance has a self signed or untrusted certificate.
  In addition, by default, Harbor Store the image on the local file system. In a production environment, consider using other storage back ends instead of local file systems, such as S3,Openstack Swif,Ceph And other object storage. But it needs to be updated common/templates/registry/config.yml File.

Start harbor

[root@c7-1 ~]#cd /usr/local/harbor/
[root@c7-1 /usr/local/harbor]#ls
common                     docker-compose.yml     harbor.v1.2.2.tar.gz  NOTICE
docker-compose.clair.yml   harbor_1_1_0_template            prepare
docker-compose.notary.yml  harbor.cfg             LICENSE               upgrade

[root@c7-1 /usr/local/harbor]#./
✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at 
For more details, please visit .

[root@c7-1 /usr/local/harbor]#docker images
REPOSITORY                  TAG               IMAGE ID       CREATED        SIZE
nginx                       latest            ea335eea17ab   2 weeks ago    141MB
registry                    latest            b8604a3fe854   2 weeks ago    26.2MB   test1             eeb6ee3f44bd   2 months ago   204MB
vmware/harbor-log           v1.2.2            36ef78ae27df   4 years ago    200MB
vmware/harbor-jobservice    v1.2.2            e2af366cba44   4 years ago    164MB
vmware/harbor-ui            v1.2.2            39efb472c253   4 years ago    178MB
vmware/harbor-adminserver   v1.2.2            c75963ec543f   4 years ago    142MB
vmware/harbor-db            v1.2.2            ee7b9fa37c5d   4 years ago    329MB
vmware/nginx-photon         1.11.13           6cc5c831fc7f   4 years ago    144MB
vmware/registry             2.6.2-photon      5d9100e4350e   4 years ago    173MB
vmware/postgresql           9.6.4-photon      c562762cbd12   4 years ago    225MB
vmware/clair                v2.0.1-photon     f04966b4af6c   4 years ago    297MB
vmware/harbor-notary-db     mariadb-10.1.10   64ed814665c6   4 years ago    324MB
vmware/notary-photon        signer-0.5.0      b1eda7d10640   4 years ago    156MB
vmware/notary-photon        server-0.5.0      6e2646682e3c   4 years ago    157MB
photon                      1.0               e6e4e4a2ba1b   5 years ago    128MB

[root@c7-1 /usr/local/harbor]#docker ps -a
CONTAINER ID   IMAGE                              COMMAND                  CREATED              STATUS              PORTS                                                                                                                 NAMES
00d3e2e74c23   vmware/nginx-photon:1.11.13        "nginx -g 'daemon of..."   About a minute ago   Up About a minute>80/tcp, :::80->80/tcp,>443/tcp, :::443->443/tcp,>4443/tcp, :::4443->4443/tcp   nginx
7778c13a8990   vmware/harbor-jobservice:v1.2.2    "/harbor/harbor_jobs..."   About a minute ago   Up About a minute                                                                                                                         harbor-jobservice
479134517ebb   vmware/harbor-ui:v1.2.2            "/harbor/harbor_ui"      About a minute ago   Up About a minute                                                                                                                         harbor-ui
0c99fc134e9f   vmware/registry:2.6.2-photon       "/ serv..."   About a minute ago   Up About a minute   5000/tcp                                                                                                              registry
9b3442b48fce   vmware/harbor-db:v1.2.2            "docker-entrypoint.s..."   About a minute ago   Up About a minute   3306/tcp                                                                                                              harbor-db
cdd81206d44f   vmware/harbor-adminserver:v1.2.2   "/harbor/harbor_admi..."   About a minute ago   Up About a minute                                                                                                                         harbor-adminserver
1a2a543cf2b2   vmware/harbor-log:v1.2.2           "/bin/sh -c 'crond &..."   About a minute ago   Up About a minute>514/tcp                                                                                               harbor-log

  Note: delete the registry image / container before starting harbor, otherwise a new registry image cannot be generated.

  View harbor boot image

[root@c7-1 /usr/local/harbor]#docker-compose ps
       Name                     Command               State                       Ports                    
harbor-adminserver   /harbor/harbor_adminserver       Up                                                   
harbor-db   mysqld      Up      3306/tcp                                     
harbor-jobservice    /harbor/harbor_jobservice        Up                                                   
harbor-log           /bin/sh -c crond && rm -f  ...   Up>514/tcp                      
harbor-ui            /harbor/harbor_ui                Up                                                   
nginx                nginx -g daemon off;             Up>443/tcp,:::443->443/tcp,        
registry             / serve /etc/ ...   Up      5000/tcp                   

Create a new project

Browser access: log in to the harbor web ui interface. The default administrator user name and password are admin / harbor 12345

After entering the user name and password, you can create a new project. click +Item button

Fill in the item name as test_project,Click OK to create a new project and select public

Available at this time Docker The command is passed locally.0.0.1 To log in and push images. By default, Registry The server listens on port 80

  Log in to Harbor

[root@c7-1 ~]#docker login -uadmin -pHarbor12345
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See

Login Succeeded

Download the image for testing

[root@c7-1 ~]#docker pull nginx:latest
[root@c7-1 ~]#docker images | grep nginx
nginx                       latest            f652ca386ed1   19 hours ago   141MB
vmware/nginx-photon         1.11.13           6cc5c831fc7f   4 years ago    144MB

Label the image

[root@c7-1 ~]#docker tag nginx:latest
[root@c7-1 ~]#docker images | grep nginx   v1                f652ca386ed1   19 hours ago   141MB
nginx                          latest            f652ca386ed1   19 hours ago   141MB
vmware/nginx-photon            1.11.13           6cc5c831fc7f   4 years ago    144MB

Upload image to Harbor

[root@c7-1 ~]#docker push
The push refers to repository []
2bed47a66c07: Pushed 
82caad489ad7: Pushed 
d3e1dca44e82: Pushed 
c9fcd9c6ced8: Pushed 
0664b7821b60: Pushed 
9321ff862abb: Pushed 
v1: digest: sha256:4424e31f2c366108433ecca7890ad527b243361577180dfd9a5bb36e828abf47 size: 1570

web view project status

2. Client upload image

The above operations are in Harbor Server local operation. If another client logs in to Harbor,The following error will be reported. The reason for this problem is Docker Registry Interactive default is HTTPS,However, the default setting for building private images is HTTP Service, so the following error occurred when interacting with the private image:

[root@client ~]#docker login -uadmin -pHarbor12345
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get "": dial tcp connect: connection refused

Configure operations on Docker client

Client (not configured on the server)
[root@client ~]#vim /usr/lib/systemd/system/docker.service
#Line 13 modification
ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry --containerd=/run/containerd/containerd.sock
#Or change it to execstart = / usr / bin / dockerd -- secure registry


[root@client ~]#cat /etc/docker/daemon.json 
"insecure-registries": [""],
"registry-mirrors": [""]

Restart Docker and log in again

[root@client ~]#systemctl daemon-reload
[root@client ~]#systemctl restart docker

  Log in to Harbor again

[root@client ~]#docker login -uadmin -pHarbor12345
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See

Login Succeeded

Download the image for testing

[root@client ~]#docker images
[root@client ~]#docker pull
[root@client ~]#docker images
REPOSITORY                         TAG       IMAGE ID       CREATED        SIZE   v1        f652ca386ed1   20 hours ago   141MB

  Upload images for testing

[root@client ~]#docker pull cirros
[root@client ~]#docker tag cirros:latest
[root@client ~]#docker images
REPOSITORY                          TAG       IMAGE ID       CREATED        SIZE    v1        f652ca386ed1   20 hours ago   141MB   v1        f9cae1daf5f6   8 months ago   12.6MB
[root@client ~]#docker push

3, Maintenance management Harbor

1. Create a project from Harbor Web

In the Harbor warehouse, any image must have its own project before being push ed to the regsitry.
Click + item and fill in the item name. If the item level is set to private, it will not be checked. If it is set as a public warehouse, the owner has read permission to the image under this project. The image can be downloaded without executing Docker login on the command line. The image operation is the same as that of Docker Hub.

2. Create Harbor user

Create users and assign permissions

1) stay Web Click system management in the management interface -> user management  -> +user
2) Fill in the user name as "test",Mailbox is "",The full name is "test",Password is "Test12345" (It requires upper and lower case English, numbers and at least 8 characters),Note as "test"((may be omitted).
Attachment: after the user is created successfully, click on the left "..." Button to set the user created above as an administrator role or delete it.

  Add project members

Click item -> test_project -> member -> + Member, fill in the user created above test And assign the role as "Developer". 

  Operate the image with a normal account on the client

Delete all local mirrors
[root@client ~]#docker rmi -f `docker images -q`
[root@client ~]#docker images

Exit the current user and use the account created above test Sign in
[root@client ~]#docker logout
Removing login credentials for
[root@client ~]#docker login -utest -pTest12345
Login Succeeded

download harbor Warehouse image
root@client ~]#docker pull
[root@client ~]#docker images
REPOSITORY                          TAG       IMAGE ID       CREATED        SIZE   v1        f9cae1daf5f6   8 months ago   12.6MB

Upload image to harbor Warehouse
root@client ~]#docker tag
[root@client ~]#docker images
REPOSITORY                          TAG       IMAGE ID       CREATED        SIZE   v1        f9cae1daf5f6   8 months ago   12.6MB   v2        f9cae1daf5f6   8 months ago   12.6MB
[root@client ~]#docker push

  Modify the harbor.cfg configuration file

To change the optional parameters in the Harbor configuration file, stop the existing Harbor instance and update Harbor.cfg; Then run the prepare script to populate the configuration; Finally, recreate and start the instance of Harbour.

use docker-compose Administration Harbor When working with docker-compose.yml Run in the same directory.
[root@c7-1 ~]#cd /usr/local/harbor/
[root@c7-1 /usr/local/harbor]#docker-compose down -v
Stopping harbor-jobservice  ... done
Stopping nginx              ... done
Stopping harbor-ui          ... done
Removing network harbor_harbor
[root@c7-1 /usr/local/harbor]#vim harbor.cfg
[root@c7-1 /usr/local/harbor]#./prepare 
Clearing the configuration file: ./common/config/adminserver/env
The configuration files are ready, please use docker-compose to start the service.

[root@c7-1 /usr/local/harbor]#docker-compose up -d

If there are the following errors, you need to turn on the firewall firewalld Service solution
Creating network "harbor_harbor" with the default driver
ERROR: Failed to Setup IP tables: Unable to enable SKIP DNAT rule:  (iptables failed: iptables --wait -t nat -I DOCKER -i br-b53c314f45e8 -j RETURN: iptables: No chain/target/match by that name.
 (exit status 1))
> systemctl restart firewalld.service
> docker-compose up -d

Remove the Harbor service container, retain the mirrored data / database, and migrate

remove Harbor Service container
[root@c7-1 ~]#cd /usr/local/harbor/
[root@c7-1 /usr/local/harbor]#docker-compose down -v

Package the image data in the project
 Persistent data, such as image, database, etc., are stored on the host /data/ Under the directory, the log is on the host computer /var/log/Harbor/ Directory
[root@c7-1 /usr/local/harbor]#ls /data/registry/docker/registry/v2/repositories/test_project/
cirros  nginx
[root@c7-1 /usr/local/harbor]#cd /data/registry/docker/registry/v2/repositories/test_project/
[root@c7-1 /data/registry/docker/registry/v2/repositories/test_project]#tar zcvf test-registry.tar.gz ./*

To redeploy, you need to remove all data from the Harbor server

[root@c7-1 ~]#cd /usr/local/harbor
[root@c7-1 /usr/local/harbor]#docker-compose down -v
Stopping nginx              ... done
Removing network harbor_harbor
[root@c7-1 /usr/local/harbor]#rm -rf /data/database
[root@c7-1 /usr/local/harbor]#rm -rf /data/registry


1. Docker harbor image warehouse - > it is a local private image warehouse, which is used to store images frequently used / customized images within the project / enterprise / platform for docker /K8S

2. The docker harbor service consists of several sub function modules, such as UI nginx MySQL registry proxy adminserver JObserver, which is used by docker harbor
The above sub function modules are deployed / displayed in the form of containers in the form of docker compose, and unified arrangement management is carried out

3. How does docker compose organize and manage containers
① Docker compose can define the method of image pulling - > 1) docker hub public warehouse pulling 2) dockerfile custom image building
② Docker compose can specify the mounting of the image, the port exposure of the service in the image, the network, the env environment variable, the restart policy, and so on

Docker compose is a technology used to batch manage images and containers

4. docker -harbor use
① tar decompression
② Upload the docker compose command tool and give execution permission
③ Execute the script
④ In the docker. Service startup file of docker's systemd management service, add secure - registry (harbor warehouse location) and overload the daemon
systemctl daemon- reload and restart the container docker restart docker so that the local docker can identify and log in to the local private warehouse (docker login)
⑤ Necessary operations for uploading image: docker tag source_ image name:tag name/ image_ name : TAG
Then docker push name/ image_ name : TAG

5. Harbor UI interface is in operation
① You can create users / roles and manage permissions
② Authentication management: database / ldap
③ Mirror operation log management
④ Token token management

6. harbor: high availability (kept), authentication (CA/LDAP), backup (backup to other host)

Tags: Operation & Maintenance Docker Container

Posted on Sat, 04 Dec 2021 19:14:08 -0500 by R0bb0b