catalogue
1. Deploy docker compose service
3, Maintenance management Harbor
1. Create a project from Harbor Web
There are rolling, blue-green, Canary / grayscale methods for online publishing. At the same time, we should be ready to roll back the scheme at any time. To briefly talk about various online publishing, blue-green means that we basically divide the business flow into two parts, half run the new system and half the old system, and finally complete the online upgrade; Scrolling is to start a new version, then stop an old version, then start a new version, and then stop an old version until the upgrade is completed; Grayscale and Canary probably start a new version of the application first, but do not directly cut the traffic. Instead, testers test the new version online, start the new version of the application, and then divide a small part of the traffic. After it is stable, they are more cautious
1, Harbor
1. Harbor introduction
Harbor is an open source enterprise Docker Registry project of VMware. Its goal is to help users quickly build an enterprise Docker Registry service.
Based on Docker's open source Registry, Harbor provides functions required by enterprise users such as graphical management UI, role-based access control, AD/LDAP integration, and audit logging. At the same time, it supports Chinese.
Each component of Harbor is built in the form of Docker container, and Docker compose is used to deploy it. The Docker compose template for deploying Harbor is located at Harbor / Docker compose.yml
2. Characteristics of Harbor
Role based control: users and warehouses are organized based on projects, and users can have different permissions in projects.
Image based replication strategy: images can be replicated (synchronized) between multiple Harbor instances.
Support LDAP/AD: Harbor can integrate the existing AD/LDAP (a table similar to the database) in the enterprise for authentication and management of existing users.
Image deletion and garbage collection: the image can be deleted or the space occupied by the image can be recycled.
Graphical user interface: users can browse, search the image warehouse and manage the project through the browser.
Audit management: all operations on the image warehouse can be recorded and traced for audit management.
Support for RESTful API: RESTful API provides administrators with more control over Harbor, making it easier to integrate with other management software.
Relationship between harbor and docker registry: Harbor essentially encapsulates docker registry and extends its own business template
3. Harbor composition
In terms of architecture, Harbor mainly includes six components: Proxy, Registry, Core services, Database (Harbor dB), Log collector (Harbor log) and Job services
1.All requests or actions considered will be handed over to the first proxy(Reverse proxy) 2.proxy The request will be forwarded to the back end first Core services，Core services Contains UI,token(Authentication service) webhook(Some service functions of the website) 3.Forward to registry(Image storage). If you need to download images and other permission operations, you need to go through Core services Medium token The authentication service for the token is OK 4.Each download and upload generates an operation record, generates a log, and saves it to database in 5.database Record and save the meta information of the image and the identity information of users and groups. Relevant operations can be allowed only after authentication and authorization
2, Deploy
| Harbor server | 192.168.10.20 | docker-ce,docker-compose,harbor-offline-v1.2.2 |
| client server | 192.168.10.30 | docker-ce |
1. Deploy docker compose service
#wget http://101.34.22.188/docker-compose/docker-compose -P /usr/local/bin [root@c7-1 ~]#curl -L https://github.com/docker/compose/releases/download/1.21.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose [root@c7-1 ~]#chmod +x /usr/local/bin/docker-compose [root@c7-1 ~]#docker-compose -v docker-compose version 1.21.1, build 5a3f1a3
Download or upload Harbor installer
#wget http://101.34.22.188/harbor/harbor-offline-installer-v1.2.2.tgz -P /opt #wget http://harbor.orientsoft.cn/harbor-1.2.2/harbor-offline-installer-v1.2.2.tgz [root@c7-1 ~]#cd /opt [root@c7-1 /opt]#rz -E #Import the harbor installation package harbor-offline-installer-v1.2.2.tgz [root@c7-1 /opt]#tar zxvf harbor-offline-installer-v1.2.2.tgz -C /usr/local/
Modify the configuration file of harbor installation
[root@c7-1 /opt]#vim /usr/local/harbor/harbor.cfg #Line 5, modify the IP address or domain name set as the Harbor server hostname = 192.168.10.20 #Line 59, specify the initial password of the administrator. The default user name / password is admin/Harbor12345 harbor_admin_password = Harbor12345
Two types of parameters in harbor.cfg configuration file
about harbor.cfg There are two types of parameters in the configuration file: required parameters and optional parameters hostname: Used to access the user interface and register Service. It should be the target machine IP Address or fully qualified domain name( FQDN)，For example 192.168.80.10 or hub.kgc.cn. Do not use localhost Or 127.0.0.1 Is the host name. ui_url_protocol: http or https(Default to http)，For access UI And token/Agreement of the notification service. If notarization is enabled, this parameter must be https. max_job_workers: Mirror copy job thread. db_password: be used for db_auth of MySQL database root User's password. customize_crt: This property can be set to on or off, which is on by default. When this property is on, the preparation script creates the private key and root certificate for generation/Validate registry token. Set this property to when the key and root certificate are provided by an external source off. ssl_cert: SSL Path to the certificate, only if the protocol is set to https Apply only when. secretkey_path: Used to encrypt or decrypt remote in replication policy register The key path of the password.
Optional parameters
These parameters are optional for updates, that is, the user can leave them as default values and update them at startup Harbor After Web UI Update on. If you enter Harbor.cfg，It will only start the first time Harbor Takes effect when, and then updates these parameters, Harbor.cfg Will be ignored. If you choose to pass UI To set these parameters, make sure you start Harbor Do this immediately after. Specifically, you must register or Harbor Set the required before creating any new users in auth_mode. When there are users in the system (except the default admin User), auth_mode Cannot be modified. The specific parameters are as follows: Email: Harbor This parameter is required to send password reset e-mail to users and is enabled only when this feature is required. Note that by default SSL Not enabled when connecting SMTP Server needs SSL，But not supported STARTTLS，Then it should be enabled by setting SSL email_ssl = TRUE. harbour_admin_password: The initial password of the administrator, only in Harbour Takes effect the first time it starts. After that, this setting will be ignored and should be UI Set the administrator's password in. Note that the default user name is/The password is admin/Harbor12345. auth_mode: The authentication type used. By default, it is db_auth，That is, the credentials are stored in the database LDAP Authentication, please set it to ldap_auth. self_registration: Enable/Disable user registration. When disabled, new users can only be registered by Admin Created by user, only administrator users can Harbour Create a new user in. Note: when auth_mode Set to ldap_auth The self registration function is always disabled and this flag is ignored. Token_expiration: The expiration time (minutes) of the token created by the token service. The default is 30 minutes. project_creation_restriction: Flag that controls which users have permission to create an item. By default, everyone can create an item. If its value is set to“ adminonly"，So only admin You can create projects. verify_remote_cert: On or off, on by default. This flag determines when Harbor With remote register Verify when communicating with instances SSL/TLS Certificate. Set this property to off Will bypass SSL/TLS Authentication, which is often used when the remote instance has a self signed or untrusted certificate. In addition, by default, Harbor Store the image on the local file system. In a production environment, consider using other storage back ends instead of local file systems, such as S3,Openstack Swif,Ceph And other object storage. But it needs to be updated common/templates/registry/config.yml File.
Start harbor
[root@c7-1 ~]#cd /usr/local/harbor/ [root@c7-1 /usr/local/harbor]#ls common docker-compose.yml harbor.v1.2.2.tar.gz NOTICE docker-compose.clair.yml harbor_1_1_0_template install.sh prepare docker-compose.notary.yml harbor.cfg LICENSE upgrade [root@c7-1 /usr/local/harbor]#./install.sh ...... ✔ ----Harbor has been installed and started successfully.---- Now you should be able to visit the admin portal at http://192.168.10.20. For more details, please visit https://github.com/vmware/harbor . [root@c7-1 /usr/local/harbor]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest ea335eea17ab 2 weeks ago 141MB registry latest b8604a3fe854 2 weeks ago 26.2MB 192.168.10.20:5000/centos test1 eeb6ee3f44bd 2 months ago 204MB vmware/harbor-log v1.2.2 36ef78ae27df 4 years ago 200MB vmware/harbor-jobservice v1.2.2 e2af366cba44 4 years ago 164MB vmware/harbor-ui v1.2.2 39efb472c253 4 years ago 178MB vmware/harbor-adminserver v1.2.2 c75963ec543f 4 years ago 142MB vmware/harbor-db v1.2.2 ee7b9fa37c5d 4 years ago 329MB vmware/nginx-photon 1.11.13 6cc5c831fc7f 4 years ago 144MB vmware/registry 2.6.2-photon 5d9100e4350e 4 years ago 173MB vmware/postgresql 9.6.4-photon c562762cbd12 4 years ago 225MB vmware/clair v2.0.1-photon f04966b4af6c 4 years ago 297MB vmware/harbor-notary-db mariadb-10.1.10 64ed814665c6 4 years ago 324MB vmware/notary-photon signer-0.5.0 b1eda7d10640 4 years ago 156MB vmware/notary-photon server-0.5.0 6e2646682e3c 4 years ago 157MB photon 1.0 e6e4e4a2ba1b 5 years ago 128MB [root@c7-1 /usr/local/harbor]#docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 00d3e2e74c23 vmware/nginx-photon:1.11.13 "nginx -g 'daemon of..." About a minute ago Up About a minute 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:4443->4443/tcp, :::4443->4443/tcp nginx 7778c13a8990 vmware/harbor-jobservice:v1.2.2 "/harbor/harbor_jobs..." About a minute ago Up About a minute harbor-jobservice 479134517ebb vmware/harbor-ui:v1.2.2 "/harbor/harbor_ui" About a minute ago Up About a minute harbor-ui 0c99fc134e9f vmware/registry:2.6.2-photon "/entrypoint.sh serv..." About a minute ago Up About a minute 5000/tcp registry 9b3442b48fce vmware/harbor-db:v1.2.2 "docker-entrypoint.s..." About a minute ago Up About a minute 3306/tcp harbor-db cdd81206d44f vmware/harbor-adminserver:v1.2.2 "/harbor/harbor_admi..." About a minute ago Up About a minute harbor-adminserver 1a2a543cf2b2 vmware/harbor-log:v1.2.2 "/bin/sh -c 'crond &..." About a minute ago Up About a minute 127.0.0.1:1514->514/tcp harbor-log
Note: delete the registry image / container before starting harbor, otherwise a new registry image cannot be generated.
View harbor boot image
[root@c7-1 /usr/local/harbor]#docker-compose ps
Name Command State Ports
-----------------------------------------------------------------------------------------------------------
harbor-adminserver /harbor/harbor_adminserver Up
harbor-db docker-entrypoint.sh mysqld Up 3306/tcp
harbor-jobservice /harbor/harbor_jobservice Up
harbor-log /bin/sh -c crond && rm -f ... Up 127.0.0.1:1514->514/tcp
harbor-ui /harbor/harbor_ui Up
nginx nginx -g daemon off; Up 0.0.0.0:443->443/tcp,:::443->443/tcp,
0.0.0.0:4443->4443/tcp,:::4443->4443/tcp,
0.0.0.0:80->80/tcp,:::80->80/tcp
registry /entrypoint.sh serve /etc/ ... Up 5000/tcp
Create a new project
Browser access: http://192.168.10.20/ log in to the harbor web ui interface. The default administrator user name and password are admin / harbor 12345
After entering the user name and password, you can create a new project. click +Item button Fill in the item name as test_project，Click OK to create a new project and select public Available at this time Docker The command is passed locally.0.0.1 To log in and push images. By default, Registry The server listens on port 80
Log in to Harbor
[root@c7-1 ~]#docker login -uadmin -pHarbor12345 http://127.0.0.1 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
Download the image for testing
[root@c7-1 ~]#docker pull nginx:latest ...... [root@c7-1 ~]#docker images | grep nginx nginx latest f652ca386ed1 19 hours ago 141MB vmware/nginx-photon 1.11.13 6cc5c831fc7f 4 years ago 144MB
Label the image
[root@c7-1 ~]#docker tag nginx:latest 127.0.0.1/test_project/nginx:v1 [root@c7-1 ~]#docker images | grep nginx 127.0.0.1/test_project/nginx v1 f652ca386ed1 19 hours ago 141MB nginx latest f652ca386ed1 19 hours ago 141MB vmware/nginx-photon 1.11.13 6cc5c831fc7f 4 years ago 144MB
Upload image to Harbor
[root@c7-1 ~]#docker push 127.0.0.1/test_project/nginx:v1 The push refers to repository [127.0.0.1/test_project/nginx] 2bed47a66c07: Pushed 82caad489ad7: Pushed d3e1dca44e82: Pushed c9fcd9c6ced8: Pushed 0664b7821b60: Pushed 9321ff862abb: Pushed v1: digest: sha256:4424e31f2c366108433ecca7890ad527b243361577180dfd9a5bb36e828abf47 size: 1570
web view project status
2. Client upload image
The above operations are in Harbor Server local operation. If another client logs in to Harbor，The following error will be reported. The reason for this problem is Docker Registry Interactive default is HTTPS，However, the default setting for building private images is HTTP Service, so the following error occurred when interacting with the private image: client [root@client ~]#docker login -uadmin -pHarbor12345 http://192.168.10.20 WARNING! Using --password via the CLI is insecure. Use --password-stdin. Error response from daemon: Get "https://192.168.10.20/v2/": dial tcp 192.168.10.20:443: connect: connection refused
Configure operations on Docker client
Client (not configured on the server)
[root@client ~]#vim /usr/lib/systemd/system/docker.service
#Line 13 modification
ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry 192.168.10.20 --containerd=/run/containerd/containerd.sock
#Or change it to execstart = / usr / bin / dockerd -- secure registry 192.168.10.20
perhaps
[root@client ~]#cat /etc/docker/daemon.json
{
"insecure-registries": ["192.168.10.20"],
"registry-mirrors": ["https://4iv7219l.mirror.aliyuncs.com"]
}
Restart Docker and log in again
[root@client ~]#systemctl daemon-reload [root@client ~]#systemctl restart docker
Log in to Harbor again
[root@client ~]#docker login -uadmin -pHarbor12345 http://192.168.10.20 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
Download the image for testing
[root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE [root@client ~]#docker pull 192.168.10.20/test_project/nginx:v1 ...... [root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.10.20/test_project/nginx v1 f652ca386ed1 20 hours ago 141MB
Upload images for testing
[root@client ~]#docker pull cirros ...... [root@client ~]#docker tag cirros:latest 192.168.10.20/test_project/cirros:v1 [root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.10.20/test_project/nginx v1 f652ca386ed1 20 hours ago 141MB 192.168.10.20/test_project/cirros v1 f9cae1daf5f6 8 months ago 12.6MB cirros [root@client ~]#docker push 192.168.10.20/test_project/cirros:v1 ......
3, Maintenance management Harbor
1. Create a project from Harbor Web
In the Harbor warehouse, any image must have its own project before being push ed to the regsitry.
Click + item and fill in the item name. If the item level is set to private, it will not be checked. If it is set as a public warehouse, the owner has read permission to the image under this project. The image can be downloaded without executing Docker login on the command line. The image operation is the same as that of Docker Hub.
2. Create Harbor user
Create users and assign permissions
1) stay Web Click system management in the management interface -> user management -> +user 2) Fill in the user name as "test"，Mailbox is "xxx@test.com"，The full name is "test"，Password is "Test12345" (It requires upper and lower case English, numbers and at least 8 characters)，Note as "test"((may be omitted). Attachment: after the user is created successfully, click on the left "..." Button to set the user created above as an administrator role or delete it.
Add project members
Click item -> test_project -> member -> + Member, fill in the user created above test And assign the role as "Developer".
Operate the image with a normal account on the client
Delete all local mirrors [root@client ~]#docker rmi -f `docker images -q` ...... [root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE Exit the current user and use the account created above test Sign in [root@client ~]#docker logout 192.168.10.20 Removing login credentials for 192.168.10.20 [root@client ~]#docker login -utest -pTest12345 http://192.168.10.20 ...... Login Succeeded download harbor Warehouse image root@client ~]#docker pull 192.168.10.20/test_project/cirros:v1 ...... [root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.10.20/test_project/cirros v1 f9cae1daf5f6 8 months ago 12.6MB Upload image to harbor Warehouse root@client ~]#docker tag 192.168.10.20/test_project/cirros:v1 192.168.10.20/test_project/cirros:v2 [root@client ~]#docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.10.20/test_project/cirros v1 f9cae1daf5f6 8 months ago 12.6MB 192.168.10.20/test_project/cirros v2 f9cae1daf5f6 8 months ago 12.6MB [root@client ~]#docker push 192.168.10.20/test_project/cirros:v2 ......
Modify the harbor.cfg configuration file
To change the optional parameters in the Harbor configuration file, stop the existing Harbor instance and update Harbor.cfg; Then run the prepare script to populate the configuration; Finally, recreate and start the instance of Harbour.
use docker-compose Administration Harbor When working with docker-compose.yml Run in the same directory. [root@c7-1 ~]#cd /usr/local/harbor/ [root@c7-1 /usr/local/harbor]#docker-compose down -v Stopping harbor-jobservice ... done Stopping nginx ... done Stopping harbor-ui ... done ...... Removing network harbor_harbor [root@c7-1 /usr/local/harbor]#vim harbor.cfg [root@c7-1 /usr/local/harbor]#./prepare Clearing the configuration file: ./common/config/adminserver/env ...... The configuration files are ready, please use docker-compose to start the service. [root@c7-1 /usr/local/harbor]#docker-compose up -d If there are the following errors, you need to turn on the firewall firewalld Service solution Creating network "harbor_harbor" with the default driver ERROR: Failed to Setup IP tables: Unable to enable SKIP DNAT rule: (iptables failed: iptables --wait -t nat -I DOCKER -i br-b53c314f45e8 -j RETURN: iptables: No chain/target/match by that name. (exit status 1)) > systemctl restart firewalld.service > docker-compose up -d
Remove the Harbor service container, retain the mirrored data / database, and migrate
remove Harbor Service container [root@c7-1 ~]#cd /usr/local/harbor/ [root@c7-1 /usr/local/harbor]#docker-compose down -v ...... Package the image data in the project Persistent data, such as image, database, etc., are stored on the host /data/ Under the directory, the log is on the host computer /var/log/Harbor/ Directory [root@c7-1 /usr/local/harbor]#ls /data/registry/docker/registry/v2/repositories/test_project/ cirros nginx [root@c7-1 /usr/local/harbor]#cd /data/registry/docker/registry/v2/repositories/test_project/ [root@c7-1 /data/registry/docker/registry/v2/repositories/test_project]#tar zcvf test-registry.tar.gz ./* ......
To redeploy, you need to remove all data from the Harbor server
[root@c7-1 ~]#cd /usr/local/harbor [root@c7-1 /usr/local/harbor]#docker-compose down -v Stopping nginx ... done ...... Removing network harbor_harbor [root@c7-1 /usr/local/harbor]#rm -rf /data/database [root@c7-1 /usr/local/harbor]#rm -rf /data/registry
summary
1. Docker harbor image warehouse - > it is a local private image warehouse, which is used to store images frequently used / customized images within the project / enterprise / platform for docker /K8S
2. The docker harbor service consists of several sub function modules, such as UI nginx MySQL registry proxy adminserver JObserver, which is used by docker harbor
The above sub function modules are deployed / displayed in the form of containers in the form of docker compose, and unified arrangement management is carried out
3. How does docker compose organize and manage containers
① Docker compose can define the method of image pulling - > 1) docker hub public warehouse pulling 2) dockerfile custom image building
② Docker compose can specify the mounting of the image, the port exposure of the service in the image, the network, the env environment variable, the restart policy, and so on
Docker compose is a technology used to batch manage images and containers
4. docker -harbor use
① tar decompression
② Upload the docker compose command tool and give execution permission
③ Execute the install.sh script
④ In the docker. Service startup file of docker's systemd management service, add secure - registry 192.168.226.128 (harbor warehouse location) and overload the daemon
systemctl daemon- reload and restart the container docker restart docker so that the local docker can identify and log in to the local private warehouse (docker login)
⑤ Necessary operations for uploading image: docker tag source_ image name:tag 192.168.226.128/project_ name/ image_ name : TAG
Then docker push 192.168.226.128/project_ name/ image_ name : TAG
5. Harbor UI interface is in operation
① You can create users / roles and manage permissions
② Authentication management: database / ldap
③ Mirror operation log management
④ Token token management
6. harbor: high availability (kept), authentication (CA/LDAP), backup (backup to other host)