Using nexus3 to configure the docker private warehouse

1. Configuration starting

1. Create blob storage

After logging in, create a space to store the image.

Define a name, and the content below will be completed automatically.

Then save.

Note: in actual production, it is recommended that the server store 500G or more.

2. Create a docker warehouse of hosted type

The Hosted type warehouse is used as our private warehouse instead of harbor.

Click the following steps:

Then we can see the variety of the supported species.

Here we see three types of docker:

  • hosted: local storage, that is, it provides local private server function as the official warehouse of docker.
  • Proxy: provides the type of proxy for other warehouses, such as the docker central warehouse.
  • Group: group type. Its real function is to combine multiple warehouses into one address.

First, create a private warehouse of hosted type.

Click Repositories – Create repository – docker(hosted) under Repository:

Name: define a name docker local

Online: check. This switch can be used to set whether the Docker repo is online or offline.

Repository Connectors
  • There are two types of port s: HTTP and HTTPS.

  • What's the use? The explanation is clear:

  • The connector allows the docker client to directly connect to the docker warehouse and implement some request operations, such as docker pull, docker push, API query, etc. But this connector does not have to be configured, especially we will use the group type docker warehouse to aggregate it later.

We check HTTP here and set the port to 8083.

Allow anonymous docker pull 

Uncheck. In this way, anonymous access is not allowed. Before executing docker pull or docker push, you need to log in: docker login

Docker Registry API Support

Docker registry uses API v2 by default, but for compatibility, we can check enable API v1.

Storage

Blob store: let's drop-down and select the created dedicated blob: docker hub.

Hosted

In the development environment, we run duplicate publishing, so we choose allow deploy for the Delpoyment policy.

The overall configuration screenshot is as follows:

3. Create a proxy type docker warehouse

proxy type warehouse can help us access the network that cannot be reached directly, such as another private warehouse, or foreign public warehouse, such as the official dockerhub image library.

Create a warehouse of proxy type

Name: proxy-docker-hub

Repository Connectors: not set.

Proxy

Remote storage: the proxy of the docker hub, fill in here: https://registry-1.docker.io This is an official default link

Docker Index: Use Docker Hub

Storage: idocker-hub

The overall configuration screenshot is as follows:





4. Create a group type docker warehouse

The group type docker warehouse is an aggregation type warehouse. It can aggregate the three warehouses we created earlier into a URL to provide external services. It can shield the differences in the back end and realize the functions similar to transparent proxy.

name: docker-group

Repository Connectors: enable an http connector listening on port 8082;

Storage: select a dedicated blob to store idocker hub.

group: add the three optional warehouses on the left to the members on the right.

The overall configuration screenshot is as follows:



Final effect

Up to now, the deployment of nexus in docker has been completed, but it can not be used well. Because the group warehouse cannot push the image, because you have to push the image you made to the warehouse through the port of the local warehouse, which is very inconvenient!

There is a solution: judge whether to push or pull the image through Nginx, and then proxy to different ports

5. nginx proxy access to the warehouse

In the deployment of Nginx, you need to sign your own SSL certificate, because you don't want to bring a port when docker pull! Two domain names are needed here, one is used to show the nexus front desk, and the other is used as a docker warehouse, for example:

  • nexus front desk: repo.ald.com
  • docker warehouse: idocker.io

1. Install nginx

yum -y install nginx

2. Generate certificate

Here we recommend a one click generation tool. You can try to use: https://github.com/Fishdrowned/ssl , please refer to the author's instructions.

Ps: if you plan to do the Internet warehouse service, you can also apply for a free SSL certificate. I use the internal oa domain name, so I can only use the self signed certificate.

Create the certificate as follows:

#Switch to application directory directly
# cd /etc/nginx/conf.d/
 
#Download tools
# git clone https://github.com/Fishdrowned/ssl.git
Cloning into 'ssl'...
remote: Enumerating objects: 106, done.
remote: Total 106 (delta 0), reused 0 (delta 0), pack-reused 106
Receiving objects: 100% (106/106), 171.53 KiB | 286.00 KiB/s, done.
Resolving deltas: 100% (48/48), done.
 
#Generate certificate
# cd ssl
# ./gen.cert.sh idocker.io

Removing dir out
Creating output structure
Done
Generating a 2048 bit RSA private key
......+++
......................................................................................................................+++
writing new private key to 'out/root.key.pem'
-----
Generating RSA private key, 2048 bit long modulus
...............................................................................+++
.................................+++
e is 65537 (0x10001)
Using configuration from ./ca.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'CN'
stateOrProvinceName   :ASN.1 12:'Guangdong'
localityName          :ASN.1 12:'Guangzhou'
organizationName      :ASN.1 12:'Fishdrowned'
organizationalUnitName:ASN.1 12:'idocker.io'
commonName            :ASN.1 12:'*.idocker.io'
Certificate is to be certified until Jun 12 04:29:18 2022 GMT (730 days)

Write out database with 1 new entries
Data Base Updated

Certificates are located in:
lrwxrwxrwx 1 root root 37 6 December 12:29 /etc/nginx/conf.d/ssl/out/idocker.io/idocker.io.bundle.crt -> ./20200612-1229/idocker.io.bundle.crt
lrwxrwxrwx 1 root root 30 6 December 12:29 /etc/nginx/conf.d/ssl/out/idocker.io/idocker.io.crt -> ./20200612-1229/idocker.io.crt
lrwxrwxrwx 1 root root 15 6 December 12:29 /etc/nginx/conf.d/ssl/out/idocker.io/idocker.io.key.pem -> ../cert.key.pem
lrwxrwxrwx 1 root root 11 6 December 12:29 /etc/nginx/conf.d/ssl/out/idocker.io/root.crt -> ../root.crt

3. Configure nginx

# ip address can be changed to intranet ip
upstream nexus_docker_get {
    server 192.168.75.11:8082;
}
 
upstream nexus_docker_put {
    server 192.168.75.11:8083;
}
server {
    listen 80;
    listen 443 ssl;
    server_name idocker.io;
    access_log /var/log/nginx/idocker.io.log;
    # certificate
    ssl_certificate /etc/nginx/conf.d/ssl/out/idocker.io/idocker.io.crt; # The certificate path is determined according to the above generated
    ssl_certificate_key /etc/nginx/conf.d/ssl/out/idocker.io/idocker.io.key.pem;
    ssl_protocols TLSv1.1 TLSv1.2;
    ssl_ciphers '!aNULL:kECDH+AESGCM:ECDH+AESGCM:RSA+AESGCM:kECDH+AES:ECDH+AES:RSA+AES:';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    # disable any limits to avoid HTTP 413 for large image uploads
    client_max_body_size 0;
    # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
    chunked_transfer_encoding on;
    # Set default use push agent
    set $upstream "nexus_docker_put";
    # When the request is GET, i.e. pulling the image, the pull agent is changed here, so that the pull and push ports are unified
    if ( $request_method ~* 'GET') {
        set $upstream "nexus_docker_get";
    }
    # Only the local warehouse supports search, so forward the search request to the local warehouse, otherwise 500 errors will be reported
    if ($request_uri ~ '/search') {
        set $upstream "nexus_docker_put"; 
    }
    index index.html index.htm index.php;
    location / {
        proxy_pass http://$upstream;
        proxy_set_header Host $host;
        proxy_connect_timeout 3600;
        proxy_send_timeout 3600;
        proxy_read_timeout 3600;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_buffering off;
        proxy_request_buffering off;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto http;
    }
}

If nginx-t check is OK, you can start nginx.

4. Client configuration

After the deployment is completed, we can find a test machine to test. However, since we just defined the domain name for internal use, we need to write the hosts resolution on the test machine and copy the certificate. Otherwise, an untrusted error will be reported.

In the one key generation self signing tool described above, a root certificate named / etc / nginx / conf.d/ssl/out will be generated/ idocker.io/root . CRT, we will upload this file to / etc / docker / certs. D of the client server/ idocker.io Just a directory (note that the directory needs to be created, and the last folder name is consistent with the warehouse domain name: idocker.io ).

Now test on a new host 192.168.75.10:

# Operation on host 192.168.75.10
echo "192.168.75.11 idocker.io" >> /etc/hosts
mkdir -p /etc/docker/certs.d/idocker.io

# Then go to the nexus host and copy the certificate
scp root.crt root@192.168.75.10:/etc/docker/certs.d/idocker.io

Next, you can start using it.

6. Formal verification

1. pull image

[root@master ~]# docker pull redis
Using default tag: latest
latest: Pulling from library/redis
8559a31e96f4: Pull complete 
85a6a5c53ff0: Pull complete 
b69876b7abed: Pull complete 
a72d84b9df6a: Pull complete 
5ce7b314b19c: Pull complete 
04c4bfb0b023: Pull complete 
Digest: sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5
Status: Downloaded newer image for redis:latest
docker.io/library/redis:latest

2. Log in to private service

This place may also report an error when logging in, saying that the certificate has expired, as follows:

Error response from daemon: Get https://idocker.io/v1/users/: x509: certificate has expired or is not yet valid

In the case of this error, there is only one possible reason. That is, the time of the two servers is not the same, just keep the time of the two servers the same.

yum -y install ntpdate && ntpdate -u cn.pool.ntp.org

After the execution of the two hosts respectively, it is found that the login is successful.

[root@master ~]# docker login -u admin -p admin idocker.io
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

3. Label

docker tag docker.io/library/redis:latest idocker.io/nginx

4. Push image

[root@master ~]# docker push idocker.io/nginx 
The push refers to repository [idocker.io/nginx]
7b9c5be81844: Pushed 
67c707dbd847: Pushed 
72d3a7e6fe02: Pushed 
cdaf0fb0082b: Pushed 
e6b49c7dcaac: Pushed 
13cb14c2acd3: Pushed 
latest: digest: sha256:76ff608805ca40008d6e0f08180d634732d8bf4728b85c18ab9bdbfa0911408d size: 1572

It's uploaded successfully here. Go to nexus 3 to see if it's up.

5. Test pulling image from private server

[root@master ~]# docker images
REPOSITORY                                                                    TAG                 IMAGE ID            CREATED             SIZE
redis                                                                         latest              235592615444        43 hours ago        104MB
idocker.io/redis                                                              latest              235592615444        43 hours ago        104MB
[root@master ~]# docker rmi idocker.io/redis 
Untagged: idocker.io/redis:latest
Untagged: idocker.io/redis@sha256:76ff608805ca40008d6e0f08180d634732d8bf4728b85c18ab9bdbfa0911408d
[root@master ~]# docker images
REPOSITORY                                                                    TAG                 IMAGE ID            CREATED             SIZE
redis                                                                         latest              235592615444        43 hours ago        104MB
[root@master ~]# docker pull idocker.io/redis    
Using default tag: latest
latest: Pulling from redis
Digest: sha256:76ff608805ca40008d6e0f08180d634732d8bf4728b85c18ab9bdbfa0911408d
Status: Downloaded newer image for idocker.io/redis:latest
idocker.io/redis:latest
[root@master ~]# docker images
REPOSITORY                                                                    TAG                 IMAGE ID            CREATED             SIZE
redis                                                                         latest              235592615444        43 hours ago        104MB
idocker.io/redis                                                              latest              235592615444        43 hours ago        104MB

7. Function display of agent

When a certain image does not exist in our local warehouse, it needs to be pulled from the remote warehouse. The operations of other private warehouses are probably pulled from the remote warehouse, and then pushed to the local private warehouse repeatedly. Because nexus has the proxy function, when we pull the remote image, the local image will be synchronized automatically

Take pulling gitlab image as an example:

docker pull idocker.io/gitlab/gitlab-ce
Using default tag: latest
Trying to pull repository docker.io/gitlab/gitlab-ce ...
latest: Pulling from docker.io/gitlab/gitlab-ce
3b37166ec614: Pull complete
504facff238f: Pull complete
ebbcacd28e10: Pull complete
c7fb3351ecad: Pull complete
2e3debadcbf7: Pull complete
8e5e9b12009c: Pull complete
0720fffe6e22: Pull complete
2f336a213238: Pull complete
1656ee3e1127: Pull complete
25fa5248fd38: Pull complete
36b8c1d869a0: Pull complete
Digest: sha256:0dd22880358959d9a9233163147adc4c8f1f5d5af90097ff8dfa383c6be7e25a
Status: Downloaded newer image for docker.io/gitlab/gitlab-ce:latest

Because there is no such image locally, pull it from the remote warehouse, and then go to the warehouse to see what the situation is:

After checking, you can find:
Not in docker local, but in proxy docker hub and docker group

Note: for deletion, it can only be deleted in docker local or proxy docker hub. When deletion is performed in both, it will not be available in docker group automatically

So far, it's basically about the knowledge points of using nexus 3 to build a docker private warehouse.

Tags: Docker Nginx Redis SSL

Posted on Sat, 13 Jun 2020 03:40:34 -0400 by psycovic23