Deploy enterprise's own Docker Hub mirror accelerator

background

Docker Hub sometimes encounters difficulties in retrieving mirrors in China. Generally, the network is slow and takes a long time. It is very likely that the connection is broken and the mirror cannot be pulled out.At this point, you can configure the Mirror Accelerator.Many cloud service providers in China provide domestic accelerator services, such as Ali Yun, Huawei Cloud, NetEase Cloud and Dow Cloud.It is common to create or modify / etc/docker/Daemon.jsonFiles:

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
    "registry-mirrors": [
        "https://1nj0zren.mirror.aliyuncs.com",
        "http://f1361db2.m.daocloud.io"
    ]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

So your machine can pull the dockerhub image.

| But it's used in the enterprise, and every machine configures thisDamon.jsonDoes the file pull the dockerhub image?

If each machine configures this accelerator, the following problems will occur:

  • In case the address of the accelerator of Ali cloud or Huawei cloud changes, you will have to change every machine, which is very troublesome. Of course, Ali cloud and Huawei cloud are very stable. This is not the case at present, but this is possible.I used Microsoft Cloud's Accelerator address before (https://dockerhub.azk8s.cn/), now the accelerator address Microsoft Cloud is no longer open to the public, only for Microsoft Cloud users.So this is still possible.
  • If you deploy a nginx deployment in k8s, with 10 copies and 10 pod s assigned to different nodes, you will need to request the Ali Cloud Mirror Acceleration Service 10 times to pull the nginx image, which means duplicate pulling, which makes the application deploy a long time.

Therefore, this article will introduce how to deploy dockerhub container image accelerator service within the enterprise, with the following functions:

  • Cache mirror.When a node comes to the Accelerator Service to pull a mirror, if the mirror already exists in the Accelerator Service, the node can pull directly from the Accelerator Service.If the Accelerator service does not have this image, you need to pull the dockerhub image and return to the request node.

  • Accelerate the speed of image pulling, thereby accelerating the deployment of applications.Because all nodes are de-mirrored by the Accelerator Service, the Accelerator Service only needs to go to dockerhub once to pull the image back to all nodes.Accelerator service backend has storage, will store the pulled image.

Deployment Accelerator Service Prerequisites

  • 1 vps host (foreign machine), can go toHttps://my.vultr.com/Or Ali cloud to buy one, Vultr rent by hour, very cheap, rent a C1G on the line
  • 1 redis service
  • 1 S3 Storage Service
  • 1 docker registry service: This is a private docker repository with an official image and can be used directly. Click here for details

1. Configure redis

Configure redis username and password, modifyRedis.confTake 6.0.3 as an example

wget http://download.redis.io/releases/redis-6.0.3.tar.gz
tar xzf redis-6.0.3.tar.gz
//modifyRedis.conffile

requirepass zhounanjun
#bind 127.0.0.1
protected-mode no

Save Configuration File

Make a redis image, the docker file of redis (will be modified)Redis.confPlace in a directory with Dockerfile:

FROM redis:6.0.3
COPY redis.conf /usr/local/etc/redis/redis.conf
CMD [ "redis-server", "/usr/local/etc/redis/redis.conf" ]

Function:

docker build -t redis:v1.0  .
docker run  -p 6379:6379 --name some-redis -d redis:v1.0

2. Configure minio

The registry backend supports S3 storage, which uses MinIO as the backend storage. MinIO is an object storage service based on the Apache License v2.0 open source protocol.It is compatible with the Amazon S3 Cloud Storage Service interface and is ideal for storing large amounts of unstructured data, such as pictures, videos, log files, backup data, container/virtual machine mirrors, etc. An object file can be of any size, ranging from a few kb to a maximum of 5T. Introduction to Minio

 docker run -p 9000:9000 --name minio1   -v /mnt/data:/data   -v /mnt/config:/root/.minio  -d  minio/minio server /data

Create / mnt/data directory locally

Access minio front-end UI: http://public network IP:9000 (host on Ali cloud needs to open port 9000 policy)

The user name and password are minioadmin and minioadmin, respectively

Click on the bottom right + sign to create a buckets named mirror-registry, as shown in the following image:

3. Configure registry

Docker registry is a repository that stores docker image s. Its location in the docke ecosystem is shown in the following figure. When running docker push, docker pull, docker search, it actually communicates with docker registry through docker daemon.In addition to being a private repository, registry also has a caching mode, which allows you to pull images from the registry instead of pushing them. This caching mode can also be used as a function of the mirror accelerator.

Then start configuring registry to act as a mirror acceleration service:

Configure registry'sConfig.yml:

version: 0.1
log:
  fields:
    service: registry
  level: info
storage:
  cache:# Regisry needs to use the cache itself, using either memory or redis, which uses redis
    blobdescriptor: redis
  s3: Configure Backend Storage
    accesskey: minioadmin #minio user name
    secretkey: minioadmin #minio password
    region: huadong #Build your own Minio and fill it in
    regionendpoint: http://8.210.20.74:9000 #minio Service Address
    bucket: mirror-registry #The name of the bucket created in minio
    encrypt: false
    secure: false
    v4auth: true
    chunksize: 5242880
    rootdirectory: /
  delete:
    enabled: true
  maintenance:
    uploadpurging:
      enabled: true
      age: 168h
      interval: 24h
      dryrun: false
    readonly:
      enabled: false
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
redis:
  addr: 8.210.20.74:6379 #redis service
  password: zhounanjun #redis password
  db: 0
  dialtimeout: 10ms
  readtimeout: 10ms
  writetimeout: 10ms
  pool:
    maxidle: 16
    maxactive: 64
    idletimeout: 300s
health: #Configure health checks for monitoring
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
  tcp:
   - addr: 8.210.20.74:6379
     timeout: 3s
     interval: 10s
     threshold: 3

Dockerfile to make a registry image:

FROM registry:latest
LABEL maintainer="zhounanjun"
COPY entrypoint.sh /entrypoint.sh
COPY config.yml /etc/docker/registry/config.yml

registry needsEntrypoint.sh:

#!/bin/sh

set -e

CONFIG_YML=/etc/docker/registry/config.yml

if [ -n "$PROXY_REMOTE_URL" -a `grep -c "$PROXY_REMOTE_URL" $CONFIG_YML` -eq 0 ]; then
    echo "proxy:" >> $CONFIG_YML
    echo "  remoteurl: $PROXY_REMOTE_URL" >> $CONFIG_YML
    echo "------ Enabled proxy to remote: $PROXY_REMOTE_URL ------"
elif [ $DELETE_ENABLED = true -a `grep -c "delete:" $CONFIG_YML` -eq 0 ]; then
    sed -i '/rootdirectory/a\  delete:' $CONFIG_YML
    sed -i '/delete/a\    enabled: true' $CONFIG_YML
    echo "------ Enabled local storage delete -----"
fi
case "$1" in
    *.yaml|*.yml) set -- registry serve "$@" ;;
    serve|garbage-collect|help|-*) set -- registry "$@" ;;
esac

exec "$@"

Makefile

VERSION ?= v1.0

image:
        docker build -t zhounanjun/mirror:${VERSION} .
run-dockerhub:
        docker run -itd -p 7669:5000 -e PROXY_REMOTE_URL=https://registry-1.docker.io  --restart=always  --name registry-mirror zhounanjun/mirror:${VERSION}
        

PROXY_in Makefile aboveREMOTE_The url environment variable is important. The value of this url is to pull the address of the dockerhub image. You can configure it as the Ali Cloud Accelerator address, or you can configure it as the official dockerhub address.If you don't have a foreign machine, write the address of the Ali cloud mirror accelerator. The official address of the dokcerhub I wrote here is because I'm deployed on foreign servers, which pull dockerhub mirror thieves fast.

4. Deploy Accelerators

Deploy the Accelerator service on a node in your intranet environment that you need to run:

make image
make run-dockerhub

Then you configure the following in the nodes of the k8s cluster:

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
    "registry-mirrors": [
        "https://192.168.1.1000:7669/"
    ]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

Then your k8s cluster can pull the dockerhub image normally. The more frequently you use the mirror accelerator service, the more obvious the acceleration effect will be, because your internal accelerators will cache more and more.

Deploying Mirror Accelerators on the Ali Cloud

Because the host on vps might be blocked, you can buy a Hong Kong host on Aliyun, which is more stable. I bought a lightweight server and object storage Oss on Aliyun.It cost 40 yuan in total.If the enterprise uses it internally, it is recommended to buy a server of Aliyun ECS, pay attention to the bandwidth!Better bandwidth, faster mirroring.
1. Create an object store and a bucket under Aliyun oss, as shown in the figure

2. Create three enterprise servers of Aliyun Hong Kong Server, with bandwidth base capacity: 25 Gbit/s is equivalent to 3G/s.No, it should be fast.Figure

In order to successfully use the object storage in Ali Cloud, you need to create AccessKey on Ali Cloud, how to create AccessKey in Ali Cloud:

  1. Log on as the main account Ali Cloud Management Console .
  2. Place your mouse over the account icon at the top right of the page and click accesskeys.
  3. On the security prompt page, choose whether to get Accesskey for the main or sub account.

Using Ali Cloud's object storage only requires modifying the registry'sConfig.yaml:

version: 0.1
log:
  fields:
    service: registry
  level: info
storage:
  cache:# Regisry needs to use the cache itself, using either memory or redis, which uses redis
    blobdescriptor: redis
  oss:
    accesskeyid: LTAI4G8T5y8m2TtfwfgD66Kj  #Create your own console
    accesskeysecret: MPQUxt8O0csgbdfdjZdcAQPIEbVgHFa  #Create your own console
    region: oss-cn-hangzhou #Domain to which you belong
    bucket: webplus-cn-hangzhou-s-5df099203c3f2876a5b6b344  #bucket name
  delete:
    enabled: true
  maintenance:
    uploadpurging:
      enabled: true
      age: 168h
      interval: 24h
      dryrun: false
    readonly:
      enabled: false
http:
  addr: :5000 # Regisry Service Address
  headers:
    X-Content-Type-Options: [nosniff]
redis:
  addr: 8.210.20.74:6379 #redis service
  password: zhounanjun #redis password
  db: 0
  dialtimeout: 10ms
  readtimeout: 10ms
  writetimeout: 10ms
  pool:
    maxidle: 16
    maxactive: 64
    idletimeout: 300s
health: #Configure health checks for monitoring
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
  tcp:
   - addr: 8.210.20.74:6379
     timeout: 3s
     interval: 10s
     threshold: 3

As with other configurations above, redis starts with containers.

Because Ali Yun Hong Kong server has its own public network address, this registry service can be reached by any machine that can connect to an external network.

The following describes my deployment scenario:

My need is to have my own dockerhub mirror accelerator, not relying on the accelerators of public clouds such as Ali Cloud.

So I'm going to find an overseas server, deploy a complete set of Accelerator services (I'm going to Ali Cloud to deploy, rent Ali Cloud's object storage), and deploy PROXY_in registry's makefile to overseas nodesREMOTE_URL parameter set toHttps://registry-1.docker.ioThen machines in the private cloud inside the enterprise may not be able to connect to the external network, so you can find several internal nodes to deploy the Accelerator service, where makefile's PROXY_REMOTE_The URL is set to the registry service address of the overseas nodes, requesting the network policy of opening these machines to the overseas nodes.Other internal nodes only need to be configured underDamon.jsonAnd the registry-mirrors parameter points to the service address of the internal acceleration node registry.Of course, if the internal node can fully open the network policy to overseas nodes, then even the internal accelerator service node can be omitted, but in this case, the speed of internal node mirroring depends entirely on the speed of internal nodes accessing the public network, which is definitely not fast!

summary

Deploying according to my deployment scenario, you don't need to rely on any public cloud mirror acceleration service to be self-reliant and self-employed within your enterprise.If you have any questions, please comment in the comments area and I will reply in time!
Welcome to my WeChat public number "Cloud Native Handbook", I will update it regularly!

Reference article:
[1]:https://docs.docker.com/registry/recipes/mirror/
[2]: https://docs.docker.com/registry/configuration/

Tags: Redis Docker network sudo

Posted on Tue, 23 Jun 2020 21:22:27 -0400 by radarhill