Deployment and publishing policy

  Grayscale / scroll Publishing

Gray publishing is an extension of Canary publishing, which divides publishing into different stages / batches, and the number of users in each stage / batch increases step by step. If no problems are found in the new version at the current stage, the number of users will be increased to the next stage until it is extended to all users.

Gray publishing can reduce publishing risk and is a publishing strategy with zero downtime. It gradually switches from one version to another by switching the routing weight between online coexisting versions. The whole release process will last for a long time. During this period, the old and new codes coexist. Therefore, in the development process, the compatibility between versions needs to be considered. The coexistence of the old and new codes cannot affect the function availability and user experience. When there is a problem with the new version of code, grayscale publishing can quickly roll back to the old version of code.

Combined with feature switch and other technologies, gray publishing can realize more complex and flexible publishing strategy.

Advantages:

  1. The impact on user experience is relatively small, and there is no need to stop publishing

  2. Ability to control release risk

inferiority:

  1. It will take a long time to publish

  2. Complex publishing systems and load balancers are required

  3. You need to consider compatibility when old and new versions coexist

Applicable scenarios:

  1. Suitable for production environments with high availability

Traffic distribution is realized through annotation  

Grayscale also has an online environment. Now when grayscale is to be sent, a small number of users should receive this new version. This small number of users can allocate it through cooki e or header, or through ingress weight.

  • During the normal operation of the old version of V1, the user request traffic is in V1
  • The new version of V2 is released and controlled by ingress. Some traffic is led to V2 through header/cookie/weight   10%
  • Update Ingress,     Flow expanded to 30%
  • Update Ingress,     Flow expanded to 60%
  • Update Ingress,     Traffic expanded to 90%
  • Update Ingress,     Flow expanded to 100%
  • At this time, the test is OK. Replace the old version of V1 with V2 helm chart deployment.
[root@master devops]# kubectl get svc -n devops
NAME       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginxapp   ClusterIP   10.233.34.137   <none>        80/TCP    46h
nginxnpm   ClusterIP   10.233.18.40    <none>        80/TCP    46h


[root@master devops]# kubectl get pod -n devops
NAME                                   READY   STATUS    RESTARTS   AGE
nginx-npm-blue-6c5ddc9ffd-jl25q        1/1     Running   1          46h
nginx-npm-blue-6c5ddc9ffd-qx2kw        1/1     Running   1          46h
nginx-npm-blue-6c5ddc9ffd-vhtnx        1/1     Running   1          46h
nginxapp-1.20-green-5668c79ff7-9l7kl   1/1     Running   1          46h
nginxapp-1.20-green-5668c79ff7-fckpc   1/1     Running   1          46h
nginxapp-1.20-green-5668c79ff7-vmvq9   1/1     Running   1          46h
[root@master ~]# curl 10.233.18.40  
<!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>vuedemo</title><link href=/static/css/app.30790115300ab27614ce176899523b62.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=/static/js/manifest.2ae2e69a05c33dfc65f8.js></script><script type=text/javascript src=/static/js/vendor.0aad2172a75b8ed4f46c.js></script><script type=text/javascript src=/static/js/app.2f2e5edd9af2c59aa514.js></script></body></html>[root@master ~]# 
[root@master ~]# 

[root@master ~]# curl 10.233.34.137
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Now production  

[root@master devops]# cat ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginxnpm-ingress
  namespace: devops
spec:
  rules:
  - host: nginxnpm.devops.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginxapp
          servicePort: 80

Weight based publishing

[root@master devops]# cat ingress-weight.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginxapp-canary
  namespace: devops
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules:
  - host: nginxnpm.devops.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginxnpm
          servicePort: 80

 

  Visit the same page twice and request different versions respectively.

Header header traffic distribution

During the visit, you can distinguish by the request header. For example, if you come from Beijing, you can let users from Beijing see it first, and then you can match it according to the information in the request header.

Here is the request from Beijing to visit the new version.

[root@master devops]# cat ingress-header.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginxapp-canary
  namespace: devops
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "Region"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bj"
spec:
  rules:
  - host: nginxnpm.devops.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginxnpm
          servicePort: 80

For users from Hangzhou, the traffic is still the old version

[root@master devops]# for i in {1..2};do curl -H "region: hz" nginxnpm.devops.com && sleep 5s;done
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>


<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

  Users from Beijing visit with a new version of traffic

[root@master devops]# for i in {1..2};do curl -H "region: bj" nginxnpm.devops.com && sleep 5s;done
<!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>vuedemo</title><link href=/static/css/app.30790115300ab27614ce176899523b62.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=/static/js/manifest.2ae2e69a05c33dfc65f8.js></script><script type=text/javascript src=/static/js/vendor.0aad2172a75b8ed4f46c.js></script><script type=text/javascript src=/static/js/app.2f2e5edd9af2c59aa514.js></script></body></html>

<!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>vuedemo</title><link href=/static/css/app.30790115300ab27614ce176899523b62.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=/static/js/manifest.2ae2e69a05c33dfc65f8.js></script><script type=text/javascript src=/static/js/vendor.0aad2172a75b8ed4f46c.js></script><script type=text/javascript src=/static/js/app.2f2e5edd9af2c59aa514.js></script></body></html>[root@master devops]# 

Distribute according to cookie traffic

A cookie has only two values, one is always (new version) and the other is never

[root@master devops]# cat ingress-cookie.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginxapp-canary
  namespace: devops
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "from"
spec:
  rules:
  - host: nginxnpm.devops.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginxnpm
          servicePort: 80
[root@master devops]#  curl  nginxnpm.devops.com --cookie "from=always"
<!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>vuedemo</title><link href=/static/css/app.30790115300ab27614ce176899523b62.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=/static/js/manifest.2ae2e69a05c33dfc65f8.js></script><script type=text/javascript src=/static/js/vendor.0aad2172a75b8ed4f46c.js></script><script type=text/javascript src=/static/js/app.2f2e5edd9af2c59aa514.js></script></body></html>[root@master devops]# 
[root@master devops]# 
[root@master devops]# 
[root@master devops]# 
[root@master devops]#  curl  nginxnpm.devops.com --cookie "from=a"
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Tags: jenkins

Posted on Wed, 06 Oct 2021 15:05:25 -0400 by ofSHIZ