1. Kubernetes Stateless Service VS Stateful Service
1) Kubernetes Stateless ServiceKubernetes stateless service characteristics:
1) means that instances running the service do not store data that needs to be persisted locally and that multiple instances respond to the same request with identical results;
2) Multiple instances can share the same persisted data.For example, nginx instance, tomcat instance, etc.
3) The related Kubernetes resources are ReplicaSet, ReplicationController, Deployment, etc. Because they are stateless services, the PD names created by these controllers are random.And when scaling, it is not explicit to scale a Pod, but random, because all instances get the same return value, so any Pod can be scaled;
Kubernetes has stateful service characteristics:
1) Stateful services can be referred to as services that require data storage capabilities, or services, queues, etc. of a multithreaded type.(e.g. mysql database, kafka, zookeeper, etc.);
2) Each instance needs its own independent persistent storage and is defined in Kubernetes by declaring templates.Persistent volume declaration templates are created prior to pod creation and are bound to pods, where templates can define multiple;
3) The related Kubernetes resources are StatefulSet.Since there are stateful services, each Pod has a specific name and network identity.For example, a Pod name is a StatefulSet name + an ordered number (0, 1, 2...);
4) When scaling, you can know which Pod will be scaled, starting with the largest number.And StatefulSet is not allowed to shrink if existing instances are unhealthy;
The main manifestations are as follows:
1) Number of instances: A stateless service can have one or more instances, thus supporting two service capacity regulation modes; a stateful service function has one instance that does not allow multiple instances to be created, and therefore does not support service capacity regulation;
2) Storage volumes: Stateless services can have or cannot back up data in storage volumes, even if they do; Stateless services must have storage volumes, and when creating services, the size of disk space allocated by the storage volumes must be specified;
3) Data Storage: All data in the running process of stateless service (except log and monitoring data) exists in the file system in the container instance, and if the instance stops or deletes, the data will be lost and cannot be retrieved. For stateful service, the contents of files in the directory where the storage volume has been mounted can be backed up at any time, and the backed up dataIt can be downloaded or used to restore new services.However, data in directories that do not have mounted volumes can still not be backed up and saved. If instances are stopped or deleted, the contents of files in these non-mounted volumes will also be lost.
StatefulSet is a load management controller API provided by Kubernetes to manage stateful applications.On the basis of Pods management, ensure the order and consistency of Pods.StatefulSet, like Deployment, uses a container's Spec to create a Pod, unlike StatefulSet, which creates a Pod that maintains a persistent marker (such as a Pod Name) throughout its life cycle.
5) Features of StatefulSet1) Stable persistent storage, i.e. access to the same persistent data after Pod rescheduling, based on PVC;
2) Stable network flags, i.e. PodName and HostName remain unchanged after Pod is rescheduled, based on Headless Service (Service without Cluster IP);
3) Ordered deployment, ordered expansion, that is, Pod is ordered, deployed or expanded in the order defined (that is, from 0 to N-1, all previous Pods must be Running and Ready before the next Pod runs), based on init containers;
4) Orderly contraction, orderly deletion (that is, from N-1 to 0);
2. Auto-create PVC using StatefulSet
1) Build NFS Shared Storage
For convenience, deploy NFS storage directly on the master node!
[root@master ~]# yum -y install nfs-utils rpcbind [root@master ~]# vim /etc/exports /nfsdata *(rw,sync,no_root_squash) [root@master ~]# mkdir /nfsdata [root@master ~]# systemctl start nfs-server [root@master ~]# systemctl start rpcbind [root@master ~]# showmount -e Export list for master: /nfsdata *
2) Create rbac authorization
[root@master ~]# vim rbac-rolebind.yaml apiVersion: v1 #Create a service account for authentication kind: ServiceAccount metadata: name: nfs-provisioner --- apiVersion: rbac.authorization.k8s.io/v1 #Create Cluster Rules kind: ClusterRole metadata: name: nfs-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["watch", "create", "update", "patch"] - apiGroups: [""] resources: ["services", "endpoints"] verbs: ["get","create","list", "watch","update"] - apiGroups: ["extensions"] resources: ["podsecuritypolicies"] resourceNames: ["nfs-provisioner"] verbs: ["use"] --- kind: ClusterRoleBinding #Binding service authenticated users to cluster rules apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-provisioner subjects: - kind: ServiceAccount name: nfs-provisioner namespace: default #Required paragraph, otherwise error will be prompted roleRef: kind: ClusterRole name: nfs-provisioner-runner apiGroup: rbac.authorization.k8s.io [root@master ~]# kubectl apply -f rbac-rolebind.yaml
3) Create nfs-deployment. Resources
[root@master ~]# vim nfs-deployment.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nfs-client-provisioner spec: replicas: 1 #Number of copies specified is 1 strategy: type: Recreate #Specify policy type as reset template: metadata: labels: app: nfs-client-provisioner spec: serviceAccount: nfs-provisioner #Specify the authenticated user account created in the rbac yanl file containers: - name: nfs-client-provisioner image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner #Mirror used volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes #Specify the directory mounted in the container env: - name: PROVISIONER_NAME #Variables inside containers are used to specify the name of the storage being provided value: lzj - name: NFS_SERVER #Variables inside the container are used to specify the IP address of the nfs service value: 192.168.1.1 - name: NFS_PATH #Variables inside the container specify the directory corresponding to the nfs server value: /nfsdata volumes: #Specify the path and IP of nfs mounted in the container - name: nfs-client-root nfs: server: 192.168.1.1 path: /nfsdata [root@master ~]# kubectl apply -f nfs-deployment.yaml [root@master ~]# kubectl get pod NAME READY STATUS RESTARTS AGE nfs-client-provisioner-66df958f9c-mbvhv 1/1 Running 0 2m34s
4) Create SC (Storage Class)
[root@master ~]# vim sc.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: stateful-nfs namespace: xiaojiang-test provisioner: lzj #This corresponds to the value of PROVISIONER_NAME in the env environment variable of nfs-client-provisioner. reclaimPolicy: Retain #Specify Recycle Policy as Retain (Manual Release) [root@master ~]# kubectl apply -f sc.yaml [root@master ~]# kubectl get StorageClass NAME PROVISIONER AGE stateful-nfs lzj 17s
5) Create Pod
[root@master ~]# vim statefulset.yaml apiVersion: v1 kind: Service metadata: name: headless-svc #Headless service can be from name labels: app: headless-svc spec: ports: - port: 80 name: myweb selector: app: headless-pod clusterIP: None #No cluster IP address is assigned, so no load balancing capability is available --- apiVersion: apps/v1 kind: StatefulSet #Define applications running in pod metadata: name: statefulset-test spec: serviceName: headless-svc replicas: 3 selector: matchLabels: app: headless-pod template: metadata: labels: app: headless-pod spec: containers: - image: httpd name: myhttpd ports: - containerPort: 80 name: httpd volumeMounts: - mountPath: /usr/local/apache2/htdocs name: test volumeClaimTemplates: #Define the template used to create PVC - metadata: name: test annotations: #This is the specified storageclass volume.beta.kubernetes.io/storage-class: stateful-nfs spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi [root@master ~]# kubectl apply -f statefulset.yaml [root@master ~]# kubectl get pod NAME READY STATUS RESTARTS AGE nfs-client-provisioner-66df958f9c-mbvhv 1/1 Running 0 10m statefulset-test-0 1/1 Running 0 29s statefulset-test-1 1/1 Running 0 18s statefulset-test-2 1/1 Running 0 11s [root@master ~]# kubectl get pv [root@master ~]# kubectl get pvc #PV and PVC have been generated [root@master ~]# ls /nfsdata/ default-test-statefulset-test-0-pvc-54d0b06c-698e-4f1a-8327-255b10978cbe default-test-statefulset-test-1-pvc-1b499d49-a787-4f2b-b238-404b05f75fd7 default-test-statefulset-test-2-pvc-7766f8da-6f3b-4c1f-9eb8-dfadda1e656f [root@master ~]# echo "hello world" > /nfsdata/default-test-statefulset-test-0-pvc-54d0b06c-698e-4f1a-8327-255b10978cbe/index.html [root@master ~]# kubectl get pod -o wide | grep test-0 statefulset-test-0 1/1 Running 0 4m53s 10.244.2.4 node02 <none> <none> [root@master ~]# curl 10.244.2.4 hello world [root@master ~]# curl -I 10.244.2.4 HTTP/1.1 200 OK Date: Wed, 12 Feb 2020 09:52:04 GMT Server: Apache/2.4.41 (Unix) Last-Modified: Wed, 12 Feb 2020 09:45:37 GMT ETag: "c-59e5dd5ac0a63" Accept-Ranges: bytes Content-Length: 12 Content-Type: text/html #You can see that the service that now provides web pages is Apache
6) Update and expand pod
[root@master ~]# vim statefulset.yaml apiVersion: v1 kind: Service metadata: name: headless-svc labels: app: headless-svc spec: ports: - port: 80 name: myweb selector: app: headless-pod clusterIP: None --- apiVersion: apps/v1 kind: StatefulSet metadata: name: statefulset-test spec: updateStrategy: rollingUpdate: partition: 2 #Specify the number of parallel upgrades serviceName: headless-svc replicas: 10 selector: matchLabels: app: headless-pod template: metadata: labels: app: headless-pod spec: containers: - image: nginx #Replace the mirror used for expansion name: myhttpd ports: - containerPort: 80 name: httpd volumeMounts: - mountPath: /usr/share/nginx/html/ #Replace the home directory in the container name: test volumeClaimTemplates: - metadata: name: test annotations: #This is the specified storageclass volume.beta.kubernetes.io/storage-class: stateful-nfs spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi [root@master ~]# kubectl apply -f statefulset.yaml [root@master ~]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nfs-client-provisioner-66df958f9c-mbvhv 1/1 Running 0 35m 10.244.2.2 node02 <none> <none> statefulset-test-0 1/1 Running 0 21m 10.244.2.4 node02 <none> <none> statefulset-test-1 1/1 Running 0 20m 10.244.1.4 node01 <none> <none> statefulset-test-2 1/1 Running 0 3m52s 10.244.1.9 node01 <none> <none> statefulset-test-3 1/1 Running 0 4m54s 10.244.2.5 node02 <none> <none> statefulset-test-4 1/1 Running 0 4m43s 10.244.1.6 node01 <none> <none> statefulset-test-5 1/1 Running 0 4m31s 10.244.2.6 node02 <none> <none> statefulset-test-6 1/1 Running 0 4m25s 10.244.1.7 node01 <none> <none> statefulset-test-7 1/1 Running 0 4m19s 10.244.2.7 node02 <none> <none> statefulset-test-8 1/1 Running 0 4m12s 10.244.1.8 node01 <none> <none> statefulset-test-9 1/1 Running 0 4m3s 10.244.2.8 node02 <none> <none> [root@master ~]# ls /nfsdata/ | wc -l 10 [root@master ~]# curl -I 10.244.2.4 HTTP/1.1 200 OK Date: Wed, 12 Feb 2020 10:05:34 GMT Server: Apache/2.4.41 (Unix) Last-Modified: Wed, 12 Feb 2020 09:45:37 GMT ETag: "c-59e5dd5ac0a63" Accept-Ranges: bytes Content-Length: 12 Content-Type: text/html [root@master ~]# curl -I 10.244.2.8 HTTP/1.1 403 Forbidden Server: nginx/1.17.8 Date: Wed, 12 Feb 2020 10:05:41 GMT Content-Type: text/html Content-Length: 153 Connection: keep-alive
It can be seen that the pod is not changed during the expansion operation, which is the characteristic of StatefulSet!
To the end of this article, thank you for reading -----------