admission control
1, What is K8S access control
After the resource is created and authenticated, Kube apiserver intercepts the data before writing it to etcd, and then changes the resource and determines its correctness.
LimitRange，ResourceQuota
Objective: to control the resource usage in a specific namespace, and finally realize the fair use and cost control of the cluster
- LimitRange: add default computing resource requirements and computing resource limits for Pod; And storage resource requirements and storage resource constraints;
Supports restrictions at the container and Pod levels respectively;
For example, specify the minimum and maximum amount of CPU and memory for each pod in the namespace - ResourceQuota: limit the number of resources, limit the total amount of calculated resources, and limit the total amount of stored resources;
For example, under the specified namespace, only the number of pods and SVCS can be created, and the minimum and maximum amount of all Pod CPUs and memory
- The functions to be realized are as follows:
Limit the consumption of computing resources of Pod in running state
Limit the number of persistent storage volumes to control access to storage
Limit the number of load balancers to control costs
Prevent misuse of network ports
Provide the default computing resource Requests to facilitate the system to make more optimized scheduling
- The restrictions provided by a LimitRange object can:
Implement minimum and maximum resource usage limits for each Pod or Container in a namespace.
Implement the minimum and maximum storage space that each PersistentVolumeClaim can apply for in a namespace.
Implement the control of the ratio of the request value and the limit value of a resource in a namespace.
Set the default request / limit value for computing resources in a namespace, and automatically inject it into multiple containers at run time.
2, Enable LimitRang
Support for LimitRange is enabled by default since kubernetes version 1.10.
LimitRange support is also enabled by default in many Kubernetes distributions.
The name of LimitRange must be a legal DNS subdomain name.
3, Overview of limits
The administrator creates a LimitRange object in a namespace.
Users create resources such as Pod, Container and PersistentVolumeClaim in the namespace.
The LimitRange admission controller sets default and limit values for all pods and containers that do not have calculation resource requirements set, and tracks their usage to ensure that the minimum and maximum resource usage and usage ratio values in any LimitRange object existing in the namespace are not exceeded.
If the creation or update of resources (Pod, Container, PersistentVolumeClaim) violates the LimitRange constraint, the request to the API server will fail and return the HTTP status code 403 FORBIDDEN and a message describing which constraint is violated.
If the LimitRange in the namespace enables restrictions on cpu and memory, the user must specify the required and limited usage of these values. Otherwise, the system will reject the creation of Pod.
The verification of LimitRange is only carried out in the Pod admission stage, and the running Pod is not verified
Example 1: create a LimitRange admission control policy
- Limit Pod, container, PVC and other resources
[root@k8s-master authfiles]# cat limitrange-demo.yaml
apiVersion: v1
kind: LimitRange
metadata:
name: core-resource-limits
namespace: dev #Namespace level resources take effect for the entire namespace
spec:
limits:
- type: Pod #Limit Pod
max:
cpu: "4" #The maximum CPU can not exceed 4 cores
memory: "4Gi" #Memory cannot exceed 4G
min:
cpu: "500m" #The minimum CPU shall not be less than 500m
memory: "16Mi" #Memory cannot be less than 16M
- type: Container #Restrictions on containers
max:
cpu: "4"
memory: "1Gi"
min:
cpu: "100m"
memory: "4Mi"
default: #Upper threshold #If there are no requests or limits in the container, the additional default value will be used
cpu: "2"
memory: "512Mi"
defaultRequest: #Lower threshold
cpu: "500m"
memory: "64Mi"
maxLimitRequestRatio: #The upper threshold / lower threshold of the maximum limit request ratio can also be limited by multiples
cpu: "4"
- type: PersistentVolumeClaim #Limit PV
max:
storage: "10Gi"
min:
storage: "1Gi"
default:
storage: "5Gi"
defaultRequest:
storage: "1Gi"
maxLimitRequestRatio: #Multiple can also be used as a limit without lower threshold
storage: "5"- Create LimitRange policy
[root@k8s-master authfiles]# kubectl apply -f limitrange-demo.yaml limitrange/core-resource-limits created [root@k8s-master authfiles]# kubectl get limitrange -n dev NAME CREATED AT core-resource-limits 2021-08-25T02:29:04Z [root@k8s-master authfiles]# kubectl get pod -n dev NAME READY STATUS RESTARTS AGE deployment-demo-fb544c5d8-4jvgh 1/1 Running 0 16h deployment-demo-fb544c5d8-7fq8f 1/1 Running 0 16h deployment-demo-fb544c5d8-hdsz4 1/1 Running 0 16h deployment-demo-fb544c5d8-qq7mp 1/1 Running 0 16h [root@k8s-master authfiles]# kubectl describe limitrange core-resource-limits -n dev Name: core-resource-limits Namespace: dev Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Pod cpu 500m 4 - - - Pod memory 16Mi 4Gi - - - Container memory 4Mi 1Gi 64Mi 512Mi - Container cpu 100m 4 500m 2 4 PersistentVolumeClaim storage 1Gi 10Gi 1Gi 5Gi 5
- Verify that the test strategy is in effect
[root@k8s-master authfiles]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev
pod/testpod created
[root@k8s-master authfiles]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
testpod-1017 1/1 Running 0 93s
[root@k8s-master authfiles]# kubectl get pods testpod -n dev -o yaml
...
spec:
containers:
- image: kubernetes/demoapp:v1.0
imagePullPolicy: IfNotPresent
name: testpod
resources: #Default value
limits:
cpu: "2" #Maximum 2 cores
memory: 512Mi
requests:
cpu: 500m #Minimum 500m
memory: 64Mi- Failed to create Pod less than memory limit
[root@k8s-master authfiles]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev --limits='cpu=2,memory=1Gi' --requests='cpu=1,memory=8Mi' Error from server (Forbidden): pods "testpod-10340" is forbidden: minimum memory usage per Pod is 16Mi, but request is 8388608
ResourceQuota
- The ResourceQuota resource limits the computational resource requirements and the total amount of computational resource limits for all Pod objects in a non terminating state in the namespace.
- CPU or requests.cpu: the total amount of requests related to CPU resources;
- Memory or requests.memory: the total amount of requests related to memory resources;
- Limits.cpu: total amount limit of CPU resource related restrictions;
- limits.memory: total limit of memory resource related limits;
- The concurcebuota resource also supports the quota for the total demand and limit of PVC storage resources in the local namespace. It can be defined from three categories: all PVC in the namespace, PVC belonging to a specific storage class and PVC based on local temporary storage.
- requests.storage: total amount limit of all PVC storage requirements; Space constraints;
- persistentvolumeclaims: total PVC quota that can be created; Quantity limit;
- < storage class name >. Storageclass. Storage. K8s. IO / requests. Storage: the total amount limit of all PVC storage requirements available on a specific storage class;
- < storage class name >. Storageclass. Storage. K8s. IO / persistentvolumeclaims: the total amount of PVC available on a specific storage class;
- requests.ephemeral-storage: the total number of requests for local temporary storage resources that can be used by all pods;
- limits.ephemeral-storage: the total limits of local temporary storage resources available to all pods.
Example 2: create a ResourceQuota admission control policy
[root@k8s-master authfiles]# cat resourcequota-demo.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: resourcequota-demo
namespace: dev
spec:
hard :
pods: "5" #Total Pod
count/services: "5" #Total svc
count/configmaps: "5"
count/secrets: "5"
count/cronjobs.batch: "2"
requests.cpu: "2" #Total threshold under cpu
requests.memory: "4Gi"
limits.cpu: "4" #Total threshold on cpu
limits.memory: "8Gi"
count/deployments.apps: "2"
count/statefulsets.apps: "2"
persistentvolumeclaims: "6"
requests.storage: "20Gi"
longhorn.storageclass.storage.k8s.io/requests.storage: "20Gi"
longhorn.storageclass.storage.k8s.io/persistentvolumeclaims: "6"
[root@k8s-master authfiles]# kubectl apply -f resourcequota-demo.yaml
resourcequota/resourcequota-demo created
[root@k8s-master authfiles]# kubectl get ResourceQuota -n dev
NAME AGE REQUEST LIMIT
resourcequota-demo 33s count/configmaps: 0/5, count/cronjobs.batch: 0/2, count/deployments.apps: 0/2, count/secrets: 3/5, count/services: 0/5, count/statefulsets.apps: 0/2, longhorn.storageclass.storage.k8s.io/persistentvolumeclaims: 0/6, longhorn.storageclass.storage.k8s.io/requests.storage: 0/20Gi, persistentvolumeclaims: 0/6, pods: 1/5, requests.cpu: 500m/2, requests.memory: 64Mi/4Gi, requests.storage: 0/20Gi limits.cpu: 2/4, limits.memory: 512Mi/8Gi- Create a Pod that does not meet the standard to verify whether the test strategy is effective
[root@k8s-master authfiles]# kubectl describe resourcequota resourcequota-demo -n dev Name: resourcequota-demo Namespace: dev Resource Used Hard -------- ---- ---- count/configmaps 0 5 count/cronjobs.batch 0 2 count/deployments.apps 0 2 count/secrets 3 5 count/services 0 5 count/statefulsets.apps 0 2 limits.cpu 2 4 limits.memory 512Mi 8Gi longhorn.storageclass.storage.k8s.io/persistentvolumeclaims 0 6 longhorn.storageclass.storage.k8s.io/requests.storage 0 20Gi persistentvolumeclaims 0 6 pods 1 5 requests.cpu 500m 2 requests.memory 64Mi 4Gi requests.storage 0 20Gi [root@k8s-master authfiles]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev pod/testpod-10678 created [root@k8s-master authfiles]# kubectl describe resourcequota resourcequota-demo -n dev Name: resourcequota-demo Namespace: dev Resource Used Hard -------- ---- ---- count/configmaps 0 5 count/cronjobs.batch 0 2 count/deployments.apps 0 2 count/secrets 3 5 count/services 0 5 count/statefulsets.apps 0 2 limits.cpu 4 4 #Quota has been used up. Cannot create Pod at limits.memory 1Gi 8Gi longhorn.storageclass.storage.k8s.io/persistentvolumeclaims 0 6 longhorn.storageclass.storage.k8s.io/requests.storage 0 20Gi persistentvolumeclaims 0 6 pods 2 5 requests.cpu 1 2 requests.memory 128Mi 4Gi requests.storage 0 20Gi [root@k8s-master authfiles]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev Error from server (Forbidden): pods "testpod-30200" is forbidden: exceeded quota: resourcequota-demo, requested: limits.cpu=2, used: limits.cpu=4, limited: limits.cpu=4 #Error reporting policy has taken effect