Why write such a tool? This is because our company has multiple kubernetes clusters (8 +), and cloud hosting services can't access the apserver configuration, which brings us a pain point. Development and sre need to log in to k8s dashbao and different departments and roles need different authorizations, which used to be through sa token Log in to the dashboard, but with the growth of k8s cluster, the user needs to be informed of the corresponding dashboard access address and the corresponding token for each additional cluster, which is very painful for both the provider and the user. Is there a tool that can provide a unified address to log on to the multi cluster dashboard? After a search, we found that there is no one. Most of the schemes on the market are single cluster integrated LDAP, mainly based on DEX. But the unified login and authorization scheme of single cluster makes people feel very difficult. Is there no simple and convenient tool for us to use? Well, I'll build such a tool.
Dashboard LDAP integration scheme:
- https://k2r2bai.com/2019/09/29/ironman2020/day14/
- https://blog.inkubate.io/access-your-kubernetes-cluster-with-your-active-directory-credentials
The above two documents are the solution of LDAP. I feel that they are good for people in need!
How to buildWell, since there isn't one, make one automatically!
Goal: easy to use!!! By accessing the same address, log in with LDAP and switch between dashboard s of different clusters. At the same time, corresponding permissions of different clusters can be configured separately!
With the above goals, how to achieve them?
In fact, the implementation method is very simple. First, write a login interface to get through with the company's AD to get users and groups, then associate users or groups with the service account in the k8s cluster to achieve the corresponding rbac and login token, and finally implement a reverse proxy service after login.
Is it very simple!!!
Implementation technology stack: golang(gin, client go, viper, ldap) + Kubernetes Dashboard
How to deployprerequisite
Before using this tool, you need to have the following constraints:
- dashboard has been deployed in k8s clusters and can be accessed by this tool
- There is ldap and management permission to access
- There are corresponding service account s in each cluster that can be mapped. If different users and groups need different operation permissions, rbac authorization can be performed on sa, which will be explained in detail below.
- This tool needs to operate the api of each cluster, so it needs to obtain the apiserver address of each cluster ca.crt And token configuration. For each cluster ca.crt And token, if obtained, will be described later
How to get ca.crt And token
This tool needs to operate the api of each cluster to obtain the corresponding sa and token, so it needs to have the authority to operate each cluster. How to generate corresponding ca certificates and tokens in each cluster? The answer is to create a sa and give it permission.
Execute the following yaml file in each k8s cluster:
apiVersion: v1 kind: ServiceAccount metadata: name: mutiboard-ldap namespace: default --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: mutiboard-ldap-crb roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: mutiboard-ldap-view subjects: - kind: ServiceAccount name: mutiboard-ldap namespace: default --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: mutiboard-ldap-view rules: - apiGroups: - "" resources: - serviceaccounts - secrets verbs: - get - list - apiGroups: - "" resources: - namespaces verbs: - get - list - watch
What this yaml file means: create a sa named mutiboard LDAP, and give serviceaccounts and secrets get and list permissions.
Get the ca.crt :
(⎈|aws-local:default)❯ echo $(kubectl get secret $(kubectl get secret | grep mutiboard-ldap | awk '') -o go-template='{}') | base64 -d -----BEGIN CERTIFICATE----- MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTE5MDEyNTEwMTgzNFoXDTI5MDEyMjEwMTgzNFowFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMmc TW0stLLP+M6Pc9wpRgZufg6eQ7puBfbYgik20QlO4LFtocgNUDa0y+aSXjxheA2C A+o9wW0IC3GHQHKgeFY8KXIJu6wM0TO+JNQy5XZAWfbsLeXU/sLhKuWET/KJzVWT 0uBE+GCADAAQIec1oQXMbQ551hU5gBFcr67NXHpa2qwEGA1mGtZ7ztmW4+IFUD74 G166z4AOgmR4YWxBs/+8NhfWudFD32xevBfSKuHRxRGG5dtffY8QnRbnrmy70HE5 yzLtBvAGfCwtHLTP2ngCAnn2Fb6IeMdIYGpI1544ZjRbzT1YIWsG1v3dlu6tvK1q X5Pj+UTDmJuf2SW52A0CAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAKE2hV0DIG8fSf4/eOi5R2sPRfBW qTwgZDDT9dxZNhbxEInALdruwRUbKRpwaUBOGVpIlaK3/rZkAfjUwoDJ+J4fmmCX w3ySrYFjx6tqVFqCPjDkBHh4xpMwUlvsvryRuCEQUQgjqBvj6sWm9GERF2n3VYBF S8bjsQQAZJoE4W+OKchlEoSFlKhxAoeZx9CD3Rxnhj2og6doVoGCUqAMh4WZWX+w pENnui6M96SysH3SkrA02RXWTGeKzK4E6Av3IG+2a2hauHorbqVfaM6HeL3hkU/B JCWpOgN3T4Fw7E359CBQxnSHPasmZ5VBoyIk/HUU6ZlMK6Xo6JlbS7ZvVl4= -----END CERTIFICATE-----
Get the token of mutiboard LDAP:
(⎈|aws-local:default)❯ echo $(kubectl get secret $(kubectl get secret | grep mutiboard-ldap | awk '') -o go-template='{{.data.token}}') | base64 -d eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Im11dGlib2FyZC1sZGFwLXRva2VuLWJ3NWdmIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6Im11dGlib2FyZC1sZGFwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMjVmZjI4MGQtYWJhMi0xMWVhLWFlOGEtMDIzOTBjMzcyNzhlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6bXV0aWJvYXJkLWxkYXAifQ.q14hqEu2p70_YczDviR6c8McDM5vfnKPzjO9usCsC-uQUxciBbuJU_PK9j3uawppUNlrs3rAPrZIGUS7Jv14rifEXpGxIIfGR6n8-le0b-9YvMZCgs9-jhf-1r01EAnZFh6gcXfxESFguFQI0vYOsX4P2LQvZ9XTMzsqXbW3KGYao5elAjCE4e8Rg4--9e_zU8NGTEycsvUMxP-9p0SaAzn9Iak3saZtAnzJq5hkSf1t7l2_CgEsYN-3b7uGpHupK_zdgAeOflj9ze4Cz2YScv5eixwVXJ-RcI4lgSFCgt5yzSbnIuHgxRZyN3NcYLrSBYKftezZysWm3jELgLPogQ
So far, the ca.crt Both token and token have been obtained, and we will tell you how to configure and use these ca.crt And token
SA RBAC column
apiVersion: v1 kind: ServiceAccount metadata: name: ops-admin namespace: default labels: kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: ops-role rules: - apiGroups: [""] resources: ["namespaces"] verbs: ["get", "list", "watch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: ops-listnamespace roleRef: #Referenced roles kind: ClusterRole name: ops-role apiGroup: rbac.authorization.k8s.io subjects: #subject - kind: ServiceAccount name: ops-admin namespace: default --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: ops-ci-admin namespace: ops-ci roleRef: #Referenced roles kind: ClusterRole name: admin apiGroup: rbac.authorization.k8s.io subjects: #subject - kind: ServiceAccount name: ops-admin namespace: default --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: ops-qa-admin namespace: ops-qa roleRef: #Referenced roles kind: ClusterRole name: admin apiGroup: rbac.authorization.k8s.io subjects: #subject - kind: ServiceAccount name: ops-admin namespace: default
This YAML means to create an OPS admin sa and give the sa two namespace (OPS Ci, OPS QA) admin permissions.
To learn more about rbac, please refer to: https://www.cnblogs.com/wlbl/p/10694364.html
ldap description
Our ldap directory rules are as follows:
|--Domain |--|---Company |--|----|----Branch |--|----|-----|----Department |--|----|-----|-----|-----UsersThe corresponding Distinguished Name is shown as follows:
CN=Peng Xu,OU = Department, OU = branch, OU = company, DC=corp,DC=xxx,DC=comHere I will get the first OU as a group. If your requirements are different from mine, you can propose issue for me to adapt
For details of ldap, please refer to: https://blog.poychang.net/ldap-introduction
configmap.yaml Configuration description
ldap: addr: ldap://192.168.3.81:389 adminUser: xxxxx adminPwd: xxxxxx baseDN: dc=corp,dc=patsnap,dc=com filter: (&(objectClass=person)(sAMAccountName=%s)) attributes: user_dn orgUnitName: OU= #Mapping of global users / user groups to SA rbac: DevOps team: sa: ops-admin ns: kube-system xupeng: sa: inno-admin ns: default clusters: #Cluster alias, the key displayed in the login drop-down box. This alias needs to match the secret.sh In ca.crt One to one correspondence with the key name of token local: #apiserver address, which can be accessed by the current tool apiServer: apiserver-dev.jiunile.com port: 6443 #kubernetes dashboard address, which can be accessed by the current tool dashboard: dashboard-dev.jiunile.com #Cluster description, the name displayed in the login drop-down box desc: Dev Cluster #Breakdown for individual clusters #rbac: # DevOps team: # sa: admin # ns: kube-system # xupeng: # sa: ops-admin # ns: default cnrelease: apiServer: apiserver-cn-release.jiunile.com port: 443 dashboard: dashboard-cn-release.jiunile.com desc: CN Release Cluster usrelease: apiServer: apiserver-us-release.jiunile.com port: 443 dashboard: dashboard-us-release.jiunile.com desc: US Release Cluster euprod: apiServer: apiserver-eu-prod.jiunile.com port: 443 dashboard: dashboard-eu-prod.jiunile.com desc: EU Prod Cluster
deploy
- Modify and deploy deploy/configmap.yaml
- Get the ca.crt And token are written to the corresponding deploy/token
-
Execute the secret.sh Script sh deploy/secret.sh
be careful: secret.sh xx in_ token/xxx_ ca.crt xx in corresponds to configmap.yaml Cluster aliases in must correspond one by one
- deploy/deployment.yaml
visit
http://:31000
Video download address: https://github.com/icyxp/kubernetes-dashboard-ldap/raw/master/assets/video/intro.webm
git address: https://github.com/icyxp/kube...