Multi cluster k8s dashboard ldap unified login and authorization tool

Origin of tools

Why write such a tool? This is because our company has multiple kubernetes clusters (8 +), and cloud hosting services can't access the apserver configuration, which brings us a pain point. Development and sre need to log in to k8s dashbao and different departments and roles need different authorizations, which used to be through sa token Log in to the dashboard, but with the growth of k8s cluster, the user needs to be informed of the corresponding dashboard access address and the corresponding token for each additional cluster, which is very painful for both the provider and the user. Is there a tool that can provide a unified address to log on to the multi cluster dashboard? After a search, we found that there is no one. Most of the schemes on the market are single cluster integrated LDAP, mainly based on DEX. But the unified login and authorization scheme of single cluster makes people feel very difficult. Is there no simple and convenient tool for us to use? Well, I'll build such a tool.

Dashboard LDAP integration scheme:

The above two documents are the solution of LDAP. I feel that they are good for people in need!

How to build

Well, since there isn't one, make one automatically!

Goal: easy to use!!! By accessing the same address, log in with LDAP and switch between dashboard s of different clusters. At the same time, corresponding permissions of different clusters can be configured separately!

With the above goals, how to achieve them?

In fact, the implementation method is very simple. First, write a login interface to get through with the company's AD to get users and groups, then associate users or groups with the service account in the k8s cluster to achieve the corresponding rbac and login token, and finally implement a reverse proxy service after login.

Is it very simple!!!

Implementation technology stack: golang(gin, client go, viper, ldap) + Kubernetes Dashboard

How to deploy

prerequisite

Before using this tool, you need to have the following constraints:

  1. dashboard has been deployed in k8s clusters and can be accessed by this tool
  2. There is ldap and management permission to access
  3. There are corresponding service account s in each cluster that can be mapped. If different users and groups need different operation permissions, rbac authorization can be performed on sa, which will be explained in detail below.
  4. This tool needs to operate the api of each cluster, so it needs to obtain the apiserver address of each cluster ca.crt And token configuration. For each cluster ca.crt And token, if obtained, will be described later

How to get ca.crt And token

This tool needs to operate the api of each cluster to obtain the corresponding sa and token, so it needs to have the authority to operate each cluster. How to generate corresponding ca certificates and tokens in each cluster? The answer is to create a sa and give it permission.

Execute the following yaml file in each k8s cluster:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: mutiboard-ldap
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: mutiboard-ldap-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: mutiboard-ldap-view
subjects:
- kind: ServiceAccount
  name: mutiboard-ldap
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: mutiboard-ldap-view
rules:
- apiGroups:
  - ""
  resources:
  - serviceaccounts
  - secrets
  verbs:
  - get
  - list
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
  - list
  - watch

What this yaml file means: create a sa named mutiboard LDAP, and give serviceaccounts and secrets get and list permissions.

Get the ca.crt :

(⎈|aws-local:default)❯ echo $(kubectl get secret $(kubectl get secret | grep mutiboard-ldap | awk '{print $1}') -o go-template='{{index .data "ca.crt"}}') | base64 -d

-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTE5MDEyNTEwMTgzNFoXDTI5MDEyMjEwMTgzNFowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMmc
TW0stLLP+M6Pc9wpRgZufg6eQ7puBfbYgik20QlO4LFtocgNUDa0y+aSXjxheA2C
A+o9wW0IC3GHQHKgeFY8KXIJu6wM0TO+JNQy5XZAWfbsLeXU/sLhKuWET/KJzVWT
0uBE+GCADAAQIec1oQXMbQ551hU5gBFcr67NXHpa2qwEGA1mGtZ7ztmW4+IFUD74
G166z4AOgmR4YWxBs/+8NhfWudFD32xevBfSKuHRxRGG5dtffY8QnRbnrmy70HE5
yzLtBvAGfCwtHLTP2ngCAnn2Fb6IeMdIYGpI1544ZjRbzT1YIWsG1v3dlu6tvK1q
X5Pj+UTDmJuf2SW52A0CAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAKE2hV0DIG8fSf4/eOi5R2sPRfBW
qTwgZDDT9dxZNhbxEInALdruwRUbKRpwaUBOGVpIlaK3/rZkAfjUwoDJ+J4fmmCX
w3ySrYFjx6tqVFqCPjDkBHh4xpMwUlvsvryRuCEQUQgjqBvj6sWm9GERF2n3VYBF
S8bjsQQAZJoE4W+OKchlEoSFlKhxAoeZx9CD3Rxnhj2og6doVoGCUqAMh4WZWX+w
pENnui6M96SysH3SkrA02RXWTGeKzK4E6Av3IG+2a2hauHorbqVfaM6HeL3hkU/B
JCWpOgN3T4Fw7E359CBQxnSHPasmZ5VBoyIk/HUU6ZlMK6Xo6JlbS7ZvVl4=
-----END CERTIFICATE-----

Get the token of mutiboard LDAP:

(⎈|aws-local:default)❯ echo $(kubectl get secret $(kubectl get secret | grep mutiboard-ldap | awk '{print $1}') -o go-template='{{.data.token}}') | base64 -d

eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Im11dGlib2FyZC1sZGFwLXRva2VuLWJ3NWdmIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6Im11dGlib2FyZC1sZGFwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMjVmZjI4MGQtYWJhMi0xMWVhLWFlOGEtMDIzOTBjMzcyNzhlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6bXV0aWJvYXJkLWxkYXAifQ.q14hqEu2p70_YczDviR6c8McDM5vfnKPzjO9usCsC-uQUxciBbuJU_PK9j3uawppUNlrs3rAPrZIGUS7Jv14rifEXpGxIIfGR6n8-le0b-9YvMZCgs9-jhf-1r01EAnZFh6gcXfxESFguFQI0vYOsX4P2LQvZ9XTMzsqXbW3KGYao5elAjCE4e8Rg4--9e_zU8NGTEycsvUMxP-9p0SaAzn9Iak3saZtAnzJq5hkSf1t7l2_CgEsYN-3b7uGpHupK_zdgAeOflj9ze4Cz2YScv5eixwVXJ-RcI4lgSFCgt5yzSbnIuHgxRZyN3NcYLrSBYKftezZysWm3jELgLPogQ

So far, the ca.crt Both token and token have been obtained, and we will tell you how to configure and use these ca.crt And token

SA RBAC column

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ops-admin
  namespace: default
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ops-role
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: ops-listnamespace
roleRef: #Referenced roles
  kind: ClusterRole
  name: ops-role
  apiGroup: rbac.authorization.k8s.io
subjects: #subject
- kind: ServiceAccount
  name: ops-admin
  namespace: default
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: ops-ci-admin
  namespace: ops-ci
roleRef: #Referenced roles
  kind: ClusterRole
  name: admin
  apiGroup: rbac.authorization.k8s.io
subjects: #subject
- kind: ServiceAccount
  name: ops-admin
  namespace: default
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: ops-qa-admin
  namespace: ops-qa
roleRef: #Referenced roles
  kind: ClusterRole
  name: admin
  apiGroup: rbac.authorization.k8s.io
subjects: #subject
- kind: ServiceAccount
  name: ops-admin
  namespace: default

This YAML means to create an OPS admin sa and give the sa two namespace (OPS Ci, OPS QA) admin permissions.

To learn more about rbac, please refer to: https://www.cnblogs.com/wlbl/p/10694364.html

ldap description

Our ldap directory rules are as follows:

|--Domain
 |--|---Company
 |--|----|----Branch
 |--|----|-----|----Department
 |--|----|-----|-----|-----Users

The corresponding Distinguished Name is shown as follows:

CN=Peng Xu,OU = Department, OU = branch, OU = company, DC=corp,DC=xxx,DC=com

Here I will get the first OU as a group. If your requirements are different from mine, you can propose issue for me to adapt

For details of ldap, please refer to: https://blog.poychang.net/ldap-introduction

configmap.yaml Configuration description

ldap:
  addr: ldap://192.168.3.81:389
  adminUser: xxxxx
  adminPwd: xxxxxx
  baseDN: dc=corp,dc=patsnap,dc=com
  filter: (&(objectClass=person)(sAMAccountName=%s))
  attributes: user_dn
  orgUnitName: OU=
#Mapping of global users / user groups to SA
rbac:
  DevOps team:
    sa: ops-admin
    ns: kube-system
  xupeng:
    sa: inno-admin
    ns: default
clusters:
  #Cluster alias, the key displayed in the login drop-down box. This alias needs to match the secret.sh In ca.crt One to one correspondence with the key name of token
  local:
    #apiserver address, which can be accessed by the current tool
    apiServer: apiserver-dev.jiunile.com
    port: 6443
    #kubernetes dashboard address, which can be accessed by the current tool
    dashboard: dashboard-dev.jiunile.com
    #Cluster description, the name displayed in the login drop-down box
    desc: Dev Cluster
    #Breakdown for individual clusters
    #rbac:
    #  DevOps team:
    #    sa: admin
    #    ns: kube-system
    #  xupeng:
    #    sa: ops-admin
    #    ns: default
  cnrelease:
    apiServer: apiserver-cn-release.jiunile.com
    port: 443
    dashboard: dashboard-cn-release.jiunile.com
    desc: CN Release Cluster
  usrelease:
    apiServer: apiserver-us-release.jiunile.com
    port: 443
    dashboard: dashboard-us-release.jiunile.com
    desc: US Release Cluster
  euprod:
    apiServer: apiserver-eu-prod.jiunile.com
    port: 443
    dashboard: dashboard-eu-prod.jiunile.com
    desc: EU Prod Cluster

deploy

  1. Modify and deploy deploy/configmap.yaml
  2. Get the ca.crt And token are written to the corresponding deploy/token
  3. Execute the secret.sh Script sh deploy/secret.sh

    be careful: secret.sh xx in_ token/xxx_ ca.crt xx in corresponds to configmap.yaml Cluster aliases in must correspond one by one

  4. deploy/deployment.yaml

visit

http://{nodeip}:31000

Video download address: https://github.com/icyxp/kubernetes-dashboard-ldap/raw/master/assets/video/intro.webm

git address: https://github.com/icyxp/kube...

Tags: Go Kubernetes AWS github git

Posted on Fri, 12 Jun 2020 01:04:46 -0400 by mistjulia