On Kubernetes Cluster Deployment

Summary

There are several ways to deploy K8S clusters: kubeadm, minikube, and binary packages.The first two are automatic deployments that simplify deployment operations.In a build environment, binary package deployment is commonly used. Here's how to deploy the Kubernetes cluster using binary packages.

Architecture master plan

Environmental preparation

Three hosts, one as master node and two as node node.

Where the master node needs to install the following components
kube-apiserver,kube-controller-manager,kube-scheduler,etcd
Node node requires the following components to be installed
kubelet,kube-proxy,docker,flannel,etcd

Component introduction

Flanner: Flannel is a network planning service designed for Kubernetes. Simply put, it allows Docker containers created by different node hosts in a cluster to have a unique virtual IP address for the entire cluster.However, in the default Docker configuration, each Node's Docker service is responsible for the IP allocation of its node container.Containers inside Node can access each other, but cross-host (Node) networks cannot communicate with each other.Flannel is designed to reprogram the rules for the use of IP addresses for all nodes in a cluster so that containers on different nodes can get "one intranet" and "no duplicate" IP addresses and allow containers on different nodes to communicate directly over intranet IP.

Deployment process

Download binary packages from the official website

Deploy etcd storage

etcd stored in master node and node need to be deployed, first deployed in master node

Edit script to download official cfssl package

vim cfssl.sh
curl -L https:#pkg.cfssl.org/ R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https:#pkg.cfssl.org/ R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https:#pkg.cfssl.org/ R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo

chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo

Execute script

bash cfssl.sh

At this point, the cd/usr/local/bin/directory generates three files

cfssl: is the certificate generation tool
cfssljson: Generate a certificate by passing in a json file
cfssl-certinfo: is to view certificate information

Add execute permissions to these three files

chmod 777 cfssl cfssl-certinfo cfssljson

Define a ca certificate, generate a certificate

Build two scripts to generate certificates
First script vim etcd-cert.sh
Define a ca certificate

Certificate Signature

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

The above command generates the required certificate

The second script vim etcd.sh


Script configuration completed, reloaded, etcd turned on and set boot self-start, firewall turned off, enhanced security features

systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd
systemctl stop firewalld.service
setenforce 0

Create the k8s directory and move the two scripts to it

mkdir k8s
cd k8s/
mkdir etcd-cert
mv etcd-cert.sh etcd-cert

Unzip ETCD Binary Package

tar zxvf etcd-v3.3.10-linux-amd64.tar.gz

Configuration File, Command File, Certificate

mkdir /opt/etcd/{cfg,bin,ssl} -p
mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/

Certificate Copy

cp etcd-cert/.pem /opt/etcd/ssl/

Enter a stuck state and wait for other nodes to join

bash etcd.sh etcd01 192.168.142.129 etcd02=https:/ /192.168.142.130:2380,etcd03=https:/ /192.168.149.131:2380

Open with another session and you will find that the etcd process is already started
ps -ef | grep etcd

Copy Certificate to Other Nodes

scp -r /opt/etcd/ root@192.168.142.130:/opt/
scp -r /opt/etcd/ root@192.168.142.131:/opt/

Start script to copy other nodes

scp /usr/lib/systemd/system/etcd.service root@192.168.142.130:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service root@192.168.142.131:/usr/lib/systemd/system/

Operation on Node 01

Modify etcd file

vim /opt/etcd/cfg/etcd

Modify Name and Address

[Member]
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https:/ /192.168.142.130:2380"
ETCD_LISTEN_CLIENT_URLS="https:/ /192.168.142.130:2379"

Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https:/ /192.168.142.130:2380"
ETCD_ADVERTISE_CLIENT_URLS="https:/ /192.168.142.130:2379"
ETCD_INITIAL_CLUSTER="etcd01=http//192.168.142.129:2380,etcd02=https:/ /192.168.14.130:2380,etcd03=https:/ /192.168.142.131:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

Start Services

systemctl start etcd
systemctl status etcd

3. Operation on Node 02

Modify etcd file

vim /opt/etcd/cfg/etcd

Modify Name and Address

[Member]
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https:#192.168.142.131:2380"
ETCD_LISTEN_CLIENT_URLS="https:#192.168.142.131:2379"

[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https:#192.168.142.131:2380"
ETCD_ADVERTISE_CLIENT_URLS="https:#192.168.142.131:2379"
ETCD_INITIAL_CLUSTER="etcd01=https:#192.168.142.129:2380,etcd02=https:#192.168.142.130:2380,etcd03=https:#192.168.142.131:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

Start Services

systemctl start etcd

Deploy flannel

Write allocated subnet segments to ETCD for flannel use

/opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https:/ /192.168.149.131:2379,h ttps://192.168.220.140:2379,https:/ /192.168.220.136:2379 set /core os.com /network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'

Copy the package to all node nodes

scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.220.140:/root
scp flannel-v0.10.0-linux-amd64.tar.gz root@192.168.220.136:/root

All node s uncompress the package:

tar zvxf flannel-v0.10.0-linux-amd64.tar.gz

On the node, create the k8s working directory first:

mkdir /opt/kubernetes/{cfg,bin,ssl} -p
mv mk-docker-opts.sh flanneld /opt/kubernetes/bin/*

Create flanner script vim flannel.sh


Start and Set Open Self-Start

systemctl daemon-reload
systemctl enable flanneld
systemctl restart flanneld

Start flanner network function

bash flannel.sh https:// 19 2.1 68.220.131:2379,https:/ /192.168.220.140:2379,http s://192.168.220.136:2379

at /run/flannel/subnet.env
DOCKER_OPT_BIP="--bip=172.17.53.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=false"
DOCKER_OPT_MTU="--mtu=1450"

DOCKER_NETWORK_OPTIONS=" --bip=172.17.53.1/24 --ip-masq=false --mtu=1450"

Restart docker

systemctl daemon-reload
systemctl restart docker

View flanner network

Tags: Linux Docker vim network

Posted on Mon, 10 Feb 2020 11:33:11 -0500 by mrfritz379