I. environmental preparation
- K8s cluster environment (here I am k8s cluster of three sets);
- A single docker server is mainly used to upload images to the private warehouse. Jenkins and gitlab are also deployed in this server;
- There are 4 servers in the above environment, all of which point to the same private warehouse for sharing docker image;
- Server IP is 192.168.20.2, 20.3, 20.4, 20.5 (the first three IP are nodes in K8s cluster)
Jenkins is deployed in the way of war package. It needs to use tomcat environment and refer to the blog for deployment;
For other environment deployment, please refer to the following blog:
Tomcat installation and optimized configuration；
Detailed configuration of Docker installation；
Introduction and installation of K8s (Kubernetes).
Note: all package files used below can be found in my network disk link download.
II. Deployment of registry private warehouse
Any node that can run the docker container can be deployed. Here I choose 192.168.20.5.
1. Run private warehouse container
[root@jenkins ~]# docker run -tid --name registry --restart=always -p 5000:5000 -v /data/registry:/var/lib/registry registry
2. Configure each server to point to the private warehouse
#Select any node to do the following [root@jenkins ~]# vim /usr/lib/systemd/system/docker.service #Fix the following configuration item to specify the listening address of its private warehouse ExecStart=/usr/bin/dockerd -H unix:// --insecure-registry 192.168.20.5:5000 #Send the modified profile to other nodes [root@jenkins ~]# scp /usr/lib/systemd/system/docker.service 192.168.20.2:/usr/lib/systemd/system/ [root@jenkins ~]# scp /usr/lib/systemd/system/docker.service 192.168.20.3:/usr/lib/systemd/system/ [root@jenkins ~]# scp /usr/lib/systemd/system/docker.service 192.168.20.4:/usr/lib/systemd/system/
3. Restart the docker service for each node to make the changes take effect
The following operations need to be configured at each node in turn.
[root@jenkins ~]# systemctl daemon-reload [root@jenkins ~]# systemctl restart docker
III. deploy Jenkins service and start it
Before deploying Jenkins, deploy tomcat service by yourself. Refer to the blog: Tomcat installation and optimized configuration.
The source package and plug-ins needed for the deployment of Jenkins can be found in my online disk link download.
[root@jenkins src]# cd /usr/local/tomcat/webapps/ #Delete all contents in the original web page directory (based on the actual situation, it's better to make a backup) [root@jenkins webapps]# rm -rf * [root@jenkins webapps]# rz #Upload Jenkins' war package [root@jenkins webapps]# ls jenkins.war #Specify Jenkins' home directory [root@jenkins webapps]# vim ../bin/catalina.sh #Edit this file #!/bin/sh #Just add it at the beginning of the file export CATALINA_OPTS="-DJENKINS_HOME=/data/jenkins" export JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Dhudson.ClassicPluginStrategy.noBytecodeTransformer=true" #Start tomcat [root@jenkins webapps]# cd ../bin/ [root@jenkins bin]# ./catalina.sh start #Make sure the port is listening [root@jenkins bin]# netstat -anpt | grep 8080 tcp6 0 0 :::8080 :::* LISTEN 58560/java
IV. configure Jenkins service
In order to realize the CI/CD of Jenkins, there must be three plug-ins: GitLab, GitLab HOOK and GitLab authentication. It is not easy to download these plug-ins in China. There are packaged plug-ins in the online disk link above me. Just unzip them to the corresponding directory.
1. Configure Jenkins web interface
When the browser accesses the IP + port / Jenkins of the following tomcat services, such as 192.168.20.5:8080/jenkins, it can see the following interface (it is better to disconnect the external network before accessing, otherwise, after entering the password, the plug-in will be installed online, which is slow and likely to fail):
Check the password according to the prompts and paste it to the corresponding location, as follows, check the password:
[root@jenkins webapps]# cat /data/jenkins/secrets/initialAdminPassword ed0a933859e0470f9095680a2059c19a
After entering the password, wait for a moment, and then click as follows:
Create corresponding user:
After successful login, it is as follows:
Now close the browser page and add a plug-in to Jenkins:
2. Configure the required plug-ins
[root@jenkins src]# rm -rf /data/jenkins/plugins/ #Delete the original empty directory [root@jenkins src]# tar zxf plugins.tar.gz -C /data/jenkins/ #Restart tomcat to take effect [root@jenkins src]# /usr/local/tomcat/bin/catalina.sh stop [root@jenkins src]# /usr/local/tomcat/bin/catalina.sh start
Visit Jenkins's web interface again:
You can see that it has become a Chinese page, indicating that the plug-in configuration is effective, as follows:
V. deployment of gitlab services
After thinking about it, I still don't want to take a screenshot to write it. Please refer to my previous blog to deploy gitlab. I also watched that blog deployment (don't operate after installing gitlab, you need to see the following precautions, change the monitoring port), and finally make sure that you can clone the remote library to the local. Here I deploy gitlab and Jenkins on the same server. Reference blog: Gitlab installation and application of continuous integration
matters needing attention!!!
After installing gitlab, you need to change its listening port (to prevent port conflicts), as follows:
[root@jenkins src]# vim /etc/gitlab/gitlab.rb #The changes are as follows external_url 'http://192.168.20.5:90' unicorn['listen'] = '192.168.20.5' #Native IP unicorn['port'] = 3000
After you change the listening port, you can follow that blog for the next operation (but the port is different from that blog, for example, to query whether the port is listening again, you should see port 90 instead of port 80. To visit the browser page, you need to add port 90).
Finally, ensure that there is a local clone library, as follows:
[root@jenkins ~]# ls -d teset01/ teset01/
Vi. configure Jenkins to create a task
Paste the following shell script content into the web interface, as follows:
#In the script, 192.168.20.5 is the IP address of the private warehouse, and 192.168.20.2 is the IP address of the master in the k8s cluster #!/bin/bash backupcode="/data/backcode/$JOB_NAME/$BUILD_NUMBER" #The default variables of Jenkins are referenced here mkdir -p $backupcode chmod 644 "$JENKINS_HOME"/workspace/"$JOB_NAME"/* rsync -acP "$JENKINS_HOME"/workspace/"$JOB_NAME"/* $backupcode echo From 192.168.20.5:5000/nginx > "$JENKINS_HOME"/workspace/Dockerfile echo COPY ./"$JOB_NAME"/* /usr/share/nginx/html/ >> "$JENKINS_HOME"/workspace/Dockerfile docker rmi 192.168.20.5:5000/nginx docker build -t 192.168.20.5:5000/nginx /"$JENKINS_HOME"/workspace/. docker push 192.168.20.5:5000/nginx ssh [email protected] kubectl delete deployment nginx ssh [email protected] kubectl apply -f /root/nginx.yaml
After the script is filled in, it is as follows:
After the script is filled in, do not save it first. Copy the Jenkins address as follows:
7. Enable anonymous access of Jenkins
VIII. Go back to gitlab and enable sending web hook to yourself
The following operations are performed in the new database. I forgot to cut the diagram of the step of switching to the new database:
After saving, pull down the page to see the newly added web hook. Click the following to test:
If status code 200 is returned, the configuration is correct, as follows:
IX. configure the master node of the Jenkins password free login k8s cluster
The jenkins server is configured as follows:
[root@jenkins ~]# ssh-copy-id [email protected] #The IP above is the master node IP in the K8s cluster
IX. test continuous CI/CD effect
1. Running nginx resource object in k8s cluster
#Upload the required image to the private warehouse [root@jenkins ~]# docker tag nginx:latest 192.168.20.5:5000/nginx:latest [root@jenkins ~]# docker push 192.168.20.5:5000/nginx:latest #Running nginx resource object on the master node [root@master ~]# vim nginx.yaml #Write yaml file apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx spec: replicas: 2 template: metadata: labels: name: nginx spec: containers: - name: nginx image: 192.168.20.5:5000/nginx:latest ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: labels: name: nginx name: nginx spec: type: NodePort ports: - port: 80 targetPort: 80 nodePort: 31234 selector: name: nginx [root@master ~]# kubectl apply -f nginx.yaml #Execute yaml file [root@master ~]# kubectl get pod | grep nginx #Make sure the container is working properly nginx-76645bc84f-rbgtl 1/1 Running 0 68s nginx-76645bc84f-s6xp6 1/1 Running 0 68s
2. Client access nginx
3. Update and iterate test the version in gitlab
#Upload files to gitlab in the local library [root@jenkins ~]# cd teset01/ [root@jenkins teset01]# git config --global user.name "test" [root@jenkins teset01]# git config --global user.email "[email protected]" [root@jenkins teset01]# echo "test ....." > index.html [root@jenkins teset01]# git add * [root@jenkins teset01]# git commit -m "Test CI/CD" [root@jenkins teset01]# git push origin master
After the above operations, you can see the successful construction information in Jenkins's new task, as follows:
Visit the homepage of nginx again and find that it has become the content we submitted on gitlab, as follows:
Test complete.