An in-depth analysis of TekTon, an open source CI/CD framework

brief introduction

Tekton is a powerful and flexible Kubernetes native open source framework for creating continuous integration and delivery (CI/CD) systems. As for Tekton, there are many introduction documents available on the Internet. This paper mainly describes my understanding of Tekton's implementation principle and the technical logic behind it.
tekton.dev

Tekton defines five core objects: Task, taskrun, pipeline, pipelinerun, and pipelineresource. By abstracting tasks and pipelines, Tekton can define any combination of pipeline templates to complete various CICD tasks. Through taskrun, pipelinerun and pipelineresource, these templates can be applied to various actual projects.

Implementation principle

The highly abstract structural design makes Tekton very flexible. So how does Tekton realize the flow of workflow.

Tekton uses Kubernetes' list watch mechanism to initialize two controllers, PipelineRunController and TaskRunController at startup.

The PipelineRunController listens for changes in the PipelineRun object. In its reconcile logic, all tasks in the pipeline are constructed into a directed acyclic graph (DAG), and the corresponding Task run object is created by traversing the DAG to find the currently schedulable Task node.

TaskRunController listens for changes in taskrun objects. In its reconcile logic, Task run and corresponding Task are transformed into executable pod, which is scheduled and executed by kubernetes. Using kubernetes' OwnerReference mechanism, pipelinerun own taskrun, taskrun own pod. When the pod state changes, it triggers the reconcile logic of taskrun, and when the taskrun state changes, it triggers the reconcile logic of pipelinerun.

DAG support

Tekton's support for DAG is relatively simple. In Tekton, a Pipeline is a DAG, and multiple tasks in the Pipeline are nodes in the DAG. The Task is executed concurrently by default. You can use the RunAfter And From Key controls the execution order.

Example:

- name: lint-repo
  taskRef:
    name: pylint
  resources:
    inputs:
      - name: workspace
        resource: my-repo
- name: test-app
  taskRef:
    name: make-test
  resources:
    inputs:
      - name: workspace
        resource: my-repo
- name: build-app
  taskRef:
    name: kaniko-build-app
  runAfter:
    - test-app
  resources:
    inputs:
      - name: workspace
        resource: my-repo
    outputs:
      - name: image
        resource: my-app-image
- name: build-frontend
  taskRef:
    name: kaniko-build-frontend
  runAfter:
    - test-app
  resources:
    inputs:
      - name: workspace
        resource: my-repo
    outputs:
      - name: image
        resource: my-frontend-image
- name: deploy-all
  taskRef:
    name: deploy-kubectl
  resources:
    inputs:
      - name: my-app-image
        resource: my-app-image
        from:
          - build-app
      - name: my-frontend-image
        resource: my-frontend-image
        from:
          - build-frontend

The rendered execution order is:

        |            |
        v            v
     test-app    lint-repo
    /        \
   v          v
build-app  build-frontend
   \          /
    v        v
    deploy-all

Compared with Argo and other projects focusing on workflow, Tekton supports very limited task choreography. Common strategies such as loop, recursion, Retry, timeout and so on are not available.

  • Condition judgment

Tekton support condition Keyword to judge the condition. Condtion only supports judging whether the current Task is executed or not. It cannot be used as a branch condition of DAG to render dynamic DAG.

*The condition check fails (exitcode! = 0), the task will not be executed, and the pipelineRun status will not fail because the condition check fails.
*And logical relationship among multiple conditions

PipelineResource data exchange between tasks

As a tool of CICD, when and how is the code implemented in WorkSpace? In Tekton, PipelineResource is abstracted for data exchange between tasks, among which GitResource is the most basic one. The usage is as follows.

  • Declare a PipelineResource of type Git:
kind: PipelineResource
metadata:
  name: skaffold-git-build-push-kaniko
spec:
  type: git
  params:
  - name: revision
    value: v0.32.0
  - name: url
    value: https://github.com/GoogleContainerTools/skaffold
  • Refer to this Resource as input in the Task:
kind: Task
metadata:
  name: build-push-kaniko
spec:
  inputs:
    resources:
    - name: workspace
      type: git
  steps:
  - name: build-and-push
    image: registry.cn-shanghai.aliyuncs.com/kaniko-project-edas/executor:v0.17.1
  • The code will be in the / workspace directory by clone.

How does Tekton handle these pipelineresources starts with how Taskrun Controller creates Pod.

A task run in Tekton corresponds to a Pod. Each Pod is composed of a series of init containers and step containers. Initialization of authentication information and workspace directory is completed in init container.

When processing a step container, the corresponding output and output will be processed according to the resource Append or Insert a step container referenced by the Task, as shown in the following figure.

Step execution sequence control in Task

Tekton comes from the known build. Init container is used to concatenate Steps in the known build to ensure the sequential execution of Steps. In the above analysis, we know that Tekton uses Containers to execute Steps and Pod Containers are executed in parallel. How does Tekton ensure the sequential execution of Steps?

This is part of the description information of a Pod created by TaskRun. You can see that all steps are encapsulated and executed by / tekton/tools/entrypoints. -wait_file specifies a file. By listening to the file handle, the encapsulated Step task is executed when the existence of the file is detected. -post_file specifies a file to be created after the Step task is completed. Sort steps by file sequence / tekton/tools/${index}.

- args:
    - -wait_file
    - /tekton/tools/0
    - -post_file
    - /tekton/tools/1
    - -termination_path
    - /tekton/termination
    - -entrypoint
    - /ko-app/git-init
    - --
    - -url
    - https://github.com/GoogleContainerTools/skaffold
    - -revision
    - v0.32.0
    - -path
    - /workspace/workspace
    command:
    - /tekton/tools/entrypoint
    image: registry.cn-shanghai.aliyuncs.com/kaniko-project-edas/git-init:v0.10.2
    name: step-git-source-skaffold-git-build-push-kaniko-rz765
  - args:
    - -wait_file
    - /tekton/tools/1
    - -post_file
    - /tekton/tools/2
    - -termination_path
    - /tekton/termination
    - -entrypoint
    - /kaniko/executor
    - --
    - --dockerfile=Dockerfile
    - --destination=localhost:5000/leeroy-web
    - --context=/workspace/workspace/examples/microservices/leeroy-web
    - --oci-layout-path=$(inputs.resources.builtImage.path)
    command:
    - /tekton/tools/entrypoint
    image: registry.cn-shanghai.aliyuncs.com/kaniko-project-edas/executor@sha256:565d31516f9bb91763dcf8e23ee161144fd4e27624b257674136c71559ce4493
    name: step-build-and-push
  - args:
    - -wait_file
    - /tekton/tools/2
    - -post_file
    - /tekton/tools/3
    - -termination_path
    - /tekton/termination
    - -entrypoint
    - /ko-app/imagedigestexporter
    - --
    - -images
    - '[{"name":"skaffold-image-leeroy-web-build-push-kaniko","type":"image","url":"localhost:5000/leeroy-web","digest":"","OutputImageDir":"/workspace/output/builtImage"}]'
    command:
    - /tekton/tools/entrypoint
    image: registry.cn-shanghai.aliyuncs.com/kaniko-project-edas/imagedigestexporter:v0.10.2
    name: step-image-digest-exporter-lvlj9
    

practice

Build code and deploy to SAE using Tekton

Serverless application engine ( SAE ) is an application-oriented Serverless PaaS platform on Alibaba cloud, which helps PaaS layer users to operate free IaaS, use it on demand and charge by volume, realize low threshold micro service application on cloud, and effectively solve cost and efficiency problems. It supports popular development frameworks such as Spring Cloud, Dubbo and HSF, and truly realizes the perfect integration of Serverless architecture and microservice architecture.

Next, we will use Tekton to deploy a Spring Cloud microservice to the SAE platform.

The demo code address in the example: https://github.com/alicloud-demo/spring-cloud-demo
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: spring-cloud-demo
spec:
  type: git
  params:
  - name: url
    value: https://github.com/alicloud-demo/spring-cloud-demo
  • Define build and deploy tasks

According to SAE Official documents Deploy.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: build-deploy-sae
spec:
  inputs:
    resources:
    - name: source
      type: git
  steps:
  - name: build-and-deploy
    image: maven:3.3-jdk-8
    command: ["mvn", "clean", "package", "-f", "source", "toolkit:deploy", "-Dtoolkit_profile=toolkit_profile.yaml", "-Dtoolkit_package=toolkit_package.yaml", "-Dtoolkit_deploy=toolkit_deploy.yaml"]
    securityContext:
      runAsUser: 0
  • Define TaskRun to run tasks
apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: build-deploy-sae
spec:
  taskRef:
    name: build-deploy-sae
  inputs:
    resources:
    - name: source
      resourceRef:
        name: spring-cloud-demo
  • Import to kubernetes to run
kubectl apply -f source-2-service-taskrun.yaml

  • view log
kubectl logs build-deploy-sae-pod-85xdk step-build-and-deploy

Build log:

Deployment log:

[INFO] Start to upload [provider3-1.0-SNAPSHOT.jar] using [Sae uploader].
[INFO] [##################################################] 100.0%
[INFO] Upload finished in 3341 ms, download url: [https://edas-hz.oss-cn-hangzhou.aliyuncs.com/apps/K8S_APP_ID/37adb12b-5f0c-4711-98ec-1f1e91e6b043/provider3-1.0-SNAPSHOT.jar]
[INFO] Begin to trace change order: e2499b9a-6a51-4904-819c-1838c1dd62cb
[INFO] PipelineName: Batch: 1, PipelineId:f029314a-88bb-450b-aa35-7cc550ff1329
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Waiting...
[INFO] Deploy application successfully!
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 32:41 min
[INFO] Finished at: 2020-04-15T10:09:39+00:00
[INFO] Final Memory: 47M/190M
[INFO] ------------------------------------------------------------------------
  • Verify deployment results

To view the change record at the SAE console:

Verify app access:

summary

Different from the traditional CICD tools (Jenkins), Tekton is a set of framework to build CICD system. Tekton can't give you immediate access to CICD. But based on Tekton, we can design a variety of fancy construction and deployment pipelines. Thanks to Tekton's good abstraction, these pipelines can be used as templates for sharing among multiple organizations and projects. Tekton comes from the build template project of knight. At the beginning of the design, an important goal is to enable people to share and reuse the components that make up the pipeline, as well as the pipeline itself. Tekton Catelog is proposed to achieve this goal in Tekton's RoadMap.

Unlike Argo, a Kubernetes based Workflow tool, Tekton has weak support for Workflow control. Some complex scenarios such as loops and recursion are not supported. Not to mention Argo's performance optimization under high concurrency and large cluster scheduling. This is related to Tekton's positioning. Tekton is positioned to realize the framework of CICD, which does not need too complex process control for CICD. Most R & D processes can be covered by several best practices. These best practices should and must be shared among different organizations, so Tekton designed the concept of pipeline resource. PipelineResource is an interface between tasks and a component shared and reused across platforms and organizations. There are still many imaginations on PipelineResource.

Author information: Jiubian, senior development engineer of Alibaba, is responsible for the research and development of Alibaba cloud EDAs (enterprise level distributed application service) application life cycle, and has long paid attention to the deployment and governance of microservices in the cloud era.

Tags: Mobile git Kubernetes Spring github

Posted on Tue, 19 May 2020 03:14:24 -0400 by verbalkint81