Build
CloudNative CI/CD Pipelines
with Tekton on Kubernetes
Nikhil Thomas
Software Engineer
Red Hat
Github: nikhil-thomas
Twitter: nikhilthomasmwb
YouTube link to this talk: https://www.youtube.com/watch?v=-ji5Z0qJmJs
Contents
● Meet our CI/CD Pipeline
● Introduction to Tekton Pipeline
● Pipelines Demo
Sample App
Web Service 1 Web Service 2
http request
http response
http request
http response
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
● Kubernetes Style Resources ⇔ Tekton Custom Resources
● Declaring CI/CD Pipelines ⇔ Tekton Controllers Manages State
“Kubernetes resource model
is coming
to an API near you!”
- Kubernetes-style APIs of the Future, Daniel Smith
KubeCon CloudNativeCon Europe, May 2018
Kubernetes Resource Model
API Operation State
Handles
API Operations
Resources
● Pods
● Deployments
● Service
● Node
● ...
Verbs
● Get
● List
● Create
● Update
● Delete
● Patch
● Watch
● Pods
● Deployments
● Service
● Node
● …
● Custom
Resources
● Get
● List
● Create
● Update
● Delete
● Patch
● Watch
API Operations
Resources Verbs
State
Controllers
● Split System State into
small pieces
● Separate Controllers
manage each piece
Resource StateController
Events
● Split System State into
small pieces
● Separate Controllers
manage each piece
Resource StateController
Events
Yes!, we can add custom controllers :)
State
Controllers
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
Tekton Pipeline Custom Resources
● Task
● PipelineResource
● Pipeline
● TaskRun
● PipelineRun
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
Task
● Defines the work that needs to be executed
● Every task runs as a Pod
● Contains multiple steps
Steps (in a Task)
● Series of commands
● Sequentially executed by the task.
● Each step in a task runs as a container
Task
step
step
step
Task Input and Outputs
● Input
● PipelineResources
● Parameters
● Output
● PipelineResources
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
PipelineResource
● Predefined input/output
● System defined resource types
ready to use
● Decoupled and reusable
● Commonly used PipelineResource
types: image, git
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
Pipeline
● Defines list of Tasks to execute
● Expresses Task order (graph)
● Links Tasks inputs and outputs
Pipeline
Task Task
Task
Task
Task
Pipeline
TaskRun
● Instance of a defined Task
● Binds the inputs/outputs to a Task
● Sets values to the parameters
● Executes Task steps
TaskRun
TaskRun
TaskRun
TaskRun
PipelineRun
● Instance of a defined Pipeline
● Binds inputs/outputs to a Pipeline
● Execute the Tasks in the Pipeline
● Create the TaskRuns for each Task
and collect the results
CI/CD with Tekton Pipeline
on Kubernetes
Demo
Sample App
Web Service 1 Web Service 2
http request
http response
http request
http response
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
Build Task
Pull source code
(Step)
Build container Image
(Step)
Push container image
(Step)
(Task)
source code (PipelineResource)
container image (PipelineResource)
execution order
inputs
Deployment Task
Update Deployment manifest
(Step)
Create Deployment
(Step)
(Task)
container image (PipelineResource)
Functional web services
execution order
inputs
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
source code (PipelineResource) source code (PipelineResource)
execution order
inputs
CI/CD Pipeline
Build Web Service 1
(Task)
Build Web Service 2
(Task)
Deploy Web Service 1
(Task)
Deploy Web Service 2
(Task)
container image (PipelineResource)
container image (PipelineResource)
(Pipeline)
tkn (Tekton CLI)
Available Commands
● task : Manage tasks
● taskrun : Manage taskruns
● pipeline : Manage pipelines
● pipelinerun : Manage pipelineruns
● completion : Prints shell completion scripts
● help : Help about any command
● version : Prints version information
tkn task
Available Sub-Commands
● list : Lists tasks in a namespace
tkn taskrun
Available Sub-Commands
● list : Lists taskruns in a namespace
● logs : Displays taskrun logs
tkn pipeline
Available Sub-Commands
● describe : Describes a pipeline in a namespace
● list : Lists pipelines in a namespace
● start : Start pipelines by creating a pipelinerun in a namespace
tkn pipelinerun
Available Sub-Commands
● describe : Describe a pipelinerun in a namespace
● list : Lists pipelineruns in a namespace
● logs : Display pipelinerun logs
Pipelines Operator
Pipelines Operator provides `As a Service Experience`
● Kubernetes Style Resources ⇔ Tekton Resources ⇔ API operations
● Declaring CI/CD Pipelines ⇔ Tekton Controllers ⇔ State Management
∙ Tekton: https://github.com/tektoncd
∙ Tekton Pipeline: https://github.com/tektoncd/pipeline
∙ Tekton CLI: https://github.com/tektoncd/cli
∙ Kubernetes Operator for Tekton Pipelines:
https://github.com/openshift/tektoncd-pipeline-operator
∙ Tekton Pipelines on Openshift:
https://github.com/openshift/tektoncd-pipeline-operator
∙ Tekton Catalog:
https://github.com/tektoncd/catalog
Thank you

Cloud-Native CI/CD on Kubernetes with Tekton Pipelines

  • 1.
    Build CloudNative CI/CD Pipelines withTekton on Kubernetes Nikhil Thomas Software Engineer Red Hat Github: nikhil-thomas Twitter: nikhilthomasmwb YouTube link to this talk: https://www.youtube.com/watch?v=-ji5Z0qJmJs
  • 2.
    Contents ● Meet ourCI/CD Pipeline ● Introduction to Tekton Pipeline ● Pipelines Demo
  • 3.
    Sample App Web Service1 Web Service 2 http request http response http request http response
  • 4.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 6.
    ● Kubernetes StyleResources ⇔ Tekton Custom Resources ● Declaring CI/CD Pipelines ⇔ Tekton Controllers Manages State
  • 7.
    “Kubernetes resource model iscoming to an API near you!” - Kubernetes-style APIs of the Future, Daniel Smith KubeCon CloudNativeCon Europe, May 2018
  • 8.
    Kubernetes Resource Model APIOperation State Handles
  • 9.
    API Operations Resources ● Pods ●Deployments ● Service ● Node ● ... Verbs ● Get ● List ● Create ● Update ● Delete ● Patch ● Watch
  • 10.
    ● Pods ● Deployments ●Service ● Node ● … ● Custom Resources ● Get ● List ● Create ● Update ● Delete ● Patch ● Watch API Operations Resources Verbs
  • 11.
    State Controllers ● Split SystemState into small pieces ● Separate Controllers manage each piece Resource StateController Events
  • 12.
    ● Split SystemState into small pieces ● Separate Controllers manage each piece Resource StateController Events Yes!, we can add custom controllers :) State Controllers
  • 15.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 16.
    Tekton Pipeline CustomResources ● Task ● PipelineResource ● Pipeline ● TaskRun ● PipelineRun
  • 17.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 18.
    Task ● Defines thework that needs to be executed ● Every task runs as a Pod ● Contains multiple steps
  • 19.
    Steps (in aTask) ● Series of commands ● Sequentially executed by the task. ● Each step in a task runs as a container Task step step step
  • 20.
    Task Input andOutputs ● Input ● PipelineResources ● Parameters ● Output ● PipelineResources
  • 21.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 22.
    PipelineResource ● Predefined input/output ●System defined resource types ready to use ● Decoupled and reusable ● Commonly used PipelineResource types: image, git
  • 23.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 24.
    Pipeline ● Defines listof Tasks to execute ● Expresses Task order (graph) ● Links Tasks inputs and outputs Pipeline Task Task Task Task Task
  • 25.
  • 26.
    TaskRun ● Instance ofa defined Task ● Binds the inputs/outputs to a Task ● Sets values to the parameters ● Executes Task steps
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
    PipelineRun ● Instance ofa defined Pipeline ● Binds inputs/outputs to a Pipeline ● Execute the Tasks in the Pipeline ● Create the TaskRuns for each Task and collect the results
  • 32.
    CI/CD with TektonPipeline on Kubernetes Demo
  • 33.
    Sample App Web Service1 Web Service 2 http request http response http request http response
  • 34.
    CI/CD Pipeline Build WebService 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline) source code (PipelineResource) source code (PipelineResource) execution order inputs
  • 35.
    Build Task Pull sourcecode (Step) Build container Image (Step) Push container image (Step) (Task) source code (PipelineResource) container image (PipelineResource) execution order inputs
  • 36.
    Deployment Task Update Deploymentmanifest (Step) Create Deployment (Step) (Task) container image (PipelineResource) Functional web services execution order inputs
  • 37.
    source code (PipelineResource)source code (PipelineResource) execution order inputs CI/CD Pipeline Build Web Service 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline)
  • 38.
    source code (PipelineResource)source code (PipelineResource) execution order inputs CI/CD Pipeline Build Web Service 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline)
  • 39.
    source code (PipelineResource)source code (PipelineResource) execution order inputs CI/CD Pipeline Build Web Service 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline)
  • 40.
    source code (PipelineResource)source code (PipelineResource) execution order inputs CI/CD Pipeline Build Web Service 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline)
  • 41.
    source code (PipelineResource)source code (PipelineResource) execution order inputs CI/CD Pipeline Build Web Service 1 (Task) Build Web Service 2 (Task) Deploy Web Service 1 (Task) Deploy Web Service 2 (Task) container image (PipelineResource) container image (PipelineResource) (Pipeline)
  • 42.
    tkn (Tekton CLI) AvailableCommands ● task : Manage tasks ● taskrun : Manage taskruns ● pipeline : Manage pipelines ● pipelinerun : Manage pipelineruns ● completion : Prints shell completion scripts ● help : Help about any command ● version : Prints version information
  • 43.
    tkn task Available Sub-Commands ●list : Lists tasks in a namespace tkn taskrun Available Sub-Commands ● list : Lists taskruns in a namespace ● logs : Displays taskrun logs
  • 44.
    tkn pipeline Available Sub-Commands ●describe : Describes a pipeline in a namespace ● list : Lists pipelines in a namespace ● start : Start pipelines by creating a pipelinerun in a namespace tkn pipelinerun Available Sub-Commands ● describe : Describe a pipelinerun in a namespace ● list : Lists pipelineruns in a namespace ● logs : Display pipelinerun logs
  • 45.
  • 46.
    Pipelines Operator provides`As a Service Experience`
  • 47.
    ● Kubernetes StyleResources ⇔ Tekton Resources ⇔ API operations ● Declaring CI/CD Pipelines ⇔ Tekton Controllers ⇔ State Management
  • 48.
    ∙ Tekton: https://github.com/tektoncd ∙Tekton Pipeline: https://github.com/tektoncd/pipeline ∙ Tekton CLI: https://github.com/tektoncd/cli ∙ Kubernetes Operator for Tekton Pipelines: https://github.com/openshift/tektoncd-pipeline-operator ∙ Tekton Pipelines on Openshift: https://github.com/openshift/tektoncd-pipeline-operator ∙ Tekton Catalog: https://github.com/tektoncd/catalog
  • 49.