Download as PDF, PPTX
Uploaded byMichel Schildmeijer
5 steps to take setting up a streamlined container pipeline
The document outlines 5 steps to set up a container pipeline: 1. Use versioning and container registries like GitHub, Docker, and private registries to manage code versions and container images. 2. Use an orchestration engine like Kubernetes to manage and orchestrate container processes. Common options are AWS EKS, GCP GKE, and Oracle OKE. 3. Provision the Kubernetes cluster using scripts or Terraform on cloud infrastructure like OCI. 4. Implement container pipelines using tools like Oracle Container Pipelines to automate building, testing, and deploying containers. 5. Use Helm to package and deploy Kubernetes applications and integrate it into the CI/CD pipeline
5 steps to take setting up a streamlined container pipeline
- 1. Michel Schildmeijer 5 stepsto set up a container pipeline
- 2. Me • From Amsterdam,the Netherlands • Lead Technologist at Qualogy • Oracle ACE since 2012 • Authored 2 books about WebLogic in 2011 • Started with UNIX in 1994 • Started with BEA technology in 2000 • Focus on containerization strategies, DevOps etc.
- 3.
- 4. Investigate increasingerror rate Increasing percentage of bugs Lack of continuous visibility in DevOps teams DevOps Challenges
- 5. Ingredients for aCloud Native Container Pipeline • Versioning & Container Registry • Containers & Orchestration Engine • Provisioning • Container Pipelines • Packaging & Deployments
- 6. Setup a CloudNative Container Pipeline 1 Versioning & Container Registry
- 7. Versioning & ContainerRegistry • Versioning repo • GitHub for Git • Subversion • BitBucket • Container Registries • Docker • Vendor Registries • Private Registries: • OCIR: • Private,HA
- 8. Versioning & ContainerRegistry • Use of Github authentication for Kubernetes • Use the webhook Module kubectl create -f https://raw.githubusercontent.com/oursky/kubernetes-github- authn/master/manifests/github-authn.yaml • K8S Webhook token authenticator for shareable access • Instruct the apiserver to use the webhook config --authentication-token-webhook-config-file --authentication-token-webhook-cache-ttl
- 9. Versioning & ContainerRegistry • Use of Github authentication for Kubernetes • Use the webhook Module • K8S Webhook token authenticator for shareable access • Deploy keys per projects
- 10. Versioning & ContainerRegistry kubectl create secret docker-registry dockerregsecret --docker- server=somewhere.com --docker-username=api.user' --docker- password=*********' --docker-email=‘me@nowhere.com • Registry Secret in Kubernetes
- 11. Containers & OrchestrationEngine 2 Containers & Orchestration Engine
- 12. Orchestration Engine -Kubernetes • Is a Framework for building distributed platforms • Manages & orchestrates container processes (docker) compute networknetwork storage
- 13. Orchestration Engine -Kubernetes • Various Cloud providers have a Kubernetes solution • Microsoft: Azure Kubernetes Engine(AKE) • Google: Google Kubernetes Engine(GKE) • IBM/ RedHat: OpenShift • Oracle: Oracle Kubernetes Engine(OKE) • Digital Ocean
- 14. OKE – OracleKubernetes Engine
- 15. Setup a CloudNative Container Pipeline 3 Provision Kubernetes
- 16. • Scripted setupKubernetes Engine • Any Cloud Infrastructure (OCI) • Terraform providers client Generate priv/pubkeys Set TF envars terraform plan terraform apply Setup compartment Verify tenancy& user details Setup API key K8S Orchestration Engine – Kubernetes - provision
- 17. Orchestration Engine –Kubernetes - provision TF init, plan and apply
- 18. Setup a CloudNative Container Pipeline 4 Container Pipeline
- 19. Oracle Container Pipelines •Former Wercker CI CD • Docker-Native CI/CD Automation platform • For Kubernetes & Microservice Deployments. • OpenSource • Artifact can be a packaged Docker Container
- 20. Container Registry AD 1 AD3 Node Pool K8S Cluster Node Pool BM VM Virtual Network PV AD 2 Kubernetes Exposed Service Orchestration Container Engine (Kubernetes) Pods LB Test Push Build Oracle Container Pipeline Flow GitHub Orchestration web API
- 21. Working group Can have many applications Users can be in many organizations Wercker Organization
- 22. Links toa project on Github, Gitlab, or Bitbucket ‘Contains’ the build workflows Has dependencies, environment, permissions Wercker Organization
- 23. Steps □ Isolatedbash script or compiled binary for accomplishing specific automation tasks. Pipelines (pipeline consists of steps) □ Pipelines: a series of steps that are triggered on a git push or the completion of another pipeline. Workflows □ Workflows is a set of chained branched pipelines to form multi- stage, multi-branch complex ci/cd flows Concepts for building pipelines
- 24. • Added towercker.yaml • Add Pipeline: deploy-to-cluster • Add K8S specifics Oracle Container Pipeline
- 25. Terraform Step - script: name:Set Terraform variables code: | export TF_VAR_my_var1=foo export TF_VAR_my_var2=bar - releasequeue/run- terraform@0.0.15: action: "plan" out_file: "output_file" state_stor_bucket: "test- terraform-state-bucket" state_stor_bucket_region: “eu-fran-1" state_stor_key: "terraform_state"
- 26. Steps: installpackages, run tests, etc. env variables, SSH keys ‘build’ pipeline created by default ‘Pipelines’ concept is in the web UI and in the wercker.yml file Pipelines
- 27. Injected intorepository creating app □ Automatically creates a ‘build’ pipeline and workflow with a hook to the repository Various templates □ Go, javascript, java, node, php, python, etc. Box: base Docker image Service dependencies □ each pipeline can have its own box and service dependencies Pipelines Steps/After-Steps Yaml file
- 28. Flexible linkingof pipelines Requires pipelines in wercker.yml and configured in the web UI. □ Depends on yml file, but workflows are not in the yml file Variation based on branch Parallel processing – concurrency Pipeline workflow
- 29.
- 30. Helm • Release andPackage Management for Kubernetes • Helm is client, Tiller runs Serverside in Kubernetes • Can be integrated with CI build tools ( Maven, Jenkins, and Wercker) curl https://raw.githubusercontent.com/helm/helm/master/scripts/get > get_helm.sh
- 31. Helm helm init $HELM_HOME hasbeen configured at /home/oracle/.helm. Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster. Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy. To prevent this, run `helm init` with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation Happy Helming!
- 32. Helm kubectl create serviceaccount--namespace kube-system tiller serviceaccount "tiller" created kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin -- serviceaccount=kube-system:tiller clusterrolebinding "tiller-cluster-rule" created kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}' deployment "tiller-deploy" patched
- 33. Helm Basic Components .helm/ charts/ templates/ Chart.yaml values.yaml Basicinfo about name, description, version of app Templates for charts; tiller parses them with values Values for parsing templates to tiller
- 34. Helm workflow architecture HelmChartHelm Chart tiller server K8s api-server Chart deployed Application
- 35. Add these allto wercker pipeline Setup the git repository Add application wercker_helm- chart-generate Don’t generate wercker.yml Set the proper variables Run the build
- 36. Add these allto wercker pipeline Setup the git repository Add application wercker_helm- chart-generate Don’t generate wercker.yml Set the proper variables Run the build
- 37. Add these allto wercker pipeline Setup the git repository Add application wercker_helm- chart-generate Don’t generate wercker.yml Set the proper variables Run the build
- 38.