Download as PDF, PPTX
Uploaded byCodefresh
Docker based-Pipelines with Codefresh
The document discusses the use of Docker-based pipelines in continuous integration, highlighting benefits such as easy tool upgrades, support for multiple versions, and dynamic image creation. Codefresh is presented as a comprehensive CI/CD solution with built-in Docker support, allowing for streamlined deployments and flexible build tooling. Various demos illustrate the capabilities of Codefresh in managing dependencies, caching artifacts, and creating on-demand Docker images.
Docker based-Pipelines with Codefresh
- 1.
- 2. About Kostis Kapelonis ●Software engineer ● Technical writer ● Manning author
- 3.
- 4. Agenda 1. Docker usagein Continuous Integration 2. Dockerizing build tools as pipeline steps 3. Upgrading build tools to new versions 4. Mixing multiple versions of the same tool in the same pipeline 5. Creating new pipeline steps on the fly ● Demos for everything using Codefresh https://github.com/containers101/docker-based-pipelines-webinar/
- 5.
- 6. “Docker-based” means 2different things: Using Docker as a deployment package (this is what most people think) Using Docker for build Tooling (this is not what most people think) 90% of cases: “We have migrated to Docker in production”
- 7. The Basic SoftwareLifecycle
- 8. Before Docker –The Dark Ages
- 9.
- 10. Adding Docker-based BuildPipelines
- 11.
- 12. Resources for Dockeras Deployment artifact
- 13. Using Docker inContinuous Integration ● EVERY build tool is placed in a Docker container ● The build node has only Docker installed and nothing else ● A pipeline is a series of commands that run inside a Docker context ● After each build the node reverts back to its original state
- 14. Container per buildstep ● Codefresh requires ALL tools to be dockerized ● You can use any public or private Docker image as tooling ● Each build step has a Docker image as context ● Pipelines are described in declarative YAML
- 15. About Codefresh ● Dockerbased CI/CD solution ● Each build step is a Docker image ● Native support for Docker, Helm, Kubernetes deployments ● Includes built-in Docker registry and Helm repository ● 20,000+ users
- 16.
- 17. Traditional VM basedproblems
- 18. Traditional CI PlatformQuestions: ● Do you support my favorite version of Node/Java/Go/Ruby/Python? ● Do you support maven, yarn, gulp, sbt, gradle, rake? ● Can I run Ansible? Terraform? GCloud? AWS CLI? ● Can I run Kubectl? Helm? Draft?
- 19.
- 20. Demo 2: Adding Goand AWS CLI https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
- 21. Does Codefresh Support… ●Node 10? ● Perl 6? ● Python2? ● Gradle? ● Vault? ● AWS cli? ● Sonar? ● Findbugs? ● Selenium? ● Snyk? ● Clair?
- 22. Does Codefresh Support… ●Node 10? ● Perl 6? ● Python2? ● Gradle? ● Vault? ● AWS cli? ● Sonar? ● Findbugs? ● Selenium? ● Snyk? ● Clair? YES! Because there is a Docker image for it
- 23. Codefresh Pipelines areFuture Proof ● You can use ANY existing Docker image from Dockerhub or any other Registry ● Every time a new tool comes out, it can be used right away if packaged in a Docker image
- 24. Tool Upgrades andVersion Clashes
- 25. Updating a Toolin a Traditional VM Pipeline
- 26.
- 27. Demo: Updating Python to3.7 https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
- 28. Using Tools fromDifferent Versions ● Version clashes are a huge pain for both developers and operators ● Legacy projects need to still use old version ● Using different versions in the same pipeline is almost impossible ● Developers want to use latest version of tool, traditional CI/CD platforms may not be able to keep up
- 29. Wasting Effort on“Version Managers”
- 30. Wasting Effort on“Version Managers” ● They allow developers to switch between different versions ● Tied to a specific technology/programming language ● Require they own installation/ maintenance ● Must be upgraded for new versions
- 31. The Problem withPython ● Different python versions are a notorious problem ● Until recently you needed dedicated support from your CI platform ● What happens if I want to test Python 2.5?
- 32. Replacing “version managers” withDocker ● Works for any language/framework ● Already installed on the build node ● Its own version is independent from the tools ● Can use any public and private image
- 33. Codefresh “Python Support” ●We support EVERY container ever made ● We support EVERY container that you can make in the future
- 34. Demo 3: Multiple Python/Nodeversions https://github.com/containers101/docker-based-pipelines-webinar/tree/master/03_multiple_versions
- 35.
- 36. Data Sharing ● Stepsneed to communicate ● Output of one step is input for the next ● Artifacts (node modules, ruby gems, maven caches) need to persist ● Test reports/Coverage statistics
- 37. Caches and Artifacts(Traditional CI solutions) ● “Cache” directive ● Need to be setup explicitly ● Different for each build tool ● “Artifact” directive ● Developers defines exact path of what needs to be archived ● Used for the result of the whole build or as shared data between steps
- 38. All Steps Sharea Volume in Codefresh
- 39. Project is onthe Volume ● Project is checked out in the volume ● Volume is also persisted between builds ● Any build tools that use the project folder for artifacts will gain automatic caching ● For other tools you just need to point their cache to /codefresh/volume ● There is no need for special “artifact settings”. Just place files in /codefresh/volume
- 40. Demo 4 –Node Modules https://github.com/containers101/docker-based-pipelines-webinar/tree/master/04_volume
- 41. Dynamic Docker Images DockerTooling on Demand – A Unique Feature
- 42. Creating Docker ImagesOn-demand ● Create a Docker image as a step ● Use image in a later step ● Maximum flexibility for build context ● Image contents are not known in advance ● Codefresh is the only platform at the moment that offers this capability
- 43. Creating Docker ImagesOn-demand ● No need for multiple Docker images ● “Create and forget” build steps ● Useful for integration tests ● Keep your Docker registry small and tidy
- 44. Demo 5: Dynamic DockerImages https://github.com/containers101/docker-based-pipelines-webinar/tree/master/05_dynamic
- 45.
- 46. Plugins in TraditionalCI/CD Platforms ● Specific to the platform (vendor lock-in) ● Tied to a specific language (e.g. Groovy) ● Developer needs to learn proprietary API ● Testing and installing them is difficult
- 47. Codefresh Plugins =Docker Images
- 48. Codefresh Plugins ● Nottied to any programming language ● Require only Docker knowledge ● Easy to test, easy to search, easy to store ● Several plugins for Codefresh already available
- 49. Case study: bintray ●JFrog bintray integration ● There is no official docker image ● A Codefresh plugin with wrap the CLI ● Plugin will be used to query Bintray
- 50.
- 51.
- 52. Summary ● Docker-based pipelinesuse Docker images as build steps ● Upgrading tools is easy ● Using multiple versions of the same tool is trivial ● Can dynamically create build steps ● Codefresh plugins are Docker images
- 53. Get 120 FREEbuilds/month! Signup & schedule a 1:1 at: Codefresh.io Build Fast, Deploy Faster Q ?
- 54.