Our Migration from Jenkins to GitLab CI
Philipp Westphalen FullStack Developer GitLab Hero & Meetup Organizer Hackerschool Inspirer /Koala_Phil /Phil404
adSoul GmbH
Our Technical Landscape
Our Technical Landscape Internet Build Tool Private Cloud
Our Challenges
Our Challenges Small Development Team Not enough time for everything Maintenance of our Build Tools / Process Build process is tough to understand Jenkins is unstable and nonintuitive Visibility of Development Status of Issues Cost on Development Tools
Our Journey to GitLab
Gather all Information
Gather all Information Ask yourself: What could be possible with GitLab? Share the insights with all members of our team How could it affect our daily work? What improvements could we achieve with a migration? Ask for the needs of every role Talk in specific rounds with your coworkers Have several feedback loops Commitment of the whole team to test GitLab
Interesting Features GitLab CI/CD Jobs based on docker containers Integration of Sentry and Prometheus Yaml based configuration file for pipelines Issue Management and automated Support A single Tool for the „Tool-Chain“
Small Projects first
Small Projects first Start with a simple Project Not enough time for everything Figure out new Ways of Building Software Try different building solutions for your software (e.g. Gradle) Find Solutions for your Challenges Use GitLab for Docker Images and other features We run GitLab-Runners in our Kubernetes Cluster to get into it
Maven vs Gradle Gradle is easier to use You can define custom tasks Unit Tests Integration Tests Environment variables are injected automatically
Challenge with GitLab-Runner in our Kubernetes Caching is having a huge impact on building times Notice: we used a private cloud Kubernetes cluster Cloud Caching Local CachingWithout Caching Download / upload sources Download sources (if needed)Download sources Stable duration Doubles duration of without Could be unstable Short duration Connection errors… Not supported by provider
Changes to our Technical Landscape Internet Build Tool Private Cloud PreviousCurrent
Migration of our Main Project
Phase 1: Move the Repository
Phase 1: Move the Repository Step 1: Transfer the Repository to GitLab Use the GitHub Import by GitLab Step 2: Modify Scripts with new Origin We had problems with migrating our Issues / Releases / Tags Exchange GitHub API calls with their GitLab equivalent
Open Source: GitHub Issue Migrator CLI-Tool Migrate the Issues from GitHub Synchronize the contributers Optional: Releases and Tags @Phil404/github-issue-migrator
Phase 2: Migrate the CI/CD Pipeline
Phase 2: Migrate the CI/CD Pipeline Challenge: our build process was far too complex for an easy Lift‘n‘Shift
Tip: Increase the Log Limit Useful if you have a lot of logs generated by your job Especially when you debug your build step UI is the only view for logs Standard size of logs: 4MiB
Phase 3: Time for Improvements
Phase 3: Time for Improvements Split the different Steps „One job does one thing perfectly“ Keep it small and simple
Phase 3: Time for Improvements Change our Build Tool to Gradle More flexibel for our DevOps strategy Parallel Job Processing Separate jobs for the same “goal“, like the test stage Standard Docker Images Improved building time a little bit Use official images which fit our requirements Less time for maintenance Use Resources efficient
Key Learnings from Migration
Phase 3: Key Learnings from Migration Plan your Migration With low maintenance and high customizability Step by Step Don‘t try it by a big bang. Keep your infrastructure running and deployable Rethink your DevOps Strategy Try out something new and question your existing pipelines / processes Start with a small Project
Tip: Keep your Pipeline User Friendly Name your jobs clear to what they do Style your config for a better overview Write comments for variables and hard to understand code pieces
Why GitLab suits us perfectly
Why GitLab suits us perfectly Powerful CI/CD Solution With low maintenance and high customizability One Home for Everything User Experience Low Maintenance Project Management
Thanks for your Attention! Questions? /Koala_Phil /Phil404

GitLab - Java User Group

  • 1.
    Our Migration fromJenkins to GitLab CI
  • 2.
    Philipp Westphalen FullStack Developer GitLab Hero& Meetup Organizer Hackerschool Inspirer /Koala_Phil /Phil404
  • 3.
  • 4.
  • 5.
    Our Technical Landscape InternetBuild Tool Private Cloud
  • 6.
  • 7.
    Our Challenges Small DevelopmentTeam Not enough time for everything Maintenance of our Build Tools / Process Build process is tough to understand Jenkins is unstable and nonintuitive Visibility of Development Status of Issues Cost on Development Tools
  • 8.
  • 9.
  • 10.
    Gather all Information Askyourself: What could be possible with GitLab? Share the insights with all members of our team How could it affect our daily work? What improvements could we achieve with a migration? Ask for the needs of every role Talk in specific rounds with your coworkers Have several feedback loops Commitment of the whole team to test GitLab
  • 11.
    Interesting Features GitLab CI/CD Jobsbased on docker containers Integration of Sentry and Prometheus Yaml based configuration file for pipelines Issue Management and automated Support A single Tool for the „Tool-Chain“
  • 12.
  • 13.
    Small Projects first Startwith a simple Project Not enough time for everything Figure out new Ways of Building Software Try different building solutions for your software (e.g. Gradle) Find Solutions for your Challenges Use GitLab for Docker Images and other features We run GitLab-Runners in our Kubernetes Cluster to get into it
  • 14.
    Maven vs Gradle Gradleis easier to use You can define custom tasks Unit Tests Integration Tests Environment variables are injected automatically
  • 15.
    Challenge with GitLab-Runnerin our Kubernetes Caching is having a huge impact on building times Notice: we used a private cloud Kubernetes cluster Cloud Caching Local CachingWithout Caching Download / upload sources Download sources (if needed)Download sources Stable duration Doubles duration of without Could be unstable Short duration Connection errors… Not supported by provider
  • 16.
    Changes to ourTechnical Landscape Internet Build Tool Private Cloud PreviousCurrent
  • 17.
    Migration of ourMain Project
  • 18.
    Phase 1: Movethe Repository
  • 19.
    Phase 1: Movethe Repository Step 1: Transfer the Repository to GitLab Use the GitHub Import by GitLab Step 2: Modify Scripts with new Origin We had problems with migrating our Issues / Releases / Tags Exchange GitHub API calls with their GitLab equivalent
  • 20.
    Open Source: GitHub IssueMigrator CLI-Tool Migrate the Issues from GitHub Synchronize the contributers Optional: Releases and Tags @Phil404/github-issue-migrator
  • 21.
    Phase 2: Migratethe CI/CD Pipeline
  • 22.
    Phase 2: Migratethe CI/CD Pipeline Challenge: our build process was far too complex for an easy Lift‘n‘Shift
  • 23.
    Tip: Increase the LogLimit Useful if you have a lot of logs generated by your job Especially when you debug your build step UI is the only view for logs Standard size of logs: 4MiB
  • 24.
    Phase 3: Timefor Improvements
  • 25.
    Phase 3: Timefor Improvements Split the different Steps „One job does one thing perfectly“ Keep it small and simple
  • 26.
    Phase 3: Timefor Improvements Change our Build Tool to Gradle More flexibel for our DevOps strategy Parallel Job Processing Separate jobs for the same “goal“, like the test stage Standard Docker Images Improved building time a little bit Use official images which fit our requirements Less time for maintenance Use Resources efficient
  • 27.
  • 28.
    Phase 3: KeyLearnings from Migration Plan your Migration With low maintenance and high customizability Step by Step Don‘t try it by a big bang. Keep your infrastructure running and deployable Rethink your DevOps Strategy Try out something new and question your existing pipelines / processes Start with a small Project
  • 29.
    Tip: Keep your PipelineUser Friendly Name your jobs clear to what they do Style your config for a better overview Write comments for variables and hard to understand code pieces
  • 30.
    Why GitLab suitsus perfectly
  • 31.
    Why GitLab suitsus perfectly Powerful CI/CD Solution With low maintenance and high customizability One Home for Everything User Experience Low Maintenance Project Management
  • 32.
    Thanks for yourAttention! Questions? /Koala_Phil /Phil404