DEV Community

Prakash Rao
Prakash Rao

Posted on

Advanced CI/CD Pipelines on AWS

#aws #cicd #cloud #devops

In this post, I’ll walk you through setting up an advanced CI/CD pipeline on AWS using native services and show you how to create and configure components like CodeBuild, CodeDeploy, and CodePipeline using AWS CLI commands and configuration files. I will also include outputs for each command so you can see what to expect when you run these commands yourself.

Flow Chart

Note:

  1. Flowchart was created using mermaid.live
  2. Since CodeCommit is no longer onboarding new customers, choosing GitHub as the source repository. If you already have a GitHub repository for your project, simply update the configuration files accordingly.

Prerequisites: Please make sure that the AWS CLI is installed and configured with the proper IAM permissions.

1. Create a Source Repository in GitHub

First, let’s create a new GitHub repository where our source code will be there.

git init git remote add origin https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo.git git add . git commit -m "Initial commit" git push -u origin main
Enter fullscreen mode Exit fullscreen mode

2. Set Up a CodeBuild Project

Next let's create a CodeBuild project that builds our application. In this example, we’ll use a simple Node.js project. Save the following JSON configuration as codebuild-project.json:

{ "name": "advanced-ci-cd-build", "description": "Build project for advanced CI/CD pipeline demo", "source": { "type": "GITHUB", "location": "https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo" }, "artifacts": { "type": "NO_ARTIFACTS" }, "environment": { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:6.0", "computeType": "BUILD_GENERAL1_MEDIUM", "environmentVariables": [ { "name": "NODE_ENV", "value": "production" } ] }, "serviceRole": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeBuildServiceRole" }
Enter fullscreen mode Exit fullscreen mode

Create project with:

aws codebuild create-project --cli-input-json file://codebuild-project.json
Enter fullscreen mode Exit fullscreen mode

Output:

{ "project": { "name": "advanced-ci-cd-build", "arn": "arn:aws:codebuild:us-east-1:123456789012:project/advanced-ci-cd-build", "description": "Build project for advanced CI/CD pipeline demo", "source": { "type": "GITHUB", "location": "https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo" }, "artifacts": { "type": "NO_ARTIFACTS" }, "environment": { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:6.0", "computeType": "BUILD_GENERAL1_MEDIUM", "environmentVariables": [ { "name": "NODE_ENV", "value": "production" } ] }, "serviceRole": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeBuildServiceRole", "created": "2024-05-10T15:00:00Z" } }
Enter fullscreen mode Exit fullscreen mode

Note: Make sure CodeBuild service role has permissions for GitHub, S3 (if you use artifacts), and CloudWatch logging.

3. Define a Build Specification (buildspec.yml)

Create a buildspec.yml file in the root of the repository. This file instructs CodeBuild on how to run our build.

version: 0.2 phases: install: runtime-versions: nodejs: 16 commands: - echo "Installing dependencies..." - npm install pre_build: commands: - echo "Running lint and tests..." - npm run lint - npm test build: commands: - echo "Building the project..." - npm run build post_build: commands: - echo "Build completed on `date`" artifacts: files: - '**/*' discard-paths: yes
Enter fullscreen mode Exit fullscreen mode

Push this file to the GitHub repository so CodeBuild uses it during the build process.

4. Set Up a CodeDeploy Application and Deployment Group

For blue/green deployments, we’ll use CodeDeploy.

Create a CodeDeploy Application

Command:

aws deploy create-application \ --application-name advanced-ci-cd-app \ --compute-platform Server
Enter fullscreen mode Exit fullscreen mode

Output:

{ "applicationId": "abcdef12-3456-7890-abcd-ef1234567890", "applicationName": "advanced-ci-cd-app", "computePlatform": "Server" }
Enter fullscreen mode Exit fullscreen mode

Create a Deployment Group
Save the following as deployment-group.json (update below values as needed):

{ "applicationName": "advanced-ci-cd-app", "deploymentGroupName": "advanced-ci-cd-deploy-group", "deploymentConfigName": "CodeDeployDefault.OneAtATime", "ec2TagFilters": [ { "Key": "Name", "Value": "MyAppServer", "Type": "KEY_AND_VALUE" } ], "serviceRoleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeDeployServiceRole", "blueGreenDeploymentConfiguration": { "terminateBlueInstancesOnDeploymentSuccess": { "action": "TERMINATE", "terminationWaitTimeInMinutes": 5 }, "deploymentReadyOption": { "actionOnTimeout": "CONTINUE_DEPLOYMENT" } }, "autoRollbackConfiguration": { "enabled": true, "events": ["DEPLOYMENT_FAILURE"] } }
Enter fullscreen mode Exit fullscreen mode

Create Deployment group with:

aws deploy create-deployment-group --cli-input-json file://deployment-group.json
Enter fullscreen mode Exit fullscreen mode

Output:

{ "deploymentGroupId": "abcdef12-3456-7890-abcd-ef1234567890", "deploymentGroupName": "advanced-ci-cd-deploy-group", "applicationName": "advanced-ci-cd-app" }
Enter fullscreen mode Exit fullscreen mode

Note: Make sure that the CodeDeploy service role has the necessary permissions to access EC2 and execute deployment actions.

5. Create a CodePipeline

Let's put everything together by creating a pipeline that orchestrates the source, build, and deployment stages.

Save the following as pipeline.json (adjust names, regions, and bucket names as needed):

{ "pipeline": { "name": "advanced-ci-cd-pipeline", "roleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodePipelineServiceRole", "artifactStore": { "type": "S3", "location": "<YOUR_ARTIFACT_BUCKET>" }, "stages": [ { "name": "Source", "actions": [ { "name": "SourceAction", "actionTypeId": { "category": "Source", "owner": "ThirdParty", "provider": "GitHub", "version": "1" }, "outputArtifacts": [ { "name": "SourceOutput" } ], "configuration": { "Owner": "<YOUR_GITHUB_USERNAME>", "Repo": "advanced-ci-cd-repo", "Branch": "main", "OAuthToken": "<YOUR_GITHUB_OAUTH_TOKEN>" }, "runOrder": 1 } ] }, { "name": "Build", "actions": [ { "name": "BuildAction", "actionTypeId": { "category": "Build", "owner": "AWS", "provider": "CodeBuild", "version": "1" }, "inputArtifacts": [ { "name": "SourceOutput" } ], "outputArtifacts": [ { "name": "BuildOutput" } ], "configuration": { "ProjectName": "advanced-ci-cd-build" }, "runOrder": 1 } ] }, { "name": "Deploy", "actions": [ { "name": "DeployAction", "actionTypeId": { "category": "Deploy", "owner": "AWS", "provider": "CodeDeploy", "version": "1" }, "inputArtifacts": [ { "name": "BuildOutput" } ], "configuration": { "ApplicationName": "advanced-ci-cd-app", "DeploymentGroupName": "advanced-ci-cd-deploy-group" }, "runOrder": 1 } ] } ], "version": 1 } }
Enter fullscreen mode Exit fullscreen mode

Create pipeline with:

aws codepipeline create-pipeline --cli-input-json file://pipeline.json
Enter fullscreen mode Exit fullscreen mode

Output:

{ "pipeline": { "name": "advanced-ci-cd-pipeline", "version": 1, "artifactStore": { "type": "S3", "location": "my-artifact-bucket" }, "stages": [ { "name": "Source", "actions": [ { ..... } ] }, { "name": "Build", "actions": [ { ..... } ] }, { "name": "Deploy", "actions": [ { ..... } ] } ] }, "created": "2024-05-10T15:30:00Z" }
Enter fullscreen mode Exit fullscreen mode

6. Testing and Monitoring the Pipeline

After creating the pipeline, trigger a test execution by pushing a commit to your GitHub repository. Then, check the pipeline status with:

aws codepipeline get-pipeline-execution --pipeline-name advanced-ci-cd-pipeline --pipeline-execution-id <execution-id>
Enter fullscreen mode Exit fullscreen mode

Output:

{ "pipelineExecution": { "pipelineName": "advanced-ci-cd-pipeline", "pipelineExecutionId": "12345678-1234-1234-1234-123456789012", "status": "Succeeded", "artifactRevisions": [ ..... ], "startTime": "2024-05-10T15:31:00Z", "lastUpdateTime": "2024-05-10T15:33:00Z" } }
Enter fullscreen mode Exit fullscreen mode

Additionally, we can set up CloudWatch dashboards and alarms to monitor key metrics such as build duration and deployment status.

Conclusion

In this blog I demonstrated how we can build an advanced CI/CD pipeline on AWS using CLI commands and configuration files. By integrating GitHub, CodeBuild, CodeDeploy (with blue/green deployments), and CodePipeline, we created a fully automated workflow.

Happy Deploying! :)

Top comments (0)