Getting Started with DevOps on AWS [Mar 2020]

DevOps Defined DevOps is a combination of cultural philosophies, practices and tools that increases an organizations ability to deliver applications and services at high velocity. Amazon Web Services

This is a Level 100-200 Session

Start Anywhere But Start Somewhere

How Amazon does DevOps? Decompose for Agility == Microservices Automate everything == CI/CD Standardized Tools Belts and Suspenders == Templates or Compliances Infrastructure as a Code

Google Search Trends for DevOps

Developer and IaaC Tools CodeCommit CodeBuild CodePipeline CodeDeploy Terraform

AWS Core Services ● Availability Zones for resiliency ● IAM ● S3 ● Key Management Service (KMS) ● CloudTrail ● CloudWatch Logs ● CloudWatch Events ● CloudFormation ● SNS

Demo Application Definition ● A Simple NodeJS / Express App ● ECS Fargate with Load Balancer

CodeCommit ● Secure and Scalable Git hosted on AWS ● Pay as you Go ● Integrated with IAM for granular control ● Approval Rules for Pull Requests ● Data is Encrypted in Transit and At Rest via KMS ● Core Services in use ○ IAM, S3, KMS

Setup SSH Access $>cat ~/.ssh/config Host appgambit HostName git-codecommit.us-east-1.amazonaws.com User APKAY4AJ6VUH4I22GU5Q IdentityFile ~/.ssh/id_rsa $>git clone ssh://appgambit/v1/repos/test-app If you are not able to connect to the repo, run this command to validate your SSH conntion $>ssh APKAY4AJ6VUH4I22GU5Q@git-codecommit.us-west-2.amazonaws.com $>cat ~/.ssh/config Host git-codecommit.*.amazonaws.com User APKAY4AJ6VUH4I22GU5Q IdentityFile ~/.ssh/id_rsa

IAM Policy - Allow commit to master branch { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codecommit:GitPush", "codecommit:Merge*" ], "Resource": "*", "Condition": { "StringEqualsIfExists": { "codecommit:References": [ "refs/heads/master" ] } } } ] }

IAM Policy - Deny Commit to Master branch { "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Action": [ "codecommit:GitPush", "codecommit:DeleteBranch", "codecommit:PutFile", "codecommit:Merge*" ], "Resource": "arn:aws:codecommit:us-east-2:111111111111:test-app", "Condition": { "StringEqualsIfExists": { "codecommit:References": [ "refs/heads/master" ] }, "Null": { "codecommit:References": false } } } ] }

CodeBuild ● Fully managed build service ● Auto-provision, manage and scale build servers ● Pay as you build, pay only for the number of build minutes ● Fully customize the build process with pre_build, build and post_build commands ● Intergrate with CodePipeline or Jenkins

Sample buildspec.yml file version: 0.2 phases: install: runtime-versions: docker: 18 commands: pre_build: commands: - echo Logging in to Amazon ECR.... - aws --version - $(aws ecr get-login --no-include-email --region $REGION) - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-8) - echo Latest commit hash $COMMIT_HASH build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t test-app:latest . - docker tag test-app:latest $REPO_URL:latest - docker tag $REPO_URL:latest $REPO_URL:$COMMIT_HASH post_build: commands: - echo Build completed on `date` - echo pushing to repo - docker push $REPO_URL:latest - docker push $REPO_URL:$COMMIT_HASH artifacts: files: - taskdef.json - appspec.yaml

CodePipeline Source Build Deploy CodePipeline CodeCommit CodeBuild CodeDeploy Source code changes are pushed to the repo Update running environment

CodePipeline ● Automate your release process ● Configurable visual workflow ● Number of built-in stages and actions, like Source, Build, Test, and Deploy ● Easy to customize ● Pay as you go, $1 per active pipeline, this is an integration service ● And most important, Rapid Delivery

CodePipeline Stages and Actions ● Source ○ S3, ECR, CodeCommit, Github, etc ● Build ○ CodeBuild, Jenkins, etc ● Test ○ CodeBuild, Device Farms, Jenkins, 3rd party services, etc ● Deploy ○ S3, CloudFormation, CodeDeploy, ECS, Alexa, etc ● Approve ○ Manual ● Invoke ○ AWS Lambda

Continuous Integration (CI) So far... ● We have created and pushed our code in CodeCommit ● We are using the CodeBuild to build the Docker Images ● We are using the CodeBuild to upload the Docker Images to ECR And ● We are using the CodePipeline to automate the above process ● We can add test scripts and commands to validate the integration

Let’s check our app using ECS Fargate ● Create ECS Cluster ● Create a Task Definition ● Create a Service ● Validate the application

Everything works! ● So far everything works, but our ECS cluster is not taking the new changes automatically ● It creates the build and pushes the docker image to the ECR repo only

CodeDeploy ● Automated Deployments ● Deploy to ECS, Lambda, EC2, On-Prem Instances ● Rolling and Blue/Green Updates ● Stop or Rollback the Deployment

Now we have a fully automated application pipeline... WITH ONE BIG PROBLEM But the Infrastructure to support that is entirely manual...

Infrastructure as a Click ● While our app works with automation ● But the infrastructure for this whole setup is done by “clicks” ● Imagine if we need to replicate this same after a few days or few weeks ● Infrastructure Automation or Infrastructure as a Code is MOST IMPORTANT THING for a growing application or organization

Infrastructure as a Code ● Easy to Replicate ● Easy to Manage ● Supports Changes and Rollbacks ● Versioning

Terraform ● Declarative configuration files ● Modules which can be imported ● Supports multiple cloud providers ● Uses Hashicorp Configuration Language

Terraform VPC Example provider "aws" { region = "${var.aws_region}" } resource "aws_vpc" "default" { cidr_block = "10.0.0.0/16" } resource "aws_internet_gateway" "default" { vpc_id = "${aws_vpc.default.id}" } resource "aws_route" "internet_access" { route_table_id = "${aws_vpc.default.main_route_table_id}" destination_cidr_block = "0.0.0.0/0" gateway_id = "${aws_internet_gateway.default.id}" } resource "aws_subnet" "default" { vpc_id = "${aws_vpc.default.id}" cidr_block = "10.0.1.0/24" map_public_ip_on_launch = true }

Other tools to checkout AWS Chatbot AWS CodeStar Jenkins Terraform CI/CD template to run containers with ECS Fargate https://github.com/AppGambitStudio/aws-terraform-cicd AWS OpsWorks

Where to start ● https://www.docker.com/ ● https://aws.amazon.com/devops/what-is-devops/ ● https://aws.amazon.com/devops/ ● https://www.terraform.io/ ● https://aws.amazon.com/serverless/

Sometimes it’s best to follow the dotted line, instead of connecting the dots. Founder of CTO.ai @ re:Invent 2019

Getting Started with DevOps on AWS [Mar 2020]

More Related Content

Similar to Getting Started with DevOps on AWS [Mar 2020]

More from Dhaval Nagar

Recently uploaded

Getting Started with DevOps on AWS [Mar 2020]