DEV Community

Cover image for How to setup AWS Billing metrics in Grafana Cloud via Terraform
AWS Community Builders profile image Johannes Konings
Johannes Konings for AWS Community Builders

Posted on • Originally published at johanneskonings.dev on

How to setup AWS Billing metrics in Grafana Cloud via Terraform

#aws #grafana #terraform

Why Terraform?

In this post I described how to display AWS Billing metrics in Grafana Cloud. Therefore it was necessary to create manually the data source and the dashboard. With Terraform, you can describe the setup as code and benefit from the full advantages of IaC.

Terraform

Terraform is a tool for infrastructure as code and works with many different provider. Terraform comes with a CLI for the deployments.

Grafana Provider and Dashboard declaration

For this use case, you need a Grafana data source and a Grafana dashboard. These configurations have to defined in a .tf file like this one

At first the provider.

provider "grafana" { url = var.grafana_url auth = var.grafana_api_key }
Enter fullscreen mode Exit fullscreen mode

Then the data source and dashboard. The dashboard section links to the file dashboards/aws-billing.json.

resource "grafana_data_source" "cloudwatch" { type = "cloudwatch" name = "CloudWatch" json_data { default_region = var.region auth_type = "keys" } secure_json_data { access_key = var.access_key_grafana secret_key = var.secret_key_grafana } } resource "grafana_dashboard" "metrics" { config_json = file("dashboards/aws-billing.json") }
Enter fullscreen mode Exit fullscreen mode

For security reasons and flexible sharing of the template, the parameters for secrets and variables like region are in a .env file. This is the template for that.

# The URL of the Grafana instance export TF_VAR_grafana_url= # The API key of the Grafana instance export TF_VAR_grafana_api_key= # IAM user like described here: https://grafana.com/docs/grafana/latest/datasources/cloudwatch/#iam-policies export TF_VAR_access_key_grafana= export TF_VAR_secret_key_grafana= # Default region of the data source export TF_VAR_region=
Enter fullscreen mode Exit fullscreen mode

The declaration of Terraform variables looks like that.

variable "grafana_url" {} variable "grafana_api_key" {} variable "access_key_grafana" {} variable "secret_key_grafana" {} variable "region" {}
Enter fullscreen mode Exit fullscreen mode

In this case it’s in the file variable.tf like described here.

Grafana API Key

Terraform can “communicate” with Grafana via an API key. Navigate to this URL “https://«Grafana instance»/org/apikeys” and create on with the role “Admin”.

grafana api key creation

Put the API key into the .env file.

Usage of the env file

Before the creation of the S3 Backend and the deployment run the command source .env.

Terraform AWS S3 backend

This setup so far works for the first deployment. Changes and a redeployment lead to an error because the resource already exists. Therefore it’s necessary to extend the setup with a Terraform backend. In this example, it’s a S3 backend.

Unfortunately, it’s not possible to use variables here. This is discussed in this issue with some approaches for workarounds. I use this one, more or less.

Concrete I put a script around the command terraform init. This script can use the environment variables and create a terraform file for the backend.

#!/bin/sh cat > ./backend.tf << EOF terraform { backend "s3" { bucket = "${TF_VAR_s3_bucket_name}" key = "${TF_VAR_backend_key}" region = "${TF_VAR_region}" } } EOF terraform init -input=false
Enter fullscreen mode Exit fullscreen mode

This script creates the bucket.

#!/bin/sh aws s3api create-bucket --bucket $TF_VAR_s3_bucket_name --region $TF_VAR_region
Enter fullscreen mode Exit fullscreen mode

For the backend, it needs an IAM user. This script creates the user and return access and secret key. Put that into the .env file.

# Name of the Terrafrom S3 backend for state handling export TF_VAR_s3_bucket_name= # Name of the state file export TF_VAR_backend_key=terraform.tfstate # IAM user credentials to access S3 and write the state file export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= #!/bin/sh aws iam create-user --user-name terraform_state aws iam create-access-key --user-name terraform_state
Enter fullscreen mode Exit fullscreen mode

This script creates and attach the missing policy.

#!/bin/bash cat > ./scripts/policy << EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::${TF_VAR_s3_bucket_name}" }, { "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject"], "Resource": "arn:aws:s3:::${TF_VAR_s3_bucket_name}/*" } ] } EOF aws iam create-policy --policy-name terraform_state --policy-document file://scripts/policy ARN=$(aws iam list-policies --query 'Policies[?PolicyName==`terraform_state`].Arn' --output text) aws iam attach-user-policy --policy-arn $ARN --user-name terraform_state
Enter fullscreen mode Exit fullscreen mode

Deployment commands

Once the S3 backend is created, you’re a few commands away from the deployment.

At first, the initialization of Terraform, which is wrapped in a script.

sh scripts/terraformInit.sh

For the next commands, the Terraform CLI is sufficient.

validate: terraform validate

plan: terraform plan

apply: terraform apply

Grafana Dasboard changes

The dashboard can now be changed directly via the JSON file in the folder dashboards. The easier way is to do that manually in Grafana and copy the changed JSON via the share functionality.

share grafana dashboard

grafana dashboard export view json

Overwrite the file aws-billing.json with the JSON from Grafana and redeploy.

CI/CD pipeline

The local deployment is also possible with a CI/CD pipeline. In this example it’s with GitHub actions. Instead of the .env file, the variables and credentials coming from GitHub secrets.

Code

https://github.com/JohannesKonings/aws-grafana-billing-dashboard/tree/terraform

Top comments (0)