DEV Community

AWS Community Builders profile image Keisuke FURUYA
Keisuke FURUYA for AWS Community Builders

Posted on

Managing AWS Resources with ACK and helmfile

#aws #helmfile #ack #kubernetes

AWS Controllers for Kubernetes (ACK) is a Kubernetes Custom Controller that allows you to manage AWS resources through Kubernetes manifests. By using ACK, you can easily manage AWS resources in the same lifecycle as your applications. It is suitable for scenarios where you need to create AWS resources alongside your applications for testing purposes and delete them together with the application when no longer needed. This article explains how to manage AWS resources using Kubernetes manifests. While there are various ways to manage Kubernetes manifest files, this article focuses on using helmfile for management.

Prerequisites

Please install the following on your local machine:

  • eksctl
  • kubectl
  • helm
  • helmfile

Set up the following in your AWS account. This IAM policy has the necessary permissions for the IAM Controller of ACK to create IAM resources.

First, let's create an EKS cluster. In this example, we will deploy it with minimal configuration using eksctl.

eksctl create cluster --name ack-helmfile
Enter fullscreen mode Exit fullscreen mode

Next, create an OIDC provider, IAM Role and service account.

eksctl utils associate-iam-oidc-provider --cluster=ack-helmfile --approve eksctl create iamserviceaccount --cluster=ack-helmfile --name=ack-dynamodb-controller --namespace=ack-system --attach-policy-arn=arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess --approve eksctl create iamserviceaccount --cluster=ack-helmfile --name=ack-iam-controller --namespace=ack-system --attach-policy-arn=arn:aws:iam::[your AWS account ID]:policy/ack-iam-controller-policy --approve
Enter fullscreen mode Exit fullscreen mode

Install ACK on the created EKS cluster. This time, we will install the IAM and DynamoDB controllers. Below is a sample configuration for the ACK controllers using helmfile.

helmfile.yaml

repositories: - name: aws-controllers-k8s url: public.ecr.aws/aws-controllers-k8s oci: true environments: {{ .Environment.Name }}: values: - environments/{{ .Environment.Name }}/values.yaml --- releases: - name: "ack-dynamodb-controller" namespace: "{{ .Namespace | default .Values.awsControllersK8s.namespace }}" createNamespace: true chart: "aws-controllers-k8s/dynamodb-chart" version: "{{ .Values.awsControllersK8s.dynamodb.version }}" installedTemplate: "{{ .Values.awsControllersK8s.dynamodb.installed }}" values: - values/dynamodb.yaml.gotmpl - name: "ack-iam-controller" namespace: "{{ .Namespace | default .Values.awsControllersK8s.namespace }}" createNamespace: true chart: "aws-controllers-k8s/iam-chart" version: "{{ .Values.awsControllersK8s.iam.version }}" installedTemplate: "{{ .Values.awsControllersK8s.iam.installed }}" values: - values/iam.yaml.gotmpl
Enter fullscreen mode Exit fullscreen mode

environments/test/values.yaml

awsControllersK8s: namespace: ack-system region: ap-northeast-1 dynamodb: installed: true version: 1.1.1 iam: installed: true version: 1.2.1
Enter fullscreen mode Exit fullscreen mode

values/dynamodb.yaml.gotmpl

aws: region: {{ .Values.awsControllersK8s.region }} serviceAccount: create: false name: ack-dynamodb-controller
Enter fullscreen mode Exit fullscreen mode

values/iam.yaml.gotmpl

aws: region: {{ .Values.awsControllersK8s.region }} serviceAccount: create: false name: ack-iam-controller
Enter fullscreen mode Exit fullscreen mode

Applying helmfile will create the IAM and DynamoDB controllers.

helmfile -e test apply .
Enter fullscreen mode Exit fullscreen mode 
kubectl get deployment -n ack-system NAME READY UP-TO-DATE AVAILABLE AGE ack-dynamodb-controller-dynamodb-chart 1/1 1 1 79s ack-iam-controller-iam-chart 1/1 1 1 79s
Enter fullscreen mode Exit fullscreen mode

Deploying Required AWS Resources in the Application

Now that the DynamoDB controller is ready, let's use it to create a DynamoDB resource. Here, we will deploy a simple table called "ack-dynamodb-table".

helmfile.yaml

environments: {{ .Environment.Name }}: values: - environments/{{ .Environment.Name }}/values.yaml --- releases: - name: "ack-dynamodb-table" namespace: default installedTemplate: "{{ .Values.dynamodb.installed }}" chart: "./manifests"
Enter fullscreen mode Exit fullscreen mode

environments/test/values.yaml

dynamodb: installed: true
Enter fullscreen mode Exit fullscreen mode

manifests/table.yaml

apiVersion: dynamodb.services.k8s.aws/v1alpha1 kind: Table metadata: name: ack-dynamodb-table spec: tableName: ack-dynamodb-table attributeDefinitions: - attributeName: id attributeType: "N" keySchema: - attributeName: id keyType: "HASH" provisionedThroughput: writeCapacityUnits: 1 readCapacityUnits: 1
Enter fullscreen mode Exit fullscreen mode 
helmfile -e test apply .
Enter fullscreen mode Exit fullscreen mode

Applying it will create the DynamoDB table defined in the manifest.

Created DynamoDB Table

Deploying Role and Service Account for IRSA

To utilize DynamoDB from the application, we need to use IRSA (IAM Roles for Service Accounts). IRSA enables the association of Kubernetes service accounts with IAM roles. Specifically, it involves setting up the IAM role, appropriate trust relationships, and annotating the service account to make it usable. While eksctl can handle this through commands(as we did earlier), we will create them using ACK this time.

helmfile.yaml

environments: {{ .Environment.Name }}: values: - environments/{{ .Environment.Name }}/values.yaml --- releases: - name: "ack-iam-role" namespace: default installedTemplate: "{{ .Values.iam.installed }}" chart: "./manifests"
Enter fullscreen mode Exit fullscreen mode

We will obtain the OIDC URL required for the trust relationship of the role used in service account from the EKS management console and add it to the values.yaml file.

EKS OIDC info

environments/test/values.yaml

iam: installed: true account_id: [your AWS account ID] oidc_url: [your EKS oidc url]
Enter fullscreen mode Exit fullscreen mode

We will define the policy, role, and service account in the manifest that have permissions to access DynamoDB.

manifests/role.yaml

apiVersion: iam.services.k8s.aws/v1alpha1 kind: Policy metadata: name: ack-dynamodb-policy spec: policyDocument: | { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Action": [ "dynamodb:UpdateTable", "dynamodb:UpdateItem", "dynamodb:Scan", "dynamodb:Query", "dynamodb:PutItem", "dynamodb:ListTables", "dynamodb:GetItem", "dynamodb:DescribeTable", "dynamodb:DeleteItem", "dynamodb:BatchWriteItem", "dynamodb:BatchGetItem" ], "Resource": [ "arn:aws:dynamodb:ap-northeast-1:{{ .Values.iam.account_id }}:table/ack-dynamodb-table" ] } ] } name: ack-dynamodb-policy --- apiVersion: iam.services.k8s.aws/v1alpha1 kind: Role metadata: name: ack-app-role spec: assumeRolePolicyDocument: | { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::{{ .Values.iam.account_id }}:oidc-provider/{{ .Values.iam.oidc_url }}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "{{ .Values.iam.oidc_url }}:aud": "sts.amazonaws.com", "{{ .Values.iam.oidc_url }}:sub": "system:serviceaccount:default:ack-app-serviceaccount" } } } ] } name: ack-app-role policies: - "arn:aws:iam::{{ .Values.iam.account_id }}:policy/ack-dynamodb-policy" --- apiVersion: v1 kind: ServiceAccount metadata: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::{{ .Values.iam.account_id }}:role/ack-app-role eks.amazonaws.com/sts-regional-endpoints: "true" eks.amazonaws.com/token-expiration: "86400" argocd.argoproj.io/sync-wave: "1" name: ack-app-serviceaccount namespace: default
Enter fullscreen mode Exit fullscreen mode 
helmfile -e test apply .
Enter fullscreen mode Exit fullscreen mode

Applying it will create the IAM policy, IAM role with the attached policy, and Kubernetes service account defined in the manifest.

Created IAM Role

kubectl get serviceaccount NAME SECRETS AGE ack-app-serviceaccount 0 7m57s default 0 49m
Enter fullscreen mode Exit fullscreen mode

Testing Access to DynamoDB

With the created service account, access to DynamoDB is now permitted from the application. Let's verify this by using the AWS CLI to access DynamoDB. First, let's try executing the PutItem operation on DynamoDB without specifying the service account.

kubectl run awscli --image=amazon/aws-cli -- dynamodb put-item --table-name ack-dynamodb-table --item '{ "id": { "N": "1" }, "value": { "S": "test" }}'
Enter fullscreen mode Exit fullscreen mode

This will result in an error:

kubectl logs awscli An error occurred (AccessDeniedException) when calling the PutItem operation: User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/eksctl-ack-helmfile-nodegroup-ng-NodeInstanceRole-EVFGB3QY0C29/i-097e6311fa2cc1da1 is not authorized to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:ap-northeast-1:xxxxxxxxxxxx:table/ack-dynamodb-table because no identity-based policy allows the dynamodb:PutItem action
Enter fullscreen mode Exit fullscreen mode

Let's now execute PutItem on DynamoDB using the service account.

kubectl run awscli --image=amazon/aws-cli --overrides='{ "spec": { "serviceAccountName": "ack-app-serviceaccount" } }' -- dynamodb put-item --table-name ack-dynamodb-table --item '{ "id": { "N": "1" }, "value": { "S": "test" }}'
Enter fullscreen mode Exit fullscreen mode

The PutItem operation is now successful.

Succeed PutItem

Summary

In this article, we used AWS Controllers for Kubernetes (ACK) to manage AWS resources using Kubernetes manifests. We discussed the configuration to make the application able to utilize AWS resources and explored how to manage Kubernetes manifest files using helmfile.

Using ACK and the described workflow, it becomes easier to manage AWS resources in conjunction with the application's lifecycle, allowing for convenient creation and deletion of AWS resources alongside the application.

Top comments (0)