DEV Community

Jamil Shaikh
Jamil Shaikh

Posted on

🚀 Building a GitOps Infrastructure Pipeline with Crossplane and Argo CD

#kubernetes #argocd #linux #crossplane

From manual kubectl commands to fully automated infrastructure management - here's how I built a production-ready GitOps pipeline

TL;DR

I built a complete GitOps infrastructure management system using:

  • 🎯 Argo CD for GitOps automation
  • ⚡ Crossplane for infrastructure provisioning
  • 🔄 App-of-Apps pattern for scalable application management
  • 📦 MetalLB as the infrastructure example
  • 🎭 Sync waves for dependency management

Result: Infrastructure changes now happen through Git commits, with full automation and zero manual intervention.

The Problem I Solved

Managing Kubernetes infrastructure traditionally sucks:

# The old way - manual and error-prone kubectl apply -f metallb-config.yaml kubectl apply -f ingress-controller.yaml kubectl apply -f monitoring-stack.yaml # Oh no! Order matters... 💥 # Which version is running in production? 🤷‍♂️ # Who made that change? 🕵️‍♂️
Enter fullscreen mode Exit fullscreen mode

I wanted infrastructure that:

  • ✅ Lives in Git (version controlled)
  • ✅ Deploys automatically (no manual steps)
  • ✅ Handles dependencies (no ordering issues)
  • ✅ Self-heals (drift detection & correction)
  • ✅ Provides audit trails (who, what, when)

The Solution Architecture 

graph LR A[Git Commit] --> B[Argo CD] B --> C[Crossplane] C --> D[Kubernetes Resources] B --> E[Sync Waves] E --> F[Ordered Deployment]
Enter fullscreen mode Exit fullscreen mode

🎯 Step 1: App-of-Apps Pattern

Instead of managing 50+ individual Argo CD applications, I use the App-of-Apps pattern:

# One app to rule them all apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: homelab-root annotations: argocd.argoproj.io/sync-wave: "-1" spec: source: repoURL: https://github.com/jamilshaikh07/homelab-gitops.git path: apps # 👈 All child apps live here syncPolicy: automated: prune: true # 🗑️ Clean up deleted resources selfHeal: true # 🔧 Fix manual changes
Enter fullscreen mode Exit fullscreen mode

Benefits:

  • One root app manages everything
  • New apps = just add YAML files
  • Automatic discovery and deployment

⚡ Step 2: Crossplane for Infrastructure

Crossplane lets me manage infrastructure through Kubernetes APIs. Here's the magic:

# Instead of direct kubectl apply... apiVersion: kubernetes.crossplane.io/v1alpha1 kind: Object metadata: name: metallb-ipaddresspool annotations: argocd.argoproj.io/sync-wave: "2" # 👈 Depends on provider spec: providerConfigRef: name: in-cluster forProvider: manifest: apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: homelab-pool namespace: metallb-system spec: addresses: - 10.20.0.81-10.20.0.99 # 🎯 My LoadBalancer IP range
Enter fullscreen mode Exit fullscreen mode

Why Crossplane Objects?

  • 🔄 Continuous reconciliation (drift detection)
  • 📊 Rich status reporting (health, errors)
  • 🎭 Dependency management (waits for providers)
  • 🔒 RBAC integration (secure access)

🎭 Step 3: Sync Waves for Dependencies

Order matters in infrastructure! I use sync waves to ensure proper sequencing:

# Wave -1: Root app-of-apps argocd.argoproj.io/sync-wave: "-1" # Wave 0: Install Crossplane providers argocd.argoproj.io/sync-wave: "0" # Wave 1: Provider configs + RBAC argocd.argoproj.io/sync-wave: "1" # Wave 2: Infrastructure resources argocd.argoproj.io/sync-wave: "2"
Enter fullscreen mode Exit fullscreen mode

Result: No more "CRD not found" or "provider not ready" errors! 🎉

🔐 Step 4: RBAC - The Critical Missing Piece

Crossplane needs permissions to manage your infrastructure. This is often overlooked:

apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: crossplane-provider-kubernetes-metallb rules: - apiGroups: ["metallb.io"] resources: ["ipaddresspools", "l2advertisements"] verbs: ["*"] --- # Bind to the provider's service account apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: crossplane-provider-kubernetes-metallb roleRef: kind: ClusterRole name: crossplane-provider-kubernetes-metallb subjects: - kind: ServiceAccount name: provider-kubernetes-xxxxx # 👈 Get from kubectl namespace: crossplane-system
Enter fullscreen mode Exit fullscreen mode

Pro tip: Restart provider pods after RBAC changes!

🧪 Testing the Complete Pipeline

Time for the moment of truth:

# 1. Bootstrap (one-time manual step) kubectl apply -f argocd/app-of-apps.yaml # 2. Check GitOps automation kubectl -n argocd get applications NAME SYNC STATUS HEALTH STATUS crossplane-provider-kubernetes Synced Healthy ✅ homelab-root Synced Healthy ✅ metallb-config Synced Healthy ✅ # 3. Verify infrastructure was created kubectl -n metallb-system get ipaddresspools.metallb.io NAME ADDRESSES homelab-pool ["10.20.0.81-10.20.0.99"] ✅ # 4. Test LoadBalancer functionality kubectl create deployment nginx --image=nginx kubectl expose deployment nginx --type=LoadBalancer --port=80 kubectl get svc nginx NAME TYPE EXTERNAL-IP PORT(S) nginx LoadBalancer 10.20.0.82 80:30114/TCP ✅ # 5. Verify connectivity curl http://10.20.0.82 <!DOCTYPE html> <html> <head><title>Welcome to nginx!</title></head> # 🎉 SUCCESS!
Enter fullscreen mode Exit fullscreen mode

It works! Infrastructure deployed entirely through GitOps! 🚀

📁 Repository Structure 

homelab-gitops/ ├── argocd/ │ └── app-of-apps.yaml # 🏠 Root application ├── apps/ │ ├── crossplane-provider-kubernetes-app.yaml │ └── metallb-config-app.yaml # 👶 Child applications ├── crossplane/ │ └── provider-kubernetes/ │ ├── provider.yaml # ⚡ Crossplane provider │ ├── providerconfig.yaml # ⚙️ Configuration │ └── rbac.yaml # 🔐 Permissions └── metallb/ ├── metallb-ipaddresspool.yaml # 🌐 Infrastructure └── metallb-l2advertisement.yaml # 📦 Resources
Enter fullscreen mode Exit fullscreen mode

💡 Key Lessons Learned

1. API Versions Are Critical

Different provider versions use different APIs:

# v0.13.0 uses v1alpha1 apiVersion: kubernetes.crossplane.io/v1alpha1 # Newer versions use v1alpha2  apiVersion: kubernetes.crossplane.io/v1alpha2
Enter fullscreen mode Exit fullscreen mode

2. Bootstrap is Still Manual

Even with full GitOps, you need one manual step:

kubectl apply -f argocd/app-of-apps.yaml
Enter fullscreen mode Exit fullscreen mode

After this, everything else is automated!

3. RBAC Debugging

If Crossplane objects stay "NotReady":

# Check provider permissions kubectl -n crossplane-system describe object metallb-ipaddresspool # Common fix: restart provider after RBAC changes kubectl -n crossplane-system delete pod -l pkg.crossplane.io/provider=provider-kubernetes
Enter fullscreen mode Exit fullscreen mode

4. YAML Formatting Matters

Watch your indentation! This kept my root app OutOfSync:

# ❌ Wrong destination: server: https://kubernetes.default.svc namespace: crossplane-system # ✅ Correct  destination: server: https://kubernetes.default.svc namespace: crossplane-system
Enter fullscreen mode Exit fullscreen mode

🚀 What's Next?

This foundation scales to manage ANY infrastructure:

# Database clusters kind: PostgreSQLCluster # Service meshes  kind: Istio # Monitoring stacks kind: PrometheusStack # Certificate management kind: ClusterIssuer # Storage solutions kind: StorageClass
Enter fullscreen mode Exit fullscreen mode

The pattern stays the same:

  1. Define in YAML
  2. Commit to Git
  3. Argo CD syncs automatically
  4. Crossplane provisions infrastructure
  5. Profit! 💰

🎉 Results

Before:

  • Manual kubectl commands
  • Configuration drift
  • No audit trail
  • Deployment anxiety 😰

After:

  • Infrastructure as Code
  • Git-driven deployments
  • Automatic drift correction
  • Pull request reviews
  • Confidence in production 😎

Infrastructure changes are now as simple as creating a pull request!

🔗 Resources

What infrastructure will you GitOps next? Drop a comment and let me know what you're planning to automate! 👇

Follow me for more cloud-native and DevOps content! 🚀

Top comments (1)

Collapse
 
anik_sikder_313 profile image
Anik Sikder •

This is a textbook example of how GitOps can evolve from a deployment strategy into full-stack infrastructure orchestration. The use of Crossplane as a control plane abstraction paired with Argo CD’s sync waves is a brilliant way to tame dependency hell and enforce sequencing without brittle scripts.

The App-of-Apps pattern adds clarity and scalability, especially in environments where infra and app lifecycles diverge. And the RBAC integration with Crossplane? That’s the kind of operational hygiene that often gets overlooked until it breaks something critical.

Appreciate how you’ve framed this not just as tooling, but as a mindset shift from manual ops to declarative, auditable infrastructure. Posts like this help teams move from “GitOps curious” to production-ready.