This repository focuses on mastering containerization technologies using Docker, Kubernetes, and related tools to build, manage, and automate container infrastructure for Internal Development Platform (IDP) development.
By working through this repository, you will:
- Master Docker container creation and management
- Implement multi-stage builds and optimization
- Work with Docker Compose for multi-container applications
- Understand Kubernetes architecture and concepts
- Deploy and manage applications on Kubernetes
- Implement container security best practices
- Build CI/CD pipelines for containerized applications
- Python 3.11 or higher
- Docker Desktop installed
- kubectl installed
- Basic understanding of Linux and networking
- Completed learning-idp-python-azure-sdk
- Git and GitHub account
learning-idp-containerization/ ├── README.md # This file ├── REFERENCES.md # Links to resources and related repos ├── pyproject.toml # Python project configuration ├── requirements.txt # Python dependencies ├── requirements-dev.txt # Development dependencies ├── .python-version # Python version for pyenv ├── .gitignore # Git ignore patterns ├── .env.example # Environment variables template │ ├── docs/ │ ├── concepts/ │ │ ├── 01-containerization-overview.md │ │ ├── 02-docker-fundamentals.md │ │ ├── 03-kubernetes-architecture.md │ │ ├── 04-container-networking.md │ │ ├── 05-container-storage.md │ │ └── 06-container-security.md │ ├── guides/ │ │ ├── getting-started.md │ │ ├── dockerfile-best-practices.md │ │ ├── kubernetes-deployment.md │ │ ├── helm-charts.md │ │ └── container-registry.md │ └── examples/ │ ├── simple-dockerfile.md │ ├── multi-stage-build.md │ ├── kubernetes-deployment.md │ ├── stateful-application.md │ └── microservices-architecture.md │ ├── src/ │ ├── __init__.py │ │ │ ├── core/ │ │ ├── __init__.py │ │ ├── authentication.py # Docker/K8s authentication │ │ ├── config.py # Configuration management │ │ ├── exceptions.py # Custom exceptions │ │ └── logging_config.py # Logging setup │ │ │ ├── docker/ │ │ ├── __init__.py │ │ ├── docker_client.py # Docker API client │ │ ├── image_builder.py # Image building │ │ ├── container_manager.py # Container management │ │ ├── network_manager.py # Network operations │ │ └── volume_manager.py # Volume operations │ │ │ ├── compose/ │ │ ├── __init__.py │ │ ├── compose_manager.py # Docker Compose operations │ │ ├── service_scaler.py # Service scaling │ │ └── health_checker.py # Health monitoring │ │ │ ├── kubernetes/ │ │ ├── __init__.py │ │ ├── k8s_client.py # Kubernetes client │ │ ├── deployment_manager.py # Deployment operations │ │ ├── service_manager.py # Service management │ │ ├── pod_manager.py # Pod operations │ │ └── namespace_manager.py # Namespace management │ │ │ ├── helm/ │ │ ├── __init__.py │ │ ├── helm_client.py # Helm operations │ │ ├── chart_manager.py # Chart management │ │ └── release_manager.py # Release operations │ │ │ ├── registry/ │ │ ├── __init__.py │ │ ├── acr_manager.py # Azure Container Registry │ │ ├── dockerhub_manager.py # Docker Hub operations │ │ └── registry_scanner.py # Security scanning │ │ │ ├── security/ │ │ ├── __init__.py │ │ ├── image_scanner.py # Image security scanning │ │ ├── secret_manager.py # Secret management │ │ ├── policy_enforcer.py # Security policies │ │ └── rbac_manager.py # RBAC configuration │ │ │ └── monitoring/ │ ├── __init__.py │ ├── metrics_collector.py # Container metrics │ ├── log_aggregator.py # Log collection │ └── health_monitor.py # Health checks │ ├── examples/ │ ├── 01_docker_basics/ │ │ ├── 01_simple_dockerfile/ │ │ │ ├── Dockerfile │ │ │ ├── app.py │ │ │ └── requirements.txt │ │ ├── 02_multi_stage_build/ │ │ │ ├── Dockerfile │ │ │ └── app.py │ │ ├── 03_docker_compose/ │ │ │ ├── docker-compose.yml │ │ │ ├── web/Dockerfile │ │ │ └── db/init.sql │ │ ├── 04_networking/ │ │ │ └── docker-compose.yml │ │ └── 05_volumes/ │ │ └── docker-compose.yml │ │ │ ├── 02_docker_python/ │ │ ├── 01_flask_app/ │ │ │ ├── Dockerfile │ │ │ ├── app.py │ │ │ └── requirements.txt │ │ ├── 02_fastapi_app/ │ │ │ ├── Dockerfile │ │ │ ├── main.py │ │ │ └── requirements.txt │ │ ├── 03_celery_worker/ │ │ │ ├── docker-compose.yml │ │ │ ├── worker/Dockerfile │ │ │ └── tasks.py │ │ └── 04_ml_model/ │ │ ├── Dockerfile │ │ └── model.py │ │ │ ├── 03_kubernetes_basics/ │ │ ├── 01_pod.yaml │ │ ├── 02_deployment.yaml │ │ ├── 03_service.yaml │ │ ├── 04_configmap.yaml │ │ ├── 05_secret.yaml │ │ └── 06_ingress.yaml │ │ │ ├── 04_kubernetes_advanced/ │ │ ├── 01_statefulset.yaml │ │ ├── 02_daemonset.yaml │ │ ├── 03_job.yaml │ │ ├── 04_cronjob.yaml │ │ ├── 05_hpa.yaml │ │ └── 06_network_policy.yaml │ │ │ ├── 05_helm_charts/ │ │ ├── simple-app/ │ │ │ ├── Chart.yaml │ │ │ ├── values.yaml │ │ │ └── templates/ │ │ │ ├── deployment.yaml │ │ │ ├── service.yaml │ │ │ └── ingress.yaml │ │ └── microservices/ │ │ ├── Chart.yaml │ │ ├── values.yaml │ │ └── templates/ │ │ │ ├── 06_container_registry/ │ │ ├── 01_push_to_acr.py │ │ ├── 02_scan_image.py │ │ ├── 03_manage_tags.py │ │ └── 04_cleanup_old_images.py │ │ │ ├── 07_security/ │ │ ├── 01_image_scanning.py │ │ ├── 02_secret_management.py │ │ ├── 03_network_policies.yaml │ │ ├── 04_pod_security.yaml │ │ └── 05_rbac_setup.yaml │ │ │ └── 08_cicd/ │ ├── 01_github_actions/ │ │ └── docker-build.yml │ ├── 02_azure_devops/ │ │ └── azure-pipelines.yml │ └── 03_gitops/ │ └── argocd-application.yaml │ ├── templates/ │ ├── dockerfiles/ │ │ ├── python-base.Dockerfile │ │ ├── python-alpine.Dockerfile │ │ ├── nodejs-base.Dockerfile │ │ └── multi-stage.Dockerfile │ ├── compose/ │ │ ├── web-db-redis.yml │ │ ├── microservices.yml │ │ └── monitoring-stack.yml │ ├── kubernetes/ │ │ ├── deployment-template.yaml │ │ ├── service-template.yaml │ │ └── ingress-template.yaml │ └── helm/ │ └── chart-skeleton/ │ ├── notebooks/ │ ├── 01_docker_basics.ipynb │ ├── 02_kubernetes_intro.ipynb │ ├── 03_helm_charts.ipynb │ ├── 04_security_practices.ipynb │ └── 05_monitoring_containers.ipynb │ ├── scripts/ │ ├── setup_docker.sh # Docker setup script │ ├── setup_kubernetes.sh # Kubernetes setup │ ├── build_images.sh # Build all images │ ├── deploy_to_k8s.sh # Deploy to Kubernetes │ └── cleanup.sh # Resource cleanup │ ├── tests/ │ ├── __init__.py │ ├── conftest.py │ ├── unit/ │ │ ├── test_docker_client.py │ │ ├── test_k8s_client.py │ │ └── test_helm_client.py │ └── integration/ │ ├── test_container_lifecycle.py │ ├── test_deployment.py │ └── test_networking.py │ └── .github/ └── workflows/ ├── docker-build.yml # Docker build workflow ├── k8s-deploy.yml # Kubernetes deployment └── security-scan.yml # Security scanning git clone https://github.com/vanHeemstraSystems/learning-idp-containerization.git cd learning-idp-containerization# Install Docker Desktop # Download from: https://www.docker.com/products/docker-desktop # Install kubectl curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl # Install Helm curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash # Set up Python environment python3 -m venv venv source venv/bin/activate pip install -r requirements.txt# Build a simple Docker image cd examples/01_docker_basics/01_simple_dockerfile docker build -t my-first-app . # Run the container docker run -p 8000:8000 my-first-app # Test it curl http://localhost:8000# Enable Kubernetes in Docker Desktop or use Minikube minikube start # Deploy a simple application kubectl apply -f examples/03_kubernetes_basics/02_deployment.yaml kubectl apply -f examples/03_kubernetes_basics/03_service.yaml # Check the deployment kubectl get pods kubectl get servicesFollow this recommended sequence:
Day 1-2: Docker Basics
- Read
docs/concepts/02-docker-fundamentals.md - Work through
examples/01_docker_basics/ - Practice building and running containers
Day 3-4: Dockerfile Best Practices
- Study
docs/guides/dockerfile-best-practices.md - Complete examples in
examples/02_docker_python/ - Implement multi-stage builds
Day 5-7: Docker Compose
- Work through Docker Compose examples
- Build multi-container applications
- Practice service networking
Day 1-3: Kubernetes Architecture
- Read
docs/concepts/03-kubernetes-architecture.md - Complete examples in
examples/03_kubernetes_basics/ - Deploy applications to Kubernetes
Day 4-7: Advanced Kubernetes
- Work through
examples/04_kubernetes_advanced/ - Implement StatefulSets and DaemonSets
- Configure autoscaling
Day 1-3: Container Registry
- Complete examples in
examples/06_container_registry/ - Push images to Azure Container Registry
- Implement image scanning
Day 4-7: Security Best Practices
- Study
docs/concepts/06-container-security.md - Work through
examples/07_security/ - Implement security policies
Day 1-3: Helm Charts
- Read
docs/guides/helm-charts.md - Create custom Helm charts
- Deploy with Helm
Day 4-7: CI/CD Integration
- Work through
examples/08_cicd/ - Build container CI/CD pipelines
- Implement GitOps workflows
# Docker docker>=24.0.0 # Kubernetes kubectl>=1.28.0 # Helm helm>=3.13.0 # Python Libraries docker>=7.0.0 # Docker SDK for Python kubernetes>=28.1.0 # Kubernetes Python client# Multi-stage build for Python application FROM python:3.11-slim as builder WORKDIR /app # Install dependencies COPY requirements.txt . RUN pip install --user --no-cache-dir -r requirements.txt # Final stage FROM python:3.11-slim WORKDIR /app # Copy dependencies from builder COPY --from=builder /root/.local /root/.local # Copy application code COPY . . # Make sure scripts in .local are usable ENV PATH=/root/.local/bin:$PATH # Non-root user RUN useradd -m appuser && chown -R appuser:appuser /app USER appuser EXPOSE 8000 CMD ["python", "app.py"]version: '3.8' services: web: build: ./web ports: - "8000:8000" environment: - DATABASE_URL=postgresql://db:5432/myapp - REDIS_URL=redis://redis:6379 depends_on: - db - redis networks: - app-network healthcheck: test: ["CMD", "curl", "-f", "http://localhost:8000/health"] interval: 30s timeout: 10s retries: 3 db: image: postgres:16-alpine environment: - POSTGRES_DB=myapp - POSTGRES_PASSWORD=secretpassword volumes: - postgres-data:/var/lib/postgresql/data networks: - app-network redis: image: redis:7-alpine volumes: - redis-data:/data networks: - app-network volumes: postgres-data: redis-data: networks: app-network: driver: bridgefrom kubernetes import client, config # Load kubeconfig config.load_kube_config() # Create API clients apps_v1 = client.AppsV1Api() core_v1 = client.CoreV1Api() # Define deployment deployment = client.V1Deployment( metadata=client.V1ObjectMeta(name="my-app"), spec=client.V1DeploymentSpec( replicas=3, selector=client.V1LabelSelector( match_labels={"app": "my-app"} ), template=client.V1PodTemplateSpec( metadata=client.V1ObjectMeta( labels={"app": "my-app"} ), spec=client.V1PodSpec( containers=[ client.V1Container( name="my-app", image="myregistry.azurecr.io/my-app:latest", ports=[client.V1ContainerPort(container_port=8000)], resources=client.V1ResourceRequirements( requests={"cpu": "100m", "memory": "128Mi"}, limits={"cpu": "500m", "memory": "512Mi"} ), liveness_probe=client.V1Probe( http_get=client.V1HTTPGetAction( path="/health", port=8000 ), initial_delay_seconds=10, period_seconds=10 ) ) ] ) ) ) ) # Create deployment apps_v1.create_namespaced_deployment( namespace="default", body=deployment ) # Create service service = client.V1Service( metadata=client.V1ObjectMeta(name="my-app"), spec=client.V1ServiceSpec( selector={"app": "my-app"}, ports=[client.V1ServicePort(port=80, target_port=8000)], type="LoadBalancer" ) ) core_v1.create_namespaced_service( namespace="default", body=service ) print("Deployment and service created successfully")import os from pathlib import Path def create_helm_chart(chart_name: str, output_dir: str): """Create a basic Helm chart structure""" chart_path = Path(output_dir) / chart_name # Create directory structure (chart_path / "templates").mkdir(parents=True, exist_ok=True) # Chart.yaml chart_yaml = f"""apiVersion: v2 name: {chart_name} description: A Helm chart for {chart_name} type: application version: 0.1.0 appVersion: "1.0" """ # values.yaml values_yaml = """replicaCount: 3 image: repository: myregistry.azurecr.io/my-app tag: latest pullPolicy: IfNotPresent service: type: ClusterIP port: 80 ingress: enabled: false resources: limits: cpu: 500m memory: 512Mi requests: cpu: 100m memory: 128Mi autoscaling: enabled: true minReplicas: 2 maxReplicas: 10 targetCPUUtilizationPercentage: 80 """ # Write files (chart_path / "Chart.yaml").write_text(chart_yaml) (chart_path / "values.yaml").write_text(values_yaml) print(f"Helm chart created at: {chart_path}") # Create chart create_helm_chart("my-app", "./helm-charts")import docker import json def scan_image(image_name: str): """Scan Docker image for vulnerabilities using Trivy""" client = docker.from_env() # Pull image if not exists try: client.images.get(image_name) except docker.errors.ImageNotFound: print(f"Pulling image: {image_name}") client.images.pull(image_name) # Run Trivy scan result = client.containers.run( "aquasec/trivy:latest", f"image --format json {image_name}", remove=True, volumes={ "/var/run/docker.sock": { "bind": "/var/run/docker.sock", "mode": "ro" } } ) # Parse results scan_results = json.loads(result) # Count vulnerabilities vulnerabilities = { "CRITICAL": 0, "HIGH": 0, "MEDIUM": 0, "LOW": 0 } for target in scan_results.get("Results", []): for vuln in target.get("Vulnerabilities", []): severity = vuln.get("Severity", "UNKNOWN") if severity in vulnerabilities: vulnerabilities[severity] += 1 print(f"\nVulnerability Scan Results for {image_name}:") print(f"CRITICAL: {vulnerabilities['CRITICAL']}") print(f"HIGH: {vulnerabilities['HIGH']}") print(f"MEDIUM: {vulnerabilities['MEDIUM']}") print(f"LOW: {vulnerabilities['LOW']}") return vulnerabilities # Scan image scan_image("python:3.11-slim")# Use specific version tags FROM python:3.11-slim # Set working directory WORKDIR /app # Copy only requirements first (layer caching) COPY requirements.txt . # Install dependencies RUN pip install --no-cache-dir -r requirements.txt # Copy application code COPY . . # Use non-root user RUN useradd -m appuser && chown -R appuser:appuser /app USER appuser # Document exposed ports EXPOSE 8000 # Use exec form for CMD CMD ["python", "app.py"]resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "512Mi"livenessProbe: httpGet: path: /health port: 8000 initialDelaySeconds: 10 periodSeconds: 10 readinessProbe: httpGet: path: /ready port: 8000 initialDelaySeconds: 5 periodSeconds: 5# Run as non-root USER 1000:1000 # Read-only root filesystem securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000- learning-internal-development-platform - Main overview
- learning-idp-python-azure-sdk - Azure SDK fundamentals
- learning-idp-azure-compute - AKS integration
- learning-idp-cicd-pipelines - Container CI/CD
- learning-idp-observability - Container monitoring
This is a personal learning repository, but suggestions and improvements are welcome!
- Fork the repository
- Create a feature branch
- Make your changes with tests
- Ensure all tests pass
- Submit a pull request
This project is for educational purposes. See LICENSE file for details.
Willem van Heemstra
- GitHub: @vanHeemstraSystems
- LinkedIn: Willem van Heemstra
Last updated: December 18, 2025 Part of the learning-internal-development-platform series