NGINX KIC in Cloud Native Kubernetes From Network to Identity Layer Koen Vanderpoorten Solution Engineer BELUX
| ©2020 F5 2 PUBLIC CLOUD KUBERNETES
| ©2020 F5 3 So everything is perfect… …or is it not?
| ©2020 F5 4 Challenge of k8s loadbalancer services in public clouds? Users Operational, Financial and Security Challenges Cloud Native Kubernetes
| ©2020 F5 5 • Accepts traffic from outside the Kubernetes platform, and load-balances it to pods (containers) running inside the platform • One single IP address and load balancer for multiple applications, routing is based on URI’s (L7 info) • Monitors the pods running in Kubernetes, and automatically updates the load balancing rules if, for example, pods are added or removed from a service The Ingress Controller Internal Network Users Ingress Controller A specialized load balancer for Kubernetes environments: Load Balancer
| ©2020 F5 6 Ingress and Annotations – the pros and cons apiVersion: extensions/v1beta1 kind: Ingress metadata: name: webapp annotations: nginx.org/lb-method: "ip_hash" nginx.org/ssl-services: "webapp" nginx.org/proxy-connect-timeout: "10s" nginx.org/proxy-read-timeout: "10s" nginx.org/proxy-send-timeout: "10s" nginx.org/rewrites: "serviceName=webapp rewrite=/v1" nginx.com/jwt-key: "webapp-jwk" nginx.com/jwt-realm: "Webb App" nginx.com/jwt-token: "$cookie_auth_token" nginx.com/jwt-login-url: "https://login.example.com"" spec: rules: - host: webapp.example.com . . . Implement features missing in Ingress spec Lack validation Not suited for granular configuration Good Things: Can quickly grow bigger than the spec Bad Things: Not portable
| ©2020 F5 7 Enter NGINX KIC CRD’s
| ©2021 F5 8 Host TLS Upstreams Routes - Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional Host TLS Upstreams Routes - Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional NGINX Ingress Resources – Rich Capabilities Host TLS Policies Upstreams Routes - Path Policies Action Split Match Route ErrorPage pass redirect return proxy delegation VirtualServer pass redirect return proxy pass redirect return proxy Host Upstreams Subroutes - Path Policies Action Split Match ErrorPage pass redirect return proxy VirtualServerRoute NGINX server configuration NGINX http configuration Server and HTTP snippets NGINX location configuration Location snippets Policies Access Control Rate Limiting Auth (JWT, OIDC) MTLS (Ingress/Egress) App Protect WAF
| ©2020 F5 9 Conditional Routing Split Routing Error Pages Rate Limiting Authentication Web Application Firewall NGINX Snippets Multiple versions, multiple clients A|B Testing for safe production deployment Implement ‘Circuit Breakers’ to contain failures Protect vulnerable apps, limit greedy clients Offload identity checking from apps and centralize Protect from known and unknown vulnerabilities “to-the-metal” with NGINX configuration Use Cases
| ©2021 F5 10 NGINX Ingress Resources – Distributed Configuration
| ©2021 F5 11 NGINX Ingress Resources – Distributed Configuration NetOps DevOps-FE DevOps-NG Identity DevSecOps
| ©2021 F5 12 NGINX Ingress Controller WITH KUBERNETES & NGINX INGRESS CONTROLLER Manage Complexity in Production Apps Teams Basic Ingress Solutions 1 Many Many 1
| ©2020 F5 13 Why not use the Community version of NGINX Ingress Controller? WAIT, THERE’S MORE THAN ONE? Footprint Latency Timeouts Security Community: 500MB NGINX Plus: 120MB Community: Slowed by timeouts NGINX Plus: Dynamically reconfigures Community: 8809 NGINX Plus: 0 Community: OpenResty = CVE problems NGINX Plus: Proactive CVE patching, Integrated WAF & service mesh Driven by innovation at the expense of feature stability Driven by enterprise-ready stability without compromising innovation
| ©2020 F5 14 2020 PERFORMANCE TEST RESULTS Latency in a Dynamic Deployment
| ©2020 F5 15 UNNECESSARY RISK IN YOUR K8S ENVIRONMENTS Delays in CVE patching Competitor
| ©2021 F5 16 THE COST OF ADDING SECURITY IS LOW COMPARED WITH ITS IMPACT NGINX App Protect Performance 0 0,5 1 1,5 2 2,5 Throughput (MB/sec) No Protection NGINX App Protect ModSec 0 2000 4000 6000 8000 10000 12000 14000 Requests/sec No Protection NGINX App Protect ModSec 0 100 200 300 400 500 600 700 800 Latency (ms) No Protection NGINX App Protect ModSec
| ©2021 F5 17 AppProtect + KIC + NSM = Mesh Ingress WAF • AppProtect runs in NGINX+ KIC at the edge • Provides L7 WAF security for all mesh-based ingress • mTLS between KIC and NSM NGINX AppProtect
| ©2021 F5 18 Mutual TLS Instrumentation Tracing Rate Limiting Traffic Splitting Access Control Egress Control Secure traffic in a zero-trust network Monitor performance, latency and availability Debug transactions and locate faults Protect vulnerable apps, limit greedy clients Implement A|B, Canary and Blue-Green upgrades Implement allow-lists to accurately govern traffic Manage and broker traffic to external services Spotlight on Service Mesh - Use Cases
| ©2021 F5 19 Demo Time
Deploying NGINX in Cloud Native Kubernetes

Deploying NGINX in Cloud Native Kubernetes

  • 1.
    NGINX KIC inCloud Native Kubernetes From Network to Identity Layer Koen Vanderpoorten Solution Engineer BELUX
  • 2.
    | ©2020 F5 2 PUBLICCLOUD KUBERNETES
  • 3.
    | ©2020 F5 3 Soeverything is perfect… …or is it not?
  • 4.
    | ©2020 F5 4 Challengeof k8s loadbalancer services in public clouds? Users Operational, Financial and Security Challenges Cloud Native Kubernetes
  • 5.
    | ©2020 F5 5 •Accepts traffic from outside the Kubernetes platform, and load-balances it to pods (containers) running inside the platform • One single IP address and load balancer for multiple applications, routing is based on URI’s (L7 info) • Monitors the pods running in Kubernetes, and automatically updates the load balancing rules if, for example, pods are added or removed from a service The Ingress Controller Internal Network Users Ingress Controller A specialized load balancer for Kubernetes environments: Load Balancer
  • 6.
    | ©2020 F5 6 Ingressand Annotations – the pros and cons apiVersion: extensions/v1beta1 kind: Ingress metadata: name: webapp annotations: nginx.org/lb-method: "ip_hash" nginx.org/ssl-services: "webapp" nginx.org/proxy-connect-timeout: "10s" nginx.org/proxy-read-timeout: "10s" nginx.org/proxy-send-timeout: "10s" nginx.org/rewrites: "serviceName=webapp rewrite=/v1" nginx.com/jwt-key: "webapp-jwk" nginx.com/jwt-realm: "Webb App" nginx.com/jwt-token: "$cookie_auth_token" nginx.com/jwt-login-url: "https://login.example.com"" spec: rules: - host: webapp.example.com . . . Implement features missing in Ingress spec Lack validation Not suited for granular configuration Good Things: Can quickly grow bigger than the spec Bad Things: Not portable
  • 7.
    | ©2020 F5 7 EnterNGINX KIC CRD’s
  • 8.
    | ©2021 F5 8 Host TLS Upstreams Routes -Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional Host TLS Upstreams Routes - Path Action Split Match Route ErrorPage pass redirect return proxy delegation optional NGINX Ingress Resources – Rich Capabilities Host TLS Policies Upstreams Routes - Path Policies Action Split Match Route ErrorPage pass redirect return proxy delegation VirtualServer pass redirect return proxy pass redirect return proxy Host Upstreams Subroutes - Path Policies Action Split Match ErrorPage pass redirect return proxy VirtualServerRoute NGINX server configuration NGINX http configuration Server and HTTP snippets NGINX location configuration Location snippets Policies Access Control Rate Limiting Auth (JWT, OIDC) MTLS (Ingress/Egress) App Protect WAF
  • 9.
    | ©2020 F5 9 ConditionalRouting Split Routing Error Pages Rate Limiting Authentication Web Application Firewall NGINX Snippets Multiple versions, multiple clients A|B Testing for safe production deployment Implement ‘Circuit Breakers’ to contain failures Protect vulnerable apps, limit greedy clients Offload identity checking from apps and centralize Protect from known and unknown vulnerabilities “to-the-metal” with NGINX configuration Use Cases
  • 10.
    | ©2021 F5 10 NGINXIngress Resources – Distributed Configuration
  • 11.
    | ©2021 F5 11 NGINXIngress Resources – Distributed Configuration NetOps DevOps-FE DevOps-NG Identity DevSecOps
  • 12.
    | ©2021 F5 12 NGINX IngressController WITH KUBERNETES & NGINX INGRESS CONTROLLER Manage Complexity in Production Apps Teams Basic Ingress Solutions 1 Many Many 1
  • 13.
    | ©2020 F5 13 Whynot use the Community version of NGINX Ingress Controller? WAIT, THERE’S MORE THAN ONE? Footprint Latency Timeouts Security Community: 500MB NGINX Plus: 120MB Community: Slowed by timeouts NGINX Plus: Dynamically reconfigures Community: 8809 NGINX Plus: 0 Community: OpenResty = CVE problems NGINX Plus: Proactive CVE patching, Integrated WAF & service mesh Driven by innovation at the expense of feature stability Driven by enterprise-ready stability without compromising innovation
  • 14.
    | ©2020 F5 14 2020PERFORMANCE TEST RESULTS Latency in a Dynamic Deployment
  • 15.
    | ©2020 F5 15 UNNECESSARYRISK IN YOUR K8S ENVIRONMENTS Delays in CVE patching Competitor
  • 16.
    | ©2021 F5 16 THECOST OF ADDING SECURITY IS LOW COMPARED WITH ITS IMPACT NGINX App Protect Performance 0 0,5 1 1,5 2 2,5 Throughput (MB/sec) No Protection NGINX App Protect ModSec 0 2000 4000 6000 8000 10000 12000 14000 Requests/sec No Protection NGINX App Protect ModSec 0 100 200 300 400 500 600 700 800 Latency (ms) No Protection NGINX App Protect ModSec
  • 17.
    | ©2021 F5 17 AppProtect+ KIC + NSM = Mesh Ingress WAF • AppProtect runs in NGINX+ KIC at the edge • Provides L7 WAF security for all mesh-based ingress • mTLS between KIC and NSM NGINX AppProtect
  • 18.
    | ©2021 F5 18 MutualTLS Instrumentation Tracing Rate Limiting Traffic Splitting Access Control Egress Control Secure traffic in a zero-trust network Monitor performance, latency and availability Debug transactions and locate faults Protect vulnerable apps, limit greedy clients Implement A|B, Canary and Blue-Green upgrades Implement allow-lists to accurately govern traffic Manage and broker traffic to external services Spotlight on Service Mesh - Use Cases
  • 19.