I am using externalTrafficPolicy set to Local for my a LoadBalancer service for an ingress controller on GKE.

Right now, when a pod gets terminated, it is immediately removed from the NodePort service, which stops traffic from routing to the pod (step 5 at https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods).

The problem is that the GCP Load Balancer doesn't update itself immediately, so it continues to send traffic to the NodePort even though Kubernetes has already removed the pod from the NodePort as part of the termination process. This results in timeouts and an inability to do zero downtime deployments when a node no longer has an active application residing on it when externalTrafficPolicy is set to Local.

I'd like to see an option where we can use Local, but allow for zero-downtime deployments.

I'm wondering if there could be a configurable option to wait until a preStop hook has finished (or grace period hits) before removing the pod from the NodePort service? With something like this, we could make a preStop hook that can make health checks fail but have the pod continue to serve traffic normally. The preStop hook could then sleep for a certain amount of time while the load balancers gracefully stop sending traffic because the health checks start to fail. Once the preStop hook completes, then it removes the pod from the NodePort. This would allow for graceful draining of outgoing pods.

Or maybe the answer is a pre-PreStop hook that can run before termination officially begins?