Authentication Policy

This task covers the primary activities you might need to perform when enabling, configuring, and using Istio authentication policies. Find out more about the underlying concepts in the authentication overview.

Before you begin

• Understand Istio authentication policy and related mutual TLS authentication concepts.

• Have a Kubernetes cluster with Istio installed, without global mutual TLS enabled (e.g use install/kubernetes/istio-demo.yaml as described in installation steps, or set global.mtls.enabled to false using Helm).

Setup

Our examples use two namespaces foo and bar, with two services, httpbin and sleep, both running with an Envoy sidecar proxy. We also use second instances of httpbin and sleep running without the sidecar in the legacy namespace. If you’d like to use the same examples when trying the tasks, run the following:

$ kubectl create ns foo $ kubectl apply -f <(istioctl kube-inject -f @samples/httpbin/httpbin.yaml@) -n foo $ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@) -n foo $ kubectl create ns bar $ kubectl apply -f <(istioctl kube-inject -f @samples/httpbin/httpbin.yaml@) -n bar $ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@) -n bar $ kubectl create ns legacy $ kubectl apply -f @samples/httpbin/httpbin.yaml@ -n legacy $ kubectl apply -f @samples/sleep/sleep.yaml@ -n legacy 

You can verify setup by sending an HTTP request with curl from any sleep pod in the namespace foo, bar or legacy to either httpbin.foo, httpbin.bar or httpbin.legacy. All requests should succeed with HTTP code 200.

For example, here is a command to check sleep.bar to httpbin.foo reachability:

$ kubectl exec $(kubectl get pod -l app=sleep -n bar -o jsonpath={.items..metadata.name}) -c sleep -n bar -- curl http://httpbin.foo:8000/ip -s -o /dev/null -w "%{http_code}\n" 200 

This one-liner command conveniently iterates through all reachability combinations:

$ for from in "foo" "bar" "legacy"; do for to in "foo" "bar" "legacy"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.foo: 200 sleep.foo to httpbin.bar: 200 sleep.foo to httpbin.legacy: 200 sleep.bar to httpbin.foo: 200 sleep.bar to httpbin.bar: 200 sleep.bar to httpbin.legacy: 200 sleep.legacy to httpbin.foo: 200 sleep.legacy to httpbin.bar: 200 sleep.legacy to httpbin.legacy: 200 

You should also verify that there is a default mesh authentication policy in the system, which you can do as follows:

$ kubectl get policies.authentication.istio.io --all-namespaces No resources found. 

$ kubectl get meshpolicies.authentication.istio.io NAME AGE default 3m 

Last but not least, verify that there are no destination rules that apply on the example services. You can do this by checking the host: value of existing destination rules and make sure they do not match. For example:

$ kubectl get destinationrules.networking.istio.io --all-namespaces -o yaml | grep "host:" host: istio-policy.istio-system.svc.cluster.local host: istio-telemetry.istio-system.svc.cluster.local 

Globally enabling Istio mutual TLS

To set a mesh-wide authentication policy that enables mutual TLS, submit mesh authentication policy like below:

$ kubectl apply -f - <<EOF apiVersion: "authentication.istio.io/v1alpha1" kind: "MeshPolicy" metadata: name: "default" spec: peers: - mtls: {} EOF 

This policy specifies that all workloads in the mesh will only accept encrypted requests using TLS. As you can see, this authentication policy has the kind: MeshPolicy. The name of the policy must be default, and it contains no targets specification (as it is intended to apply to all services in the mesh).

At this point, only the receiving side is configured to use mutual TLS. If you run the curl command between Istio services (i.e those with sidecars), all requests will fail with a 503 error code as the client side is still using plain-text.

$ for from in "foo" "bar"; do for to in "foo" "bar"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.foo: 503 sleep.foo to httpbin.bar: 503 sleep.bar to httpbin.foo: 503 sleep.bar to httpbin.bar: 503 

To configure the client side, you need to set destination rules to use mutual TLS. It’s possible to use multiple destination rules, one for each applicable service (or namespace). However, it’s more convenient to use a rule with the * wildcard to match all services so that it is on par with the mesh-wide authentication policy.

$ kubectl apply -f - <<EOF apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "default" namespace: "istio-system" spec: host: "*.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF 

Don’t forget that destination rules are also used for non-auth reasons such as setting up canarying, but the same order of precedence applies. So if a service requires a specific destination rule for any reason - for example, for a configuration load balancer - the rule must contain a similar TLS block with ISTIO_MUTUAL mode, as otherwise it will override the mesh- or namespace-wide TLS settings and disable TLS.

Re-running the testing command as above, you will see all requests between Istio-services are now completed successfully:

$ for from in "foo" "bar"; do for to in "foo" "bar"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.foo: 200 sleep.foo to httpbin.bar: 200 sleep.bar to httpbin.foo: 200 sleep.bar to httpbin.bar: 200 

Request from non-Istio services to Istio services

The non-Istio service, e.g sleep.legacy doesn’t have a sidecar, so it cannot initiate the required TLS connection to Istio services. As a result, requests from sleep.legacy to httpbin.foo or httpbin.bar will fail:

$ for from in "legacy"; do for to in "foo" "bar"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.legacy to httpbin.foo: 000 command terminated with exit code 56 sleep.legacy to httpbin.bar: 000 command terminated with exit code 56 

This works as intended, and unfortunately, there is no solution for this without reducing authentication requirements for these services.

Request from Istio services to non-Istio services

Try to send requests to httpbin.legacy from sleep.foo (or sleep.bar). You will see requests fail as Istio configures clients as instructed in our destination rule to use mutual TLS, but httpbin.legacy does not have a sidecar so it’s unable to handle it.

$ for from in "foo" "bar"; do for to in "legacy"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.legacy: 503 sleep.bar to httpbin.legacy: 503 

To fix this issue, we can add a destination rule to overwrite the TLS setting for httpbin.legacy. For example:

$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: "httpbin-legacy" namespace: "legacy" spec: host: "httpbin.legacy.svc.cluster.local" trafficPolicy: tls: mode: DISABLE EOF 

Test it again after you add the destination rule to ensure it passes:

$ for from in "foo" "bar"; do for to in "legacy"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.legacy: 200 sleep.bar to httpbin.legacy: 200 

Request from Istio services to Kubernetes API server

The Kubernetes API server doesn’t have a sidecar, thus request from Istio services such as sleep.foo will fail due to the same problem as when sending requests to any non-Istio service.

$ TOKEN=$(kubectl describe secret $(kubectl get secrets | grep default-token | cut -f1 -d ' ' | head -1) | grep -E '^token' | cut -f2 -d':' | tr -d '\t') $ kubectl exec $(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name}) -c sleep -n foo -- curl https://kubernetes.default/api --header "Authorization: Bearer $TOKEN" --insecure -s -o /dev/null -w "%{http_code}\n" 000 command terminated with exit code 35 

Again, we can correct this by overriding the destination rule for the API server (kubernetes.default)

$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: "api-server" namespace: istio-system spec: host: "kubernetes.default.svc.cluster.local" trafficPolicy: tls: mode: DISABLE EOF 

Re-run the testing command above to confirm that it returns 200 after the rule is added:

$ TOKEN=$(kubectl describe secret $(kubectl get secrets | grep default-token | cut -f1 -d ' ' | head -1) | grep -E '^token' | cut -f2 -d':' | tr -d '\t') $ kubectl exec $(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name}) -c sleep -n foo -- curl https://kubernetes.default/api --header "Authorization: Bearer $TOKEN" --insecure -s -o /dev/null -w "%{http_code}\n" 200 

Cleanup part 1

Remove global authentication policy and destination rules added in the session:

$ kubectl delete meshpolicy default $ kubectl delete destinationrules httpbin-legacy -n legacy $ kubectl delete destinationrules api-server -n istio-system $ kubectl delete destinationrules default -n istio-system 

Enable mutual TLS per namespace or service

In addition to specifying an authentication policy for your entire mesh, Istio also lets you specify policies for particular namespaces or services. A namespace-wide policy takes precedence over the mesh-wide policy, while a service-specific policy has higher precedence still.

Namespace-wide policy

The example below shows the policy to enable mutual TLS for all services in namespace foo. As you can see, it uses kind: “Policy” rather than “MeshPolicy”, and specifies a namespace, in this case, foo. If you don’t specify a namespace value the policy will apply to the default namespace.

$ kubectl apply -f - <<EOF apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "default" namespace: "foo" spec: peers: - mtls: {} EOF 

Add corresponding destination rule:

$ kubectl apply -f - <<EOF apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "default" namespace: "foo" spec: host: "*.foo.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF 

As these policy and destination rule are applied on services in namespace foo only, you should see only request from client-without-sidecar (sleep.legacy) to httpbin.foo start to fail.

$ for from in "foo" "bar" "legacy"; do for to in "foo" "bar" "legacy"; do kubectl exec $(kubectl get pod -l app=sleep -n ${from} -o jsonpath={.items..metadata.name}) -c sleep -n ${from} -- curl "http://httpbin.${to}:8000/ip" -s -o /dev/null -w "sleep.${from} to httpbin.${to}: %{http_code}\n"; done; done sleep.foo to httpbin.foo: 200 sleep.foo to httpbin.bar: 200 sleep.foo to httpbin.legacy: 200 sleep.bar to httpbin.foo: 200 sleep.bar to httpbin.bar: 200 sleep.bar to httpbin.legacy: 200 sleep.legacy to httpbin.foo: 000 command terminated with exit code 56 sleep.legacy to httpbin.bar: 200 sleep.legacy to httpbin.legacy: 200 

Service-specific policy

You can also set authentication policy and destination rule for a specific service. Run this command to set another policy only for httpbin.bar service.

$ cat <<EOF | kubectl apply -n bar -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "httpbin" spec: targets: - name: httpbin peers: - mtls: {} EOF 

And a destination rule:

$ cat <<EOF | kubectl apply -n bar -f - apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "httpbin" spec: host: "httpbin.bar.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF 

Again, run the probing command. As expected, request from sleep.legacy to httpbin.bar starts failing with the same reasons.

... sleep.legacy to httpbin.bar: 000 command terminated with exit code 56 

If we have more services in namespace bar, we should see traffic to them won’t be affected. Instead of adding more services to demonstrate this behavior, we edit the policy slightly to apply on a specific port:

$ cat <<EOF | kubectl apply -n bar -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "httpbin" spec: targets: - name: httpbin ports: - number: 1234 peers: - mtls: {} EOF 

And a corresponding change to the destination rule:

$ cat <<EOF | kubectl apply -n bar -f - apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "httpbin" spec: host: httpbin.bar.svc.cluster.local trafficPolicy: tls: mode: DISABLE portLevelSettings: - port: number: 1234 tls: mode: ISTIO_MUTUAL EOF 

This new policy will apply only to the httpbin service on port 1234. As a result, mutual TLS is disabled (again) on port 8000 and requests from sleep.legacy will resume working.

$ kubectl exec $(kubectl get pod -l app=sleep -n legacy -o jsonpath={.items..metadata.name}) -c sleep -n legacy -- curl http://httpbin.bar:8000/ip -s -o /dev/null -w "%{http_code}\n" 200 

Policy precedence

To illustrate how a service-specific policy takes precedence over namespace-wide policy, you can add a policy to disable mutual TLS for httpbin.foo as below. Note that you’ve already created a namespace-wide policy that enables mutual TLS for all services in namespace foo and observe that requests from sleep.legacy to httpbin.foo are failing (see above).

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "overwrite-example" spec: targets: - name: httpbin EOF 

and destination rule:

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "overwrite-example" spec: host: httpbin.foo.svc.cluster.local trafficPolicy: tls: mode: DISABLE EOF 

Re-running the request from sleep.legacy, you should see a success return code again (200), confirming service-specific policy overrides the namespace-wide policy.

$ kubectl exec $(kubectl get pod -l app=sleep -n legacy -o jsonpath={.items..metadata.name}) -c sleep -n legacy -- curl http://httpbin.foo:8000/ip -s -o /dev/null -w "%{http_code}\n" 200 

Cleanup part 2

Remove policies and destination rules created in the above steps:

$ kubectl delete policy default overwrite-example -n foo $ kubectl delete policy httpbin -n bar $ kubectl delete destinationrules default overwrite-example -n foo $ kubectl delete destinationrules httpbin -n bar 

End-user authentication

To experiment with this feature, you need a valid JWT. The JWT must correspond to the JWKS endpoint you want to use for the demo. In this tutorial, we use this JWT test and this JWKS endpoint from the Istio code base.

Also, for convenience, expose httpbin.foo via ingressgateway (for more details, see the ingress task).

$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: httpbin-gateway namespace: foo spec: selector: istio: ingressgateway # use Istio default gateway implementation servers: - port: number: 80 name: http protocol: HTTP hosts: - "*" EOF 

$ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: httpbin namespace: foo spec: hosts: - "*" gateways: - httpbin-gateway http: - route: - destination: port: number: 8000 host: httpbin.foo.svc.cluster.local EOF 

Get ingress IP

$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') 

And run a test query

$ curl $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n" 200 

Now, add a policy that requires end-user JWT for httpbin.foo. The next command assumes there is no service-specific policy for httpbin.foo (which should be the case if you run cleanup as described). You can run kubectl get policies.authentication.istio.io -n foo to confirm.

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "jwt-example" spec: targets: - name: httpbin origins: - jwt: issuer: "testing@secure.istio.io" jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/jwks.json" principalBinding: USE_ORIGIN EOF 

The same curl command from before will return with 401 error code, as a result of server is expecting JWT but none was provided:

$ curl $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n" 401 

Attaching the valid token generated above returns success:

$ TOKEN=$(curl https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/demo.jwt -s) $ curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n" 200 

To observe other aspects of JWT validation, use the script gen-jwt.py to generate new tokens to test with different issuer, audiences, expiry date, etc. The script can be downloaded from the Istio repository:

$ wget https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/gen-jwt.py $ chmod +x gen-jwt.py 

You also need the key.pem file:

$ wget https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/key.pem 

For example, the command below creates a token that expires in 5 seconds. As you see, Istio authenticates requests using that token successfully at first but rejects them after 5 seconds:

$ TOKEN=$(./gen-jwt.py ./key.pem --expire 5) $ for i in `seq 1 10`; do curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n"; sleep 1; done 200 200 200 200 200 401 401 401 401 401 

You can also add a JWT policy to an ingress gateway (e.g., service istio-ingressgateway.istio-system.svc.cluster.local). This is often used to define a JWT policy for all services bound to the gateway, instead of for individual services.

End-user authentication with per-path requirements

End-user authentication can be enabled or disabled based on request path. This is useful if you want to disable authentication for some paths, for example, the path used for health check or status report. You can also specify different JWT requirements on different paths.

Disable End-user authentication for specific paths

Modify the jwt-example policy to disable End-user authentication for path /user-agent:

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "jwt-example" spec: targets: - name: httpbin origins: - jwt: issuer: "testing@secure.istio.io" jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/jwks.json" trigger_rules: - excluded_paths: - exact: /user-agent principalBinding: USE_ORIGIN EOF 

Confirm it’s allowed to access the path /user-agent without JWT tokens:

$ curl $INGRESS_HOST/user-agent -s -o /dev/null -w "%{http_code}\n" 200 

Confirm it’s denied to access paths other than /user-agent without JWT tokens:

$ curl $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n" 401 

Enable End-user authentication for specific paths

Modify the jwt-example policy to enable End-user authentication only for path /ip:

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "jwt-example" spec: targets: - name: httpbin origins: - jwt: issuer: "testing@secure.istio.io" jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/jwks.json" trigger_rules: - included_paths: - exact: /ip principalBinding: USE_ORIGIN EOF 

Confirm it’s allowed to access paths other than /ip without JWT tokens:

$ curl $INGRESS_HOST/user-agent -s -o /dev/null -w "%{http_code}\n" 200 

Confirm it’s denied to access the path /ip without JWT tokens:

$ curl $INGRESS_HOST/ip -s -o /dev/null -w "%{http_code}\n" 401 

Confirm it’s allowed to access the path /ip with a valid JWT token:

$ TOKEN=$(curl https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/demo.jwt -s) $ curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST/ip -s -o /dev/null -w "%{http_code}\n" 200 

End-user authentication with mutual TLS

End-user authentication and mutual TLS can be used together. Modify the policy above to define both mutual TLS and end-user JWT authentication:

$ cat <<EOF | kubectl apply -n foo -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "jwt-example" spec: targets: - name: httpbin peers: - mtls: {} origins: - jwt: issuer: "testing@secure.istio.io" jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/jwks.json" principalBinding: USE_ORIGIN EOF 

And add a destination rule:

$ kubectl apply -f - <<EOF apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "httpbin" namespace: "foo" spec: host: "httpbin.foo.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF 

After these changes, traffic from Istio services, including ingress gateway, to httpbin.foo will use mutual TLS. The test command above will still work. Requests from Istio services directly to httpbin.foo also work, given the correct token:

$ TOKEN=$(curl https://raw.githubusercontent.com/istio/istio/release-1.3/security/tools/jwt/samples/demo.jwt -s) $ kubectl exec $(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name}) -c sleep -n foo -- curl http://httpbin.foo:8000/ip -s -o /dev/null -w "%{http_code}\n" --header "Authorization: Bearer $TOKEN" 200 

However, requests from non-Istio services, which use plain-text will fail:

$ kubectl exec $(kubectl get pod -l app=sleep -n legacy -o jsonpath={.items..metadata.name}) -c sleep -n legacy -- curl http://httpbin.foo:8000/ip -s -o /dev/null -w "%{http_code}\n" --header "Authorization: Bearer $TOKEN" 000 command terminated with exit code 56 

Cleanup part 3

1. Remove authentication policy:

   $ kubectl -n foo delete policy jwt-example 

2. Remove destination rule:

   $ kubectl -n foo delete destinationrule httpbin 

3. If you are not planning to explore any follow-on tasks, you can remove all resources simply by deleting test namespaces.

   $ kubectl delete ns foo bar legacy