Skip to content

ComputeEngineCredentials does not handle error response from the metadata server correctly #1409

@jianglai

Description

@jianglai

ComputeEngineCredentials assumes that the response from from the metadata server is either 200 or 503:

try { response = request.execute(); } catch (UnknownHostException exception) { throw new IOException( "ComputeEngineCredentials cannot find the metadata server. This is" + " likely because code is not running on Google Compute Engine.", exception); } if (response.getStatusCode() == 503) { throw GoogleAuthException.createWithTokenEndpointResponseException( new HttpResponseException(response)); } return response;

This is not necessarily true. For example, on GKE, when Workload Identity Federation is used, one needs to bind the k8s service account to an IAM service account in order to receive an OIDC token by calling idTokenWithAudience. Without the binding, the metadata server returns a 404 with the following body:

Your Kubernetes service account (default/nomulus) is not annotated with a target Google service account, which is a requirement for retrieving Identity Tokens using Workload Identity. Please add the [iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_ID](http://iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_ID)] annotation to your Kubernetes service account. Refer to [https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity?authuser=0) 

The 404 code is silently ignored and the response body (i. e. the error message above) was then treated as the base64-encoded ID token, which causes the decoding step to fail:

Caused by: java.lang.IllegalArgumentException: com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20	at com.google.common.io.BaseEncoding.decode(BaseEncoding.java:222)	at com.google.api.client.util.Base64.decodeBase64(Base64.java:106)	at com.google.api.client.json.webtoken.JsonWebSignature$Parser.parse(JsonWebSignature.java:545)	at com.google.api.client.json.webtoken.JsonWebSignature.parse(JsonWebSignature.java:479)	at com.google.auth.oauth2.IdToken.create(IdToken.java:80)	at com.google.auth.oauth2.IdToken.create(IdToken.java:68)	at com.google.auth.oauth2.ComputeEngineCredentials.idTokenWithAudience(ComputeEngineCredentials.java:335)	at google.registry.util.OidcTokenUtils.createOidcToken(OidcTokenUtils.java:57)	at google.registry.proxy.ProxyModule.lambda$provideOidcToken$1(ProxyModule.java:278)	at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:300)	at google.registry.proxy.handler.HttpsRelayServiceHandler.decodeFullHttpRequest(HttpsRelayServiceHandler.java:105)	at google.registry.proxy.handler.WhoisServiceHandler.decodeFullHttpRequest(WhoisServiceHandler.java:49)	at google.registry.proxy.handler.HttpsRelayServiceHandler.decode(HttpsRelayServiceHandler.java:129)	at io.netty.handler.codec.ByteToMessageCodec$1.decode(ByteToMessageCodec.java:42)	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)	... 30 more Caused by: com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20	at com.google.common.io.BaseEncoding$Alphabet.decode(BaseEncoding.java:541)	at com.google.common.io.BaseEncoding$Base64Encoding.decodeTo(BaseEncoding.java:1037)	at com.google.common.io.BaseEncoding.decodeChecked(BaseEncoding.java:237)	at com.google.common.io.BaseEncoding.decode(BaseEncoding.java:220)	... 45 more 

I had to manually send recreate the HTTP request to the metadata server to obtain the response body and understand the problem, as java.lang.IllegalArgumentException: com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20 is not helpful at all.

In my opinion getMetadataResponse should throw (with the message body, if present) whenever the response code is not 200, instead of only throwing for 503.

Metadata

Metadata

Assignees

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions