docs: add documentation and sample for GKE sidecar proxy (#718)

Adds documentation and sample for running PGAdapter as a sidecar proxy on GKE. Fixes #701

Author: olavloite

README.md

@@ -35,7 +35,9 @@ in-process server (the latter is only supported for Java applications).
 
 ### Docker
 
-See [running PGAdapter using Docker](docs/docker.md) for more examples for running PGAdapter in Docker.
+* See [running PGAdapter using Docker](docs/docker.md) for more examples for running PGAdapter in Docker.
+* See [running PGAdapter as a sidecar proxy](docs/sidecar-proxy.md) for how to run PGAdapter as a
+ sidecar proxy in a Kubernetes cluster.
 
 Replace the project, instance and database names and the credentials file in the example below to
 run PGAdapter from a pre-built Docker image.

docs/sidecar-proxy.md

@@ -0,0 +1,44 @@
+# Google Cloud Spanner PGAdapter - Sidecar Proxy
+
+PGAdapter can be used as a sidecar proxy in for example a Kubernetes cluster. Kubernetes sidecar
+containers are those containers that run parallel with the main container in the pod.
+
+Running PGAdapter in a "sidecar" pattern is recommend over running as a separate service for several reasons:
+
+* Prevents a single point of failure - each application's access to your database is independent
+ of the others, making it more resilient.
+* Allows you to scope resource requests more accurately - because PGAdapter consumes resources
+ linearly to usage, this pattern allows you to more accurately scope and request resources to match
+ your applications as it scales.
+
+## Configuration
+
+Add PGAdapter to the pod configuration under `containers`:
+
+```yaml
+ containers:
+ - name: pgadapter
+ image: gcr.io/cloud-spanner-pg-adapter/pgadapter
+ ports:
+ - containerPort: 5432
+ args:
+ - "-p my-project"
+ - "-i my-instance"
+ - "-d my-database"
+ - "-x"
+ resources:
+ requests:
+ # PGAdapter's memory use scales linearly with the number of active
+ # connections. Fewer open connections will use less memory. Adjust
+ # this value based on your application's requirements.
+ # A minimum of 384MiB + 2MiB per open connection is recommended.
+ memory: "512Mi"
+ # PGAdapter's CPU use scales linearly with the amount of IO between
+ # the database and the application. Adjust this value based on your
+ # application's requirements.
+ cpu: "1"
+```
+
+[This sample application](../samples/sidecar-proxy) contains a step-by-step guide for how to set up
+PGAdapter as a sidecar proxy in a Google Kubernetes cluster. This example also includes setting up
+a Workload Identity with a Kubernetes Service Account.

samples/sidecar-proxy/Dockerfile

@@ -0,0 +1,27 @@
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM golang:1.19.2 as builder
+WORKDIR /app
+RUN go mod init example-app
+RUN go get github.com/jackc/pgx/v4
+COPY *.go ./
+RUN CGO_ENABLED=0 GOOS=linux go build -o /example-app
+
+FROM gcr.io/distroless/base-debian11
+WORKDIR /
+COPY --from=builder /example-app /example-app
+ENV PORT 8080
+USER nonroot:nonroot
+CMD ["/example-app"]

samples/sidecar-proxy/README.md

@@ -0,0 +1,160 @@
+# Using PGAdapter on Kubernetes
+
+PGAdapter can be used as a sidecar proxy on Kubernetes. This directory contains a small sample
+application that shows how to set this up.
+
+## Creating a GKE cluster
+
+These steps create a GKE cluster where the sample app can be deployed. You can skip these
+steps if you already have a GKE cluster, and instead modify your existing deployment to
+include PGAdapter.
+
+See https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster for more details on
+how to set up a GKE cluster.
+
+1. Create an Autopilot cluster named example-cluster and update it to use Workload Identity.
+
+```shell
+export PROJECT_ID=<YOUR_PROJECT_ID>
+export REGION=<YOUR_COMPUTE_REGION>
+
+gcloud config set project $PROJECT_ID
+gcloud container clusters create-auto example-cluster \
+ --region=$REGION
+gcloud container clusters update example-cluster \
+ --region=$REGION \
+ --workload-pool=$PROJECT_ID.svc.id.goog
+```
+
+2. Get authentication credentials for the cluster.
+
+```shell
+gcloud container clusters get-credentials example-cluster \
+ --region $REGION
+```
+
+This command configures `kubectl` to use the cluster you created.
+
+3. Create a repository and build the sample application
+
+```shell
+gcloud artifacts repositories create example-repo \
+ --repository-format=docker \
+ --location=$REGION \
+ --description="Example Docker repository"
+docker build --platform linux/amd64 \
+ -t ${REGION}-docker.pkg.dev/${PROJECT_ID}/example-repo/example-app .
+```
+
+4. Authenticate and push the Docker image to Artifact Registry
+
+```shell
+gcloud auth configure-docker ${REGION}-docker.pkg.dev
+docker push ${REGION}-docker.pkg.dev/${PROJECT_ID}/example-repo/example-app
+```
+
+## Setting up a service account
+
+The first step to running PGAdapter in Kubernetes is creating a Google Cloud IAM
+service account (GSA) to represent your application. It is recommended that you create
+a service account unique to each application, instead of using the same service
+account everywhere. This model is more secure since it allows you to limit
+permissions on a per-application basis.
+
+The service account must be granted access to the Cloud Spanner database that the
+application will be connecting to. It is enough to assign the `Cloud Spanner Database User`
+role for this example to the service account.
+
+```shell
+export GSA_NAME=pgadapter-example-gsa
+gcloud iam service-accounts create $GSA_NAME \
+ --description="PGAdapter Example Service Account" \
+ --display-name="PGAdapter Example Service Account"
+gcloud projects add-iam-policy-binding $PROJECT_ID \
+ --member="serviceAccount:${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
+ --role="roles/spanner.databaseUser"
+```
+
+## Providing the service account to PGAdapter
+
+Next, you need to configure Kubernetes to provide the service account to PGAdapter.
+This example uses GKE's [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity)
+for this. This method allows you to bind a [Kubernetes Service Account (KSA)](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/)
+to a Google Service Account (GSA). The GSA will then be accessible to applications
+using the matching KSA.
+
+1. Create a Kubernetes Service Account (KSA)
+
+```shell
+export KSA_NAME=pgadapter-example-ksa
+kubectl create serviceaccount $KSA_NAME --namespace default
+```
+
+2. Enable the IAM binding between your `$GSA_NAME` and `$KSA_NAME`:
+
+```sh
+gcloud iam service-accounts add-iam-policy-binding \
+ --role roles/iam.workloadIdentityUser \
+ --member "serviceAccount:${PROJECT_ID}.svc.id.goog[default/$KSA_NAME]" \
+ ${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
+```
+
+3. Add an annotation to `$KSA_NAME` to complete the binding:
+
+```sh
+kubectl annotate serviceaccount \
+ $KSA_NAME \
+ iam.gke.io/gcp-service-account=${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
+```
+
+## Deploying the application
+
+1. Deploy the example application and PGAdapter to Kubernetes. The script below replaces the
+ placeholders in the deployment manifest with your values.
+
+```shell
+export SPANNER_PROJECT_ID=$PROJECT_ID
+export SPANNER_INSTANCE_ID=<YOUR_SPANNER_INSTANCE_ID>
+export SPANNER_DATABASE_ID=<YOUR_SPANNER_DATABASE_ID>
+
+sed -e 's|<YOUR_KSA_NAME>|'"${KSA_NAME}"'|g' \
+ -e 's|<YOUR_REGION>|'"${REGION}"'|g' \
+ -e 's|<YOUR_PROJECT_ID>|'"${PROJECT_ID}"'|g' \
+ -e 's|<YOUR_SPANNER_PROJECT_ID>|'"${SPANNER_PROJECT_ID}"'|g' \
+ -e 's|<YOUR_SPANNER_INSTANCE_ID>|'"${SPANNER_INSTANCE_ID}"'|g' \
+ -e 's|<YOUR_SPANNER_DATABASE_ID>|'"${SPANNER_DATABASE_ID}"'|g' \
+ manifests/example-app-deployment.yaml > manifests/my-app-deployment.yaml
+
+kubectl apply -f manifests/my-app-deployment.yaml
+```
+
+2. Expose the deployment to the Internet
+
+```shell
+kubectl expose deployment pgadapter-gke-example \
+ --type LoadBalancer --port 80 --target-port 8080
+```
+
+## Inspect and view the application
+
+1. Inspect the running Pods by using `kubectl get pods`
+
+```shell
+kubectl get pods
+```
+
+2. Inspect the pgadapter-gke-example Service by using `kubectl get service`
+
+```shell
+kubectl get service pgadapter-gke-example
+```
+
+From this command's output, copy the Service's external IP address from the EXTERNAL-IP column.
+
+3. View the output of the application
+
+Use the EXTERNAL-IP address from the previous command.
+
+```shell
+curl http://<EXTERNAL-IP>
+```

samples/sidecar-proxy/main.go

@@ -0,0 +1,65 @@
+/**
+ * Copyright 2023 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package main
+
+import (
+"context"
+"fmt"
+"log"
+"net/http"
+"os"
+
+"github.com/jackc/pgx/v4"
+)
+
+func main() {
+// register greet function to handle all requests
+mux := http.NewServeMux()
+mux.HandleFunc("/", greet)
+
+// use PORT environment variable, or default to 8080
+port := os.Getenv("PORT")
+if port == "" {
+port = "8080"
+}
+
+// start the web server on port and accept requests
+log.Printf("Server listening on port %s", port)
+log.Fatal(http.ListenAndServe(":"+port, mux))
+}
+
+// greet connects to PGAdapter and responds to the request with a greeting from Cloud Spanner.
+func greet(w http.ResponseWriter, r *http.Request) {
+log.Printf("Serving request: %s", r.URL.Path)
+host, _ := os.Hostname()
+fmt.Fprintf(w, "Hostname: %s\n", host)
+
+conn, err := pgx.Connect(context.Background(), "postgres://uid:pwd@127.0.0.1:5432/?sslmode=disable")
+if err != nil {
+fmt.Fprintf(w, "Unable to connect to database: %v\n", err)
+return
+}
+defer conn.Close(context.Background())
+
+var greeting string
+err = conn.QueryRow(context.Background(), "select 'Greeting from PGAdapter!'").Scan(&greeting)
+if err != nil {
+fmt.Fprintf(w, "QueryRow failed: %v\n", err)
+return
+}
+fmt.Fprintln(w, greeting)
+}

samples/sidecar-proxy/manifests/example-app-deployment.yaml

@@ -0,0 +1,55 @@
+
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: pgadapter-gke-example
+spec:
+ selector:
+ matchLabels:
+ app: pgadapter-example
+ template:
+ metadata:
+ labels:
+ app: pgadapter-example
+ spec:
+ # TODO(developer): replace this value with the actual Kubernetes Service Account name
+ serviceAccountName: <YOUR_KSA_NAME>
+ containers:
+ - name: example-app
+ # TODO(developer): replace these values
+ image: <YOUR_REGION>-docker.pkg.dev/<YOUR_PROJECT_ID>/example-repo/example-app
+ imagePullPolicy: Always
+ ports:
+ - containerPort: 8080
+ resources:
+ requests:
+ cpu: 200m
+ - name: pgadapter
+ image: gcr.io/cloud-spanner-pg-adapter/pgadapter
+ ports:
+ - containerPort: 5432
+ # TODO(developer): Replace with actual project, instance and database name
+ args:
+ - "-p <YOUR_SPANNER_PROJECT_ID>"
+ - "-i <YOUR_SPANNER_INSTANCE_ID>"
+ - "-d <YOUR_SPANNER_DATABASE_ID>"
+ - "-x"
+ resources:
+ requests:
+ memory: "512Mi"
+ cpu: 200m
+---