[run] add Pub/Sub events sample (#4011)

* Initialized run/events-pubsub dir Mostly copied contents from https://github.com/gcpevents/eventsforcloudrun/tree/master/python Signed-off-by: Curtis Mason <cumason@google.com> * Edited README Signed-off-by: Curtis Mason <cumason@google.com> * Added testing for events-pubsub sample code Signed-off-by: Curtis Mason <cumason@google.com> * added dockerignore Signed-off-by: Curtis Mason <cumason@google.com> * updated requirements.txt Signed-off-by: Curtis Mason <cumason@google.com> * fixed requirements.txt Signed-off-by: Curtis Mason <cumason@google.com> * Stable sample codebase Signed-off-by: Curtis Mason <cumason@google.com> * added identifiers Signed-off-by: Curtis Mason <cumason@google.com> * Updated events-pubsub README Signed-off-by: Curtis Mason <cumason@google.com> * Removed json.dumps Signed-off-by: Curtis Mason <cumason@google.com> * Reformatted README.md with 80char ruler Signed-off-by: Curtis Mason <cumason@google.com> * cloud run events-pubsub README changed message Signed-off-by: Curtis Mason <cumason@google.com> * events-pubsub added pytest to requirements Signed-off-by: Curtis Mason <cumason@google.com> * fixing linter imports events-pubsub * lint changing import order Signed-off-by: Curtis Mason <cumason@google.com> * requirements.txt with only flask and pytest requirements.txt has flask and pytest, and their codependencies Signed-off-by: Curtis Mason <cumason@google.com> * Further minimized requirements files Signed-off-by: Curtis Mason <cumason@google.com> * added flask to requirements-test Signed-off-by: Curtis Mason <cumason@google.com> * removed requirements-test.txt Signed-off-by: Curtis Mason <cumason@google.com> * added apache license Signed-off-by: Curtis Mason <cumason@google.com> * upgraded dockerfile to 3.8 Signed-off-by: Curtis Mason <cumason@google.com> * Update run/events-pubsub/README.md Co-authored-by: Averi Kitsch <akitsch@google.com> * codebase similar to pubsub samplecode * fixed README typo Signed-off-by: Curtis Mason <cumason@google.com> * Revised README and renamed app.py to main.py Signed-off-by: Curtis Mason <cumason@google.com> * added automated subscription update to README Signed-off-by: Curtis Mason <cumason@google.com> * Added SERVICE_ACCOUNT var to readme Signed-off-by: Curtis Mason <cumason@google.com> * Added more env variables to README Signed-off-by: Curtis Mason <cumason@google.com> * using python:3.8-slim in dockerfile * removed pytest from requirements.txt Signed-off-by: Curtis Mason <cumason@google.com> * removed Flask from requirements-test.txt * removed subscription= from readme Signed-off-by: Curtis Mason <cumason@google.com> * Documented --push-endpoint workaround in readme Signed-off-by: Curtis Mason <cumason@google.com> * Fixed linter in main.py * Finalized events-pubsub readme Signed-off-by: Curtis Mason <cumason@google.com> * removed duplicate region tag Signed-off-by: Curtis Mason <cumason@google.com> * removed port 8080 from dockerfile Signed-off-by: Curtis Mason <cumason@google.com> * Added required fields testing * Readme nitpick * lint fix: reordered imports * changed readme log command * added ce-id to response in main.py * Fixed README typo Co-authored-by: Averi Kitsch <akitsch@google.com>

Author: cumason123

run/events-pubsub/.dockerignore

@@ -0,0 +1,6 @@
+Dockerfile
+README.md
+*.pyc
+*.pyo
+*.pyd
+__pycache__

run/events-pubsub/Dockerfile

@@ -0,0 +1,41 @@
+# Copyright 2020 Google, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# [START run_events_pubsub_dockerfile]
+
+# Use the official Python image.
+# https://hub.docker.com/_/python
+FROM python:3.8-slim
+
+# Allow statements and log messages to immediately appear in the Cloud Run logs
+ENV PYTHONUNBUFFERED True
+
+# Copy application dependency manifests to the container image.
+# Copying this separately prevents re-running pip install on every code change.
+COPY requirements.txt ./
+
+# Install production dependencies.
+RUN pip install -r requirements.txt
+
+# Copy local code to the container image.
+ENV APP_HOME /app
+WORKDIR $APP_HOME
+COPY . ./
+
+# Run the web service on container startup. 
+# Use gunicorn webserver with one worker process and 8 threads.
+# For environments with multiple CPU cores, increase the number of workers
+# to be equal to the cores available.
+CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
+# [END run_events_pubsub_dockerfile]

run/events-pubsub/README.md

@@ -0,0 +1,70 @@
+# Events for Cloud Run – Pub/Sub tutorial
+
+This sample shows how to create a service that processes Pub/Sub events.
+
+## Setup
+
+Login to gcloud:
+
+```sh
+gcloud auth login
+```
+
+Configure project id:
+
+```sh
+gcloud config set project [PROJECT-ID]
+```
+
+Configure environment variables:
+
+```sh
+MY_RUN_SERVICE=pubsub-service
+MY_RUN_CONTAINER=pubsub-container
+MY_TOPIC=pubsub-topic
+MY_PUBSUB_TRIGGER=pubsub-trigger
+```
+
+## Quickstart
+
+Deploy your Cloud Run service:
+
+```sh
+gcloud builds submit \
+ --tag gcr.io/$(gcloud config get-value project)/$MY_RUN_CONTAINER
+gcloud run deploy $MY_RUN_SERVICE \
+ --image gcr.io/$(gcloud config get-value project)/$MY_RUN_CONTAINER \
+ --allow-unauthenticated
+```
+
+Create a Cloud Pub/Sub topic:
+
+```sh
+gcloud pubsub topics create $MY_TOPIC
+```
+
+Create a Cloud Pub/Sub trigger:
+
+```sh
+gcloud alpha events triggers create $MY_PUBSUB_TRIGGER \
+--target-service $MY_RUN_SERVICE \
+--type com.google.cloud.pubsub.topic.publish \
+--parameters topic=$MY_TOPIC
+```
+
+## Test
+
+Test your Cloud Run service by publishing a message to the topic: 
+
+```sh
+gcloud pubsub topics publish $MY_TOPIC --message="John Doe"
+```
+
+You may observe the Cloud Run service printing upon receiving an event in 
+Cloud Logging.
+
+```sh
+gcloud logging read "resource.type=cloud_run_revision AND \
+resource.labels.service_name=$MY_RUN_SERVICE" --project \
+$(gcloud config get-value project) --limit 30 --format 'value(textPayload)'
+```

run/events-pubsub/main.py

@@ -0,0 +1,64 @@
+# Copyright 2020 Google, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# [START run_events_pubsub_server_setup]
+import base64
+import os
+
+from flask import Flask, request
+
+
+required_fields = ['Ce-Id', 'Ce-Source', 'Ce-Type', 'Ce-Specversion']
+
+app = Flask(__name__)
+# [END run_events_pubsub_server_setup]
+
+
+# [START run_events_pubsub_handler]
+@app.route('/', methods=['POST'])
+def index():
+ for field in required_fields:
+ if field not in request.headers:
+ errmsg = f'Bad Request: missing required header {field}'
+ print(errmsg)
+ return errmsg, 400
+
+ envelope = request.get_json()
+ if not envelope:
+ msg = 'no Pub/Sub message received'
+ print(f'error: {msg}')
+ return f'Bad Request: {msg}', 400
+
+ if not isinstance(envelope, dict) or 'message' not in envelope:
+ msg = 'invalid Pub/Sub message format'
+ print(f'error: {msg}')
+ return f'Bad Request: {msg}', 400
+
+ pubsub_message = envelope['message']
+
+ name = 'World'
+ if isinstance(pubsub_message, dict) and 'data' in pubsub_message:
+ name = base64.b64decode(pubsub_message['data']).decode('utf-8').strip()
+
+ ce_id = request.headers.get('Ce-Id')
+ resp = f'Hello, {name}! ID: {ce_id}'
+ print(resp)
+ return (resp, 200)
+# [END run_events_pubsub_handler]
+
+
+# [START run_events_pubsub_server]
+if __name__ == "__main__":
+ app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))
+# [END run_events_pubsub_server]

run/events-pubsub/main_test.py

@@ -0,0 +1,81 @@
+# Copyright 2020 Google, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import base64
+
+import copy
+
+from uuid import uuid4
+
+import pytest
+
+import main
+
+
+required_fields = ['Ce-Id', 'Ce-Source', 'Ce-Type', 'Ce-Specversion']
+
+header_data = {field: str(uuid4()) for field in required_fields}
+
+
+@pytest.fixture
+def client():
+ main.app.testing = True
+ return main.app.test_client()
+
+
+def test_empty_payload(client):
+ r = client.post('/', json='', headers=header_data)
+ assert r.status_code == 400
+
+
+def test_invalid_payload(client):
+ r = client.post('/', json={'nomessage': 'invalid'}, headers=header_data)
+ assert r.status_code == 400
+
+
+def test_invalid_mimetype(client):
+ r = client.post('/', json="{ message: true }", headers=header_data)
+ assert r.status_code == 400
+
+
+def test_minimally_valid_message(client, capsys):
+ r = client.post('/', json={'message': True}, headers=header_data)
+ assert r.status_code == 200
+
+ out, _ = capsys.readouterr()
+ ce_id = header_data['Ce-Id']
+ assert f'Hello, World! ID: {ce_id}' in out
+
+
+def test_populated_message(client, capsys):
+ name = str(uuid4())
+ data = base64.b64encode(name.encode()).decode()
+
+ r = client.post('/', json={'message': {'data': data}}, headers=header_data)
+ assert r.status_code == 200
+
+ out, _ = capsys.readouterr()
+ ce_id = header_data['Ce-Id']
+ assert f'Hello, {name}! ID: {ce_id}' in out
+
+
+def test_missing_required_fields(client, capsys):
+ for field in required_fields:
+ test_headers = copy.copy(header_data)
+ test_headers.pop(field)
+
+ r = client.post('/', headers=test_headers)
+ assert r.status_code == 400
+
+ out, _ = capsys.readouterr()
+ assert f'Bad Request: missing required header {field}' in out

run/events-pubsub/requirements-test.txt

@@ -0,0 +1 @@
+pytest==5.4.3

run/events-pubsub/requirements.txt

@@ -0,0 +1,2 @@
+Flask==1.1.2
+gunicorn==20.0.4