Connect to a Redis instance from a Compute Engine VM

You can connect to your Redis instance from Compute Engine VM instances that use the same authorized network as your Redis instance.

Setup

If you have already installed the Google Cloud CLI, created a Redis instance, and created a Cloud Storage bucket you can skip these steps.

  1. Install the gcloud CLI and initialize:

    gcloud init 
  2. Follow the Quickstart Guide to create a Redis instance. Take note of the zone, IP address, and port of the Redis instance.

  3. Create a Cloud Storage bucket where the application artifact for this sample application will be uploaded. For more, see Create buckets.

Configuring the gcloud settings for the sample application

  1. Set the gcloud default project to your sample app project.
    gcloud config set project [PROJECT_ID]

Sample application

This sample HTTP server application establishes a connection to a Redis instance from a Compute Engine VM instance.

Clone the repository for your desired programming language and navigate to the folder that contains the sample code:

Go

git clone https://github.com/GoogleCloudPlatform/golang-samples cd golang-samples/memorystore/redis 

Java

git clone https://github.com/GoogleCloudPlatform/java-docs-samples cd java-docs-samples/memorystore/redis 

Node.js

git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples cd nodejs-docs-samples/memorystore/redis 

Python

git clone https://github.com/GoogleCloudPlatform/python-docs-samples cd python-docs-samples/memorystore/redis 

This sample application increments a Redis counter every time the / endpoint is accessed.

Go

This application uses the github.com/gomodule/redigo/redis client. Install it by running the following command:

go get github.com/gomodule/redigo/redis 
 // Command redis is a basic app that connects to a managed Redis instance. package main import ( "fmt" "log" "net/http" "os" "github.com/gomodule/redigo/redis" ) var redisPool *redis.Pool func incrementHandler(w http.ResponseWriter, r *http.Request) { conn := redisPool.Get() defer conn.Close() counter, err := redis.Int(conn.Do("INCR", "visits")) if err != nil { http.Error(w, "Error incrementing visitor counter", http.StatusInternalServerError) return } fmt.Fprintf(w, "Visitor number: %d", counter) } func main() { redisHost := os.Getenv("REDISHOST") redisPort := os.Getenv("REDISPORT") redisAddr := fmt.Sprintf("%s:%s", redisHost, redisPort) const maxConnections = 10 redisPool = &redis.Pool{ MaxIdle: maxConnections, Dial: func() (redis.Conn, error) { return redis.Dial("tcp", redisAddr) }, } http.HandleFunc("/", incrementHandler) port := os.Getenv("PORT") if port == "" { port = "8080" } log.Printf("Listening on port %s", port) if err := http.ListenAndServe(":"+port, nil); err != nil { log.Fatal(err) } } 

Java

This application is Jetty 3.1 servlet-based.

It uses the Jedis library:

<dependency>  <groupId>redis.clients</groupId>  <artifactId>jedis</artifactId>  <version>5.1.0</version> </dependency>

The AppServletContextListener class is used to create a long-lived Redis connection pool:

 package com.example.redis; import java.io.IOException; import java.util.Properties; import javax.servlet.ServletContextEvent; import javax.servlet.ServletContextListener; import javax.servlet.annotation.WebListener; import redis.clients.jedis.JedisPool; import redis.clients.jedis.JedisPoolConfig; @WebListener public class AppServletContextListener implements ServletContextListener {  private Properties config = new Properties();  private JedisPool createJedisPool() throws IOException {  String host;  Integer port;  config.load(  Thread.currentThread()  .getContextClassLoader()  .getResourceAsStream("application.properties"));  host = config.getProperty("redis.host");  port = Integer.valueOf(config.getProperty("redis.port", "6379"));  JedisPoolConfig poolConfig = new JedisPoolConfig();  // Default : 8, consider how many concurrent connections into Redis you will need under load  poolConfig.setMaxTotal(128);  return new JedisPool(poolConfig, host, port);  }  @Override  public void contextDestroyed(ServletContextEvent event) {  JedisPool jedisPool = (JedisPool) event.getServletContext().getAttribute("jedisPool");  if (jedisPool != null) {  jedisPool.destroy();  event.getServletContext().setAttribute("jedisPool", null);  }  }  // Run this before web application is started  @Override  public void contextInitialized(ServletContextEvent event) {  JedisPool jedisPool = (JedisPool) event.getServletContext().getAttribute("jedisPool");  if (jedisPool == null) {  try {  jedisPool = createJedisPool();  event.getServletContext().setAttribute("jedisPool", jedisPool);  } catch (IOException e) {  // handle exception  }  }  } }

The VisitCounterServlet class is a web servlet that increments a Redis counter:

 package com.example.redis; import java.io.IOException; import java.net.SocketException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import redis.clients.jedis.Jedis; import redis.clients.jedis.JedisPool; @WebServlet(name = "Track visits", value = "") public class VisitCounterServlet extends HttpServlet {  @Override  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {  try {  JedisPool jedisPool = (JedisPool) req.getServletContext().getAttribute("jedisPool");  if (jedisPool == null) {  throw new SocketException("Error connecting to Jedis pool");  }  Long visits;  try (Jedis jedis = jedisPool.getResource()) {  visits = jedis.incr("visits");  }  resp.setStatus(HttpServletResponse.SC_OK);  resp.getWriter().println("Visitor counter: " + String.valueOf(visits));  } catch (Exception e) {  resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());  }  } }

Node.js

This application uses the redis module.

{  "name": "memorystore-redis",  "description": "An example of using Memorystore(Redis) with Node.js",  "version": "0.0.1",  "private": true,  "license": "Apache Version 2.0",  "author": "Google Inc.",  "engines": {  "node": ">=16.0.0"  },  "dependencies": {  "redis": "^4.0.0"  } } 
'use strict'; const http = require('http'); const redis = require('redis'); const REDISHOST = process.env.REDISHOST || 'localhost'; const REDISPORT = process.env.REDISPORT || 6379; const client = redis.createClient(REDISPORT, REDISHOST); client.on('error', err => console.error('ERR:REDIS:', err)); // create a server http  .createServer((req, res) => {  // increment the visit counter  client.incr('visits', (err, reply) => {  if (err) {  console.log(err);  res.status(500).send(err.message);  return;  }  res.writeHead(200, {'Content-Type': 'text/plain'});  res.end(`Visitor number: ${reply}\n`);  });  })  .listen(8080);

Python

This application uses Flask for web serving and the redis-py package to communicate with the Redis instance.

Flask==3.0.3 gunicorn==23.0.0 redis==6.0.0 Werkzeug==3.0.3
import logging import os from flask import Flask import redis app = Flask(__name__) redis_host = os.environ.get("REDISHOST", "localhost") redis_port = int(os.environ.get("REDISPORT", 6379)) redis_client = redis.StrictRedis(host=redis_host, port=redis_port) @app.route("/") def index(): value = redis_client.incr("counter", 1) return f"Visitor number: {value}" @app.errorhandler(500) def server_error(e): logging.exception("An error occurred during a request.") return (  """  An internal error occurred: <pre>{}</pre>  See logs for full stacktrace.  """.format( e ), 500, ) if __name__ == "__main__": # This is used when running locally. Gunicorn is used to run the # application on Google App Engine and Cloud Run. # See entrypoint in app.yaml or Dockerfile. app.run(host="127.0.0.1", port=8080, debug=True)

Deploying the application to a Compute Engine VM

Navigate to the gce_deployment directory:

cd gce_deployment 

The deployment script uploads the artifact to the Cloud Storage path. It then launches a Compute Engine instance, creating a firewall to expose the port 8080. Then the startup script prepares the instance.

Set the REDISHOST and REDISPORT environmental variables:

 export REDISHOST=[REDISHOST] export REDISPORT=[REDISPORT] 

where:

  • REDISHOST is the managed Redis instance IP.
  • REDISPORT is the managed Redis instance port by default 6379.

Set the GCS_BUCKET_NAME environmental variable:

 export GCS_BUCKET_NAME=[BUCKET_NAME]/[PATH]

where:

  • BUCKET_NAME is the name of your Cloud Storage bucket.
  • PATH is an optional path for the directory where you want to store the application artifact.

Here is a sample deployment script that deploys this application on a new Compute Engine VM instance.

Go

if [ -z "$REDISHOST" ]; then  echo "Must set \$REDISHOST. For example: REDISHOST=127.0.0.1"  exit 1 fi if [ -z "$REDISPORT" ]; then  echo "Must set \$REDISPORT. For example: REDISPORT=6379"  exit 1 fi if [ -z "$GCS_BUCKET_NAME" ]; then  echo "Must set \$GCS_BUCKET_NAME. For example: GCS_BUCKET_NAME=my-bucket"  exit 1 fi if [ -z "$ZONE" ]; then  ZONE=$(gcloud config get-value compute/zone -q)  echo "$ZONE" fi # Cross compile the app for linux/amd64 GOOS=linux GOARCH=amd64 go build -v -o app ../main.go # Add the app binary tar -cvf app.tar app # Copy to GCS bucket gsutil cp app.tar gs://"$GCS_BUCKET_NAME"/gce/ # Create an instance gcloud compute instances create my-instance \  --image-family=debian-9 \  --image-project=debian-cloud \  --machine-type=g1-small \  --scopes cloud-platform \  --metadata-from-file startup-script=startup-script.sh \  --metadata gcs-bucket="$GCS_BUCKET_NAME",redis-host="$REDISHOST",redis-port="$REDISPORT" \  --zone "$ZONE" \  --tags http-server gcloud compute firewall-rules create allow-http-server-8080 \  --allow tcp:8080 \  --source-ranges 0.0.0.0/0 \  --target-tags http-server \  --description "Allow port 8080 access to http-server"

Java

if [ -z "$GCS_BUCKET_NAME" ]; then  echo "Must set \$GCS_BUCKET_NAME. For example: GCS_BUCKET_NAME=my-bucket"  exit 1 fi if [ -z "$ZONE" ]; then  ZONE=$(gcloud config get-value compute/zone -q)  echo $ZONE fi if [ -z "$WAR" ]; then  WAR=visitcounter-1.0-SNAPSHOT.war fi #Build the WAR package cd .. mvn clean package #Copy the WAR artifact to the GCS bucket location gsutil cp -r target/${WAR} gs://"$GCS_BUCKET_NAME"/gce/ cd gce_deployment # Create an instance gcloud compute instances create my-instance \  --image-family=debian-9 \  --image-project=debian-cloud \  --machine-type=g1-small \  --scopes cloud-platform \  --metadata-from-file startup-script=startup-script.sh \  --metadata gcs-bucket=$GCS_BUCKET_NAME,app-war=$WAR \  --zone $ZONE \  --tags http-server gcloud compute firewall-rules create allow-http-server-8080 \  --allow tcp:8080 \  --source-ranges 0.0.0.0/0 \  --target-tags http-server \  --description "Allow port 8080 access to http-server"

Node.js

if [ -z "$REDISHOST" ]; then  echo "Must set \$REDISHOST. For example: REDISHOST=127.0.0.1"  exit 1 fi if [ -z "$REDISPORT" ]; then  echo "Must set \$REDISPORT. For example: REDISPORT=6379"  exit 1 fi if [ -z "$GCS_BUCKET_NAME" ]; then  echo "Must set \$GCS_BUCKET_NAME. For example: GCS_BUCKET_NAME=my-bucket"  exit 1 fi if [ -z "$ZONE" ]; then  ZONE=$(gcloud config get-value compute/zone -q)  echo $ZONE fi #Upload the tar to GCS tar -cvf app.tar -C .. package.json server.js gsutil cp app.tar gs://"$GCS_BUCKET_NAME"/gce/ # Create an instance gcloud compute instances create my-instance \  --image-family=debian-9 \  --image-project=debian-cloud \  --machine-type=g1-small \  --scopes cloud-platform \  --metadata-from-file startup-script=startup-script.sh \  --metadata gcs-bucket=$GCS_BUCKET_NAME,redis-host=$REDISHOST,redis-port=$REDISPORT \  --zone $ZONE \  --tags http-server gcloud compute firewall-rules create allow-http-server-8080 \  --allow tcp:8080 \  --source-ranges 0.0.0.0/0 \  --target-tags http-server \  --description "Allow port 8080 access to http-server"

Python

if [ -z "$REDISHOST" ]; then  echo "Must set \$REDISHOST. For example: REDISHOST=127.0.0.1"  exit 1 fi if [ -z "$REDISPORT" ]; then  echo "Must set \$REDISPORT. For example: REDISPORT=6379"  exit 1 fi if [ -z "$GCS_BUCKET_NAME" ]; then  echo "Must set \$GCS_BUCKET_NAME. For example: GCS_BUCKET_NAME=my-bucket"  exit 1 fi if [ -z "$ZONE" ]; then  ZONE=$(gcloud config get-value compute/zone -q)  echo $ZONE fi #Upload the tar to GCS tar -cvf app.tar -C .. requirements.txt main.py # Copy to GCS bucket gsutil cp app.tar gs://"$GCS_BUCKET_NAME"/gce/ # Create an instance gcloud compute instances create my-instance \  --image-family=debian-11 \  --image-project=debian-cloud \  --machine-type=g1-small \  --scopes cloud-platform \  --metadata-from-file startup-script=startup-script.sh \  --metadata gcs-bucket=$GCS_BUCKET_NAME,redis-host=$REDISHOST,redis-port=$REDISPORT \  --zone $ZONE \  --tags http-server gcloud compute firewall-rules create allow-http-server-8080 \  --allow tcp:8080 \  --source-ranges 0.0.0.0/0 \  --target-tags http-server \  --description "Allow port 8080 access to http-server"

Run the deployment script:

 chmod +x deploy.sh ./deploy.sh 

Application startup script

This startup script is used in the sample deployment script to prepare the instance. For more information about start up scripts and viewing startup script execution logs, see Running Startup Scripts.

Go

set -ex # Talk to the metadata server to get the project id and location of application binary. PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") export PROJECTID GCS_BUCKET_NAME=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/gcs-bucket" -H "Metadata-Flavor: Google") REDISHOST=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-host" -H "Metadata-Flavor: Google") REDISPORT=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-port" -H "Metadata-Flavor: Google") # Install dependencies from apt apt-get update apt-get install -yq ca-certificates supervisor # Install logging monitor. The monitor will automatically pickup logs send to # syslog. curl "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" --output google-fluentd-install.sh checksum=$(sha256sum google-fluentd-install.sh | awk '{print $1;}') if [ "$checksum" != "ec78e9067f45f6653a6749cf922dbc9d79f80027d098c90da02f71532b5cc967" ]; then  echo "Checksum does not match"  exit 1 fi chmod +x google-fluentd-install.sh && ./google-fluentd-install.sh service google-fluentd restart & gsutil cp gs://"$GCS_BUCKET_NAME"/gce/app.tar /app.tar mkdir -p /app tar -x -f /app.tar -C /app chmod +x /app/app # Create a goapp user. The application will run as this user. getent passwd goapp || useradd -m -d /home/goapp goapp chown -R goapp:goapp /app # Configure supervisor to run the Go app. cat >/etc/supervisor/conf.d/goapp.conf << EOF [program:goapp] directory=/app environment=HOME="/home/goapp",USER="goapp",REDISHOST=$REDISHOST,REDISPORT=$REDISPORT command=/app/app autostart=true autorestart=true user=goapp stdout_logfile=syslog stderr_logfile=syslog EOF supervisorctl reread supervisorctl update

Java

set -ex # Talk to the metadata server to get the project id and location of application binary. PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") GCS_BUCKET_NAME=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/gcs-bucket" -H "Metadata-Flavor: Google") WAR=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/app-war" -H "Metadata-Flavor: Google") gsutil cp gs://"$GCS_BUCKET_NAME"/gce/"$WAR" . # Install dependencies from apt apt-get update apt-get install -qq openjdk-8-jdk-headless # Make Java8 the default update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java # Jetty Setup mkdir -p /opt/jetty/temp mkdir -p /var/log/jetty # Get Jetty curl -L https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.10.v20180503/jetty-distribution-9.4.10.v20180503.tar.gz -o jetty9.tgz tar xf jetty9.tgz --strip-components=1 -C /opt/jetty # Add a Jetty User useradd --user-group --shell /bin/false --home-dir /opt/jetty/temp jetty cd /opt/jetty # Add running as "jetty" java -jar /opt/jetty/start.jar --add-to-startd=setuid cd / # very important - by renaming the war to root.war, it will run as the root servlet. mv $WAR /opt/jetty/webapps/root.war # Make sure "jetty" owns everything. chown --recursive jetty /opt/jetty # Configure the default paths for the Jetty service cp /opt/jetty/bin/jetty.sh /etc/init.d/jetty echo "JETTY_HOME=/opt/jetty" > /etc/default/jetty {  echo "JETTY_BASE=/opt/jetty"  echo "TMPDIR=/opt/jetty/temp"  echo "JAVA_OPTIONS=-Djetty.http.port=8080"  echo "JETTY_LOGS=/var/log/jetty" } >> /etc/default/jetty # Reload daemon to pick up new service systemctl daemon-reload # Install logging monitor. The monitor will automatically pickup logs sent to syslog. curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash service google-fluentd restart & service jetty start service jetty check echo "Startup Complete"

Node.js

set -ex # Talk to the metadata server to get the project id and location of application binary. PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") GCS_BUCKET_NAME=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/gcs-bucket" -H "Metadata-Flavor: Google") REDISHOST=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-host" -H "Metadata-Flavor: Google") REDISPORT=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-port" -H "Metadata-Flavor: Google") # Install dependencies from apt apt-get update # Install Node.js 9 curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash - apt-get install -yq ca-certificates supervisor nodejs build-essential # Install logging monitor. The monitor will automatically pickup logs send to # syslog. curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash service google-fluentd restart & gsutil cp gs://"$GCS_BUCKET_NAME"/gce/app.tar /app.tar mkdir -p /app tar -x -f /app.tar -C /app cd /app # Install the app dependencies npm install # Create a nodeapp user. The application will run as this user. getent passwd nodeapp || useradd -m -d /home/nodeapp nodeapp chown -R nodeapp:nodeapp /app # Configure supervisor to run the Go app. cat >/etc/supervisor/conf.d/nodeapp.conf << EOF [program:nodeapp] directory=/app environment=HOME="/home/nodeapp",USER="nodeapp",REDISHOST=$REDISHOST,REDISPORT=$REDISPORT command=node server.js autostart=true autorestart=true user=nodeapp stdout_logfile=syslog stderr_logfile=syslog EOF supervisorctl reread supervisorctl update

Python

set -v # Talk to the metadata server to get the project id and location of application binary. PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") GCS_BUCKET_NAME=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/gcs-bucket" -H "Metadata-Flavor: Google") REDISHOST=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-host" -H "Metadata-Flavor: Google") REDISPORT=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/redis-port" -H "Metadata-Flavor: Google") # Install dependencies from apt apt-get update apt-get install -yq \  git build-essential supervisor python python-dev python-pip libffi-dev \  libssl-dev # Install logging monitor. The monitor will automatically pickup logs send to # syslog. curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash service google-fluentd restart & gsutil cp gs://"$GCS_BUCKET_NAME"/gce/app.tar /app.tar mkdir -p /app tar -x -f /app.tar -C /app cd /app # Install the app dependencies pip install --upgrade pip virtualenv virtualenv /app/env /app/env/bin/pip install -r /app/requirements.txt # Create a pythonapp user. The application will run as this user. getent passwd pythonapp || useradd -m -d /home/pythonapp pythonapp chown -R pythonapp:pythonapp /app # Configure supervisor to run the app. cat >/etc/supervisor/conf.d/pythonapp.conf << EOF [program:pythonapp] directory=/app environment=HOME="/home/pythonapp",USER="pythonapp",REDISHOST=$REDISHOST,REDISPORT=$REDISPORT command=/app/env/bin/gunicorn main:app --bind 0.0.0:8080 autostart=true autorestart=true user=pythonapp stdout_logfile=syslog stderr_logfile=syslog EOF supervisorctl reread supervisorctl update

It may take several minutes for the sample application to deploy on the newly created Compute Engine instance.

Once the instance is ready and the startup script has completed execution, go to the Compute Engine instances page and copy the External Ip address.

To view the sample app you deployed, visit http://[EXTERNAL-IP]:8080

You can use the teardown.sh script to delete the instance and firewall created by the deployment script:

gcloud compute instances delete my-instance gcloud compute firewall-rules delete allow-http-server-8080