The SNS Fork Pattern is an architectural pattern where an Amazon SNS topic is used to send messages to multiple processing pipelines. The high-level architecture looks like this:

Each processing pipeline creates a separate subscription to the Amazon SNS topic. SNS Subscription Filter Policies can be applied for each subscription to ensure each pipeline only receives the messages they want to process.

This repository implements the SNS Fork Pattern as a suite of Serverless applications. Each application implements general purpose, reusable message processing pipelines. All of the apps have been published to the AWS Serverless Application Repository (SAR) and can easily be integrated into an existing AWS SAM application using the new nested apps feature of AWS SAM/SAR. An example application is also included that demonstrates composing the different processing pipeline apps together using nested apps.

This repository includes the following serverless apps:

1. Storage and Backup - Processing pipeline that saves topic messages to an Amazon S3 bucket for use as backups or other purposes, e.g., to query with Amazon Athena.
2. Search and Analytics - Processing pipeline that saves topic messages to an AWS Elasticsearch cluster for search and analytics.
3. Message Replay - Processing pipeline that saves topic messages to a replay buffer SQS queue. In a disaster recovery scenario, messages from up to 14 days ago can be replayed back to another processing pipeline's SQS queue.
4. SNS Fork Example (TODO: link to SAR app page) - Example application showing how the above pipelines can be used in an AWS SAM application.

Deploying the SNS Fork Example application is simple using the Serverless Application Repository. There is no need to build or package the application from the repository source code. Simply follow these steps:

1. Create an AWS account if you do not already have one and login.
2. Go to the sns-fork-example app's page on the Serverless Application Repository and click "Deploy".
3. Edit the stack name if you would like and click "Deploy".

This sample code is made available under a modified MIT license. See the LICENSE file.