DNM: kinesis support for bridge #924
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This is a copy/paste from the repo where this work was being done originally.
Embeds a deno runtime in the bridge binary to allow ad hoc reshaping of messages read from queues before they are sent to Svix. ## Motivation When consuming JSON from messaging systems to generate new webhooks, it's common to need to transform the payload before making the Create Message request to Svix. ## Solution Embedding a JS runtime into the bridge process allows us to run user-defined functions on the payloads as they move from input to output. With this diff, plugin instances provided by `svix-webhook-bridge-plugin-queue-consumer` can now evaluate js included in the plugin config: ```yaml plugins: - type: "rabbitmqconsumer" input: uri: "amqp://guest:guest@localhost:5672/%2f" queue_name: "local" requeue_on_nack: false transformation: | function handler(input) { return { app_id: input.key, message: { eventType: input.event_type, payload: input.data } }; } output: token: "***************" ``` The `transformation` key can now be set on any of these consumers. The JS fragment should include a default export of a function that accepts an object and returns an object. These functions can be used to reshape the payload as necessary. Very incomplete/untested. The sdk doesn't seem to have APIs for consuming. Follow <awslabs/aws-sdk-rust#213> for updates on the sdk side.
svix-onelson force-pushed the onelson/bridge branch 2 times, most recently from c2f47e8 to e567257 Compare Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Do not merge. Only a partial impl.
Needs consumer APIs which are not yet implemented in the SDK awslabs/aws-sdk-rust#213
Much of the code in the new kinesis module is actually copypasta from SQS, so don't be fooled. It probably looks more complete than it is.