Route records to ElasticSearch indexes

The routing section of the /etc/aerospike-elasticsearch-outbound/aerospike-elasticsearch-outbound.yml file controls how Aerospike records are routed to a destination index in Elasticsearch.

Routing Mode

An outbound connector can route messages to one or more indexes. You can use the static routing mode to specify in advance the exact names of the indexes to route messages to. You can also use three different dynamic routing modes to have the outbound connector determine the names of indexes dynamically, based on information in records.

Static

• static Always routes to the specified index.
• skip Skip dispatch of record to Elasticsearch and ack success to XDR.

Dynamic

• namespace Uses the namespace of each Aerospike record as the name of the index to publish messages to.
• set Uses the set of each Aerospike record as the name of the index to publish to. For records without a set, you can specify a fallback route that uses the “static” routing mode.
• bin Uses the value of a bin in each Aerospike record as the name of the index to publish to. Only string, blob, and integer bin types are supported.

When you use one of the dynamic modes, you can transform the names of namespaces, sets, and bins by changing their case, removing whitespace, or using regular expressions to replace characters.

Static routing

Here are the configuration options:

Option	Required	Expected value	Description
mode	yes	static	Specifies to write messages to the index that is named in the destination option.
destination	yes		Name of the destination index.

Example

...
routing:
 mode: static
 destination: users
...

Skip routing

For skip routing the configuration options are

Option	Required	Expected value	Description
mode	yes	skip	Skip dispatch of record to Elasticsearch and ack success to XDR.

Example

...
routing:
 mode: skip
...

Namespace routing

Here are the configuration options:

Option	Required	Expected value	Description
mode	yes	namespace	Specifies to write to index that have name that matches name of namespace in record metadata.
transforms	no		List of transformations to apply to the namespace name. See the Transforming dynamically derived names section for details.

Example

...
routing:
 mode: namespace
 transforms:
 - trim
 - regex:
 pattern: '(.*):(.*)'
 replacement: '$2:$1'
 - regex:
 pattern: '$'
 replacement: ':please'
 - uppercase
...

Set name routing

Here are the configuration options:

Option	Required	Expected value	Description
mode	yes	set	Specifies to write to index that have name that matches name of a set in record metadata.
default	no		Default destination index to use in case the set name is missing in the record.
transforms	no		List of transformations to apply to the set name. See the Transforming dynamically derived names section for details.

Example

...
routing:
 mode: set
 transforms:
 - trim
 - regex:
 pattern: '(.*):(.*)'
 replacement: '$2:$1'
 - regex:
 pattern: '$'
 replacement: ':please'
 - uppercase
...

Bin value routing

Here are the configuration options:

Option	Required	Expected value	Description
mode	yes	bin	Specifies to write to index that have name that matches value in the specified bin.
bin	yes		Name of the bin to pick value from.
default	yes		Default destination index to use in case the bin is missing in the record.
transforms	no		List of transformations to apply to the bin value. See the Transforming dynamically derived names section for details.

Example

...
routing:
 mode: bin
 bin: category
 transforms:
 - trim
 - regex:
 pattern: '[^A-Za-z0-9]'
 replacement: '-'
 - lowercase
...

Custom Routing

Record can also be routed with custom code. See Routing Transform.

Example

...
routing:
 mode: custom
 class: com.aerospike.connect.outbound.example.GenerationRouter
...

Transforming dynamically derived names

You can configure a list of transforms that will be applied, in order, to the record’s set name, namespace or bin value to derive the name of the destination index.

The following transformations are supported:

Name	Description
lowercase	Converts to lowercase.
uppercase	Converts to uppercase
trim	Trims leading and trailing whitespace.
regex	Matches against a regex pattern and replaces all occurrences with a replacement. The regex and replacement use Java regex syntax.

Example

The following transform configuration trims the route, replaces all non-alphanumeric characters with ’-’, and then converts the result to lowercase.

...
routing:
 mode: bin
 bin: category
 default: test-index
 transforms:
 - trim
 - regex:
 pattern: '[^A-Za-z0-9]'
 replacement: '-'
 - lowercase
...