Definition #
Route specifies the rule of matching a list of source-table and mapping to sink-table. The most typical scenario is the merge of sub-databases and sub-tables, routing multiple upstream source tables to the same sink table.
Parameters #
To describe a route, the follows are required:
| parameter | meaning | optional/required |
|---|---|---|
| source-table | Source table id, supports regular expressions | required |
| sink-table | Sink table id, supports symbol replacement | required |
| replace-symbol | Special symbol in sink-table for pattern replacing, will be replaced by original table name | optional |
| description | Routing rule description(a default value provided) | optional |
A route module can contain a list of source-table/sink-table rules.
Example #
Route one Data Source table to one Data Sink table #
if synchronize the table web_order in the database mydb to a Doris table ods_web_order, we can use this yaml file to define this route：
route: - source-table: mydb.web_order sink-table: mydb.ods_web_order description: sync table to one destination table with given prefix ods_ Route multiple Data Source tables to one Data Sink table #
What’s more, if you want to synchronize the sharding tables in the database mydb to a Doris table ods_web_order, we can use this yaml file to define this route：
route: - source-table: mydb\.* sink-table: mydb.ods_web_order description: sync sharding tables to one destination table Complex Route via combining route rules #
What’s more, if you want to specify many different mapping rules, we can use this yaml file to define this route：
route: - source-table: mydb.orders sink-table: ods_db.ods_orders description: sync orders table to orders - source-table: mydb.shipments sink-table: ods_db.ods_shipments description: sync shipments table to ods_shipments - source-table: mydb.products sink-table: ods_db.ods_products description: sync products table to ods_products Pattern Replacement in routing rules #
If you’d like to route source tables and rename them to sink tables with specific patterns, replace-symbol could be used to resemble source table names like this:
route: - source-table: source_db.\.* sink-table: sink_db.<> replace-symbol: <> description: route all tables in source_db to sink_db Then, all tables including source_db.XXX will be routed to sink_db.XXX without hassle.