Configure an Amazon Aurora MySQL database for CDC

This page describes how to configure change data capture (CDC) to stream data from an Amazon Aurora MySQL database to a supported destination, such as BigQuery or Cloud Storage.

Create a parameter group

  1. Launch your Amazon RDS Dashboard.

  2. In the Navigation Drawer, click Parameter Groups.

  3. Click Create Parameter Group. The Create Parameter Group page appears.

  4. Use the following table to populate the fields of this page:

    FieldDescription
    Parameter group familySelect the family that matches your database.
    TypeSelect DB Cluster Parameter Group.
    Group nameProvide a name for the parameter group.
    DescriptionProvide a description for the parameter group.

  5. Click Create.

  6. Select the checkbox to the left of your newly created parameter group.

  7. Under Parameter group actions, click Edit.

  8. Set the following parameter values:

    • For GTID-based replication:

      ParameterValue
      binlog_formatROW
      log_bin_use_v1_row_events1
      read_only0
      net_read_timeout3600
      net_write_timeout3600
      wait_timeout86400
      expire-logs-days7
      binlog_row_imageFULL
      max_allowed_packet1G (recommended value)
      log-replica-updates1
      enforce-gtid-consistencyON
      gtid_mode

      ON

      You can only set this parameter to ON after you set enforce-gtid-consistency to ON. In general, gtid_mode can have the following values:
      • OFF
      • OFF_PERMISSIVE
      • ON_PERMISSIVE
      • ON
      You can only change the values of the gtid_mode parameter one step at a time. For example, if the current mode is set to OFF, do the following:
      1. Change the setting to OFF_PERMISSIVE.
      2. Change the setting to ON_PERMISSIVE.
      3. Change the setting to ON.
      For more information, see MySQL documentation.

    • For binlog-based replication:

      ParameterValue
      binlog_formatROW

  9. Click Save Changes.

Assign the parameter group to the database instance

  1. Launch your Amazon RDS Dashboard.

  2. In the Navigation Drawer, click Databases, and then select your database instance.

  3. From the Instance Actions menu, select Modify. The Modify DB Instance dialog appears.

  4. In the Additional configuration section, select the database cluster parameter group that you created.

  5. Set the Backup retention period to 7 days.

  6. Click Continue.

  7. In the Scheduling of Modifications pane, select the Apply immediately option.

Set the binary log retention period

  1. Enter the following MySQL command:

    call mysql.rds_set_configuration('binlog retention hours', 168);

  2. Restart your MySQL server so that the changes you made can take effect.

Create a Datastream user

  1. Create a Datastream user:

    CREATE USER 'datastream'@'%' IDENTIFIED BY 'YOUR_PASSWORD';

  2. Grant the following privileges to them:

    GRANT REPLICATION SLAVE, SELECT, REPLICATION CLIENT ON *.* TO 'datastream'@'%'; GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO 'datastream'@'%'; FLUSH PRIVILEGES;

What's next