Schema-and-data subcommand

This subcommand will generate a schema as well as perform data migration This subcommand can be used to do both POC and minimal downtime migrations. In practice, we have seen this command used more frequently for POC migrations, in order to get started quickly.

The command below assumes that the open-source version of SMT is being used. For the CLI reference of the gCloud version of SMT, please refer here.

Table of contents
  1. NAME
  2. SYNOPSIS
  3. DESCRIPTION
  4. EXAMPLES
  5. REQUIRED FLAGS
  6. OPTIONAL FLAGS

NAME 

./spanner-migration-tool schema-and-data - migrate data from a source database to Cloud Spanner given a schema

SYNOPSIS 

./spanner-migration-tool schema-and-data --source=SOURCE [--dry-run] [--log-level=LOG_LEVEL] [--prefix=PREFIX] [--skip-foreign-keys] [--source-profile=SOURCE_PROFILE] [--target=TARGET] [--target-profile=TARGET_PROFILE] [--write-limit=WRITE_LIMIT] [--project=PROJECT] [GCLOUD_WIDE_FLAG ...]

DESCRIPTION 

Migrate schema and data from a source database to Cloud Spanner.

EXAMPLES 

To generate schema and copy data to Cloud Spanner from a source PostgreSQL database using pg_dump: $ ./spanner-migration-tool schema-and-data --source=postgresql \ < ~/cart.pg_dump --target-profile='instance=spanner-instance' To run a minimal downtime schema and data migration: $ ./spanner-migration-tool schema-and-data --source=MySQL \ --source-profile='host=host,port=3306,user=user,password=pwd,dbN\ ame=db,streamingCfg=streaming.json' \ --target-profile='project=spanner-project,instance=spanner-insta\ nce' --project='migration-project' To run a minimal downtime schema and data migration for a sharded source: $ ./spanner-migration-tool schema-and-data --source=MySQL \ --source-profile='config=shard-config.json' \ --target-profile='project=spanner-project,instance=spanner-insta\ nce' --project='migration-project'

For sharded migrations, you can refer to the example source-config.json passed in the source-profile option here.

REQUIRED FLAGS 

 --source=SOURCE Flag for specifying source database (e.g., PostgreSQL, MySQL, DynamoDB).

OPTIONAL FLAGS

Detailed description of optional flags can be found here.

 --dry-run Flag for generating DDL and schema conversion report without creating a Cloud Spanner database. --log-level=LOG_LEVEL To configure the log level for the execution (INFO, VERBOSE). --prefix=PREFIX File prefix for generated files. --skip-foreign-keys Skip creating foreign keys after data migration is complete. This is flag is only valid for POC migrations. --source-profile=SOURCE_PROFILE Flag for specifying connection profile for source database (e.g., "file=<path>,format=dump"). --target=TARGET Specifies the target database, defaults to Spanner (accepted values: Spanner) (default "Spanner"). --target-profile=TARGET_PROFILE Flag for specifying connection profile for target database (e.g., "dialect=postgresql"). --write-limit=WRITE_LIMIT Number of parallel writers to Cloud Spanner during bulk data migrations (default 40). --project=PROJECT Flag for specifying the name of the Google Cloud Project in which the Spanner migration tool can create resources required for migration. If the project is not specified, Spanner migration tool will try to fetch the configured project in the gCloud CLI. --dataflow-template=DATAFLOW_TEMPLATE The google cloud storage path of the minimal downtime migration template to use to run the migration job. Default value is the latest dataflow template. --session-file-name=SESSION_FILENAME Optional. Specifies the name of the file we store session state in.