Migrating NuoDB CLOUD DATABASE MIGRATION MADE EASY October 19, 2017
Copyright © 2017 NuoDB - NuoDB Confidential LOGISTICS + You will be on mute + Submit questions in the Q&A box on the right + Webinar is being recorded & will be available for replay Joe Leslie Senior Product Manager
Copyright © 2017 NuoDB - NuoDB Confidential MOVING TO THE CLOUD, RETHINKING EVERYTHING 3 Microservices and Containers Growing Data Workloads Developer Scarcity Dev-ops & Agile Data Security Purchasing Patterns
Copyright © 2017 NuoDB - NuoDB Confidential CLOUD DATABASE REQUIREMENTS 4 Required Capabilities SQL Business Database of Record ANSI-Standard SQL Migrate existing SQL apps In-memory performance Elastic Simple elastic scale-out/in (reads and writes) Zero downtime Hybrid cloud deployment
Copyright © 2017 NuoDB - NuoDB Confidential CLOUD DATABASE OPTIONS 5 Traditional RDBMS NoSQL Required Capabilities SQL Business Database of Record ✓ ✗ ✓ ANSI-Standard SQL ✓ ✗ ✓ Migrate existing SQL apps (✓) ✗ (✓) In-memory performance ✓ ✓ ✓ Elastic Simple elastic scale-out/in (reads and writes) ✗ ✓ (✗) Zero downtime ✗ ✓ ✗ Hybrid cloud deployment ✗ ✓ ✗
Copyright © 2017 NuoDB - NuoDB Confidential ELASTIC SQL: A BETTER CHOICE 6 Traditional RDBMS NoSQL Elastic SQL Required Capabilities SQL Business Database of Record ✓ ✗ ✓ ✓ ✓ ANSI-Standard SQL ✓ ✗ ✓ (✓) ✓ Migrate existing SQL apps (✓) ✗ (✓) ✗ (✓) In-memory performance ✓ ✓ ✓ ✓ ✓ Elastic Simple elastic scale-out/in (reads and writes) ✗ ✓ (✗) ✓ ✓ Zero downtime ✗ ✓ ✗ ✓ ✓ Hybrid cloud deployment ✗ ✓ ✗ ✗ ✓
Copyright © 2017 NuoDB - NuoDB Confidential NuoDB: THE ELASTIC SQL DATABASE NuoDB combines the scale-out simplicity, elasticity, and continuous availability that cloud applications require with the transactional consistency and durability that databases of record demand.
Copyright © 2017 NuoDB - NuoDB Confidential NuoDB: THE ELASTIC SQL DATABASE 8 Fast in-memory transaction processing with dynamic application data affinity Durable storage management with scale-out storage NuoDB Architecture Legacy RDBMS Architecture Modern distributed, peer-to-peer services acting as a single, logical database + New modern database architecture with independent database services + Services scale independently and elastically + Straightforward ANSI SQL interface + Deploy across containers, clouds & data centers + Continuous availability for planned (upgrades) and unplanned outages Storage
Copyright © 2017 NuoDB - NuoDB Confidential MIGRATING DATA INTO NuoDB: 4 EASY STEPS Once the Setup is complete you can run the Get, Dump, and Load from a single migration command. For this video we will migrate the MySQL “employees” ~ 3M rows star-schema demo database to NuoDB 9 Setup Get Schema Dump Data Load Data 1 1 1 1 1 1 1 1 1 2 1 1 1 1 3 1 1 1 1 4
ON TO THE DEMO! 10 Let’s get started!
Copyright © 2017 NuoDB - NuoDB Confidential 11 Setting Value Source JDBC driver class name --source.driver=com.mysql.jdbc.Driver Source database connection URL --source.url=jdbc:mysql://localhost:3306/schema? NuoDB target database connection URL . --target.url=jdbc:com.nuodb://localhost/test CLASSPATH environment variable export CLASSPATH=/mysql-connector-java- 5.1.41/mysql-connector-java-5.1.41- bin.jar:$CLASSPATH Capture the source schema (e.g. tables, columns, data types, constraints, sequences, and indexes) and apply to the target schema Extract source data into a common file format (e.g. csv, xml, bson) Insert extracted data into NuoDB! Setup Get Schema Dump Data Load Data 1 1 1 1 1 1 1 1 1 2 1 1 1 1 3 1 1 1 1 4
Copyright © 2017 NuoDB - NuoDB Confidential Register for our December webinar with NayaTech to hear about the other steps! www.nuodb.com/mig ration-webinar …OBVIOUSLY THIS IS ONLY ONE PART OF THE MIGRATION 12
READY TO TRY IT? Get the scripts nuodb.com/migrate-scripts Watch the NuoDB demo: nuodb.com/full-demo Download Community Edition: www.nuodb.com/download
Copyright © 2017 NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Get schema $ nuodb-migrator schema --source.driver=com.mysql.jdbc.Driver --source.url=jdbc:mysql://localhost:3306/employees? --source.username=dbuser --source.password=Dbuser:10 --identifier.quoting=minimal --output.path=schema.sql $ nuosql test --user dba --password goalie --file schema.sql 14
Copyright © 2017 NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Dump schema $ nuodb-migrator dump --source.driver=com.mysql.jdbc.Driver --source.url=jdbc:mysql://localhost:3306/employees? --source.username=dbuser --source.password=Dbuser:10 --output.type=csv --output.csv.delimiter=~ --output.path=/tmp/migrate_mysql_to_nuodb 15
Copyright © 2017 NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Load schema $ nuodb-migrator load --target.url=jdbc:com.nuodb://localhost/test --target.schema=employees --target.username=dba --target.password=goalie --input.csv.delimiter=~ --input.path=/tmp/migrate_mysql_to_nuodb 16

Cloud Database Migration Made Easy: Migrating MySQL to NuoDB

  • 1.
    Migrating NuoDB CLOUD DATABASE MIGRATIONMADE EASY October 19, 2017
  • 2.
    Copyright © 2017NuoDB - NuoDB Confidential LOGISTICS + You will be on mute + Submit questions in the Q&A box on the right + Webinar is being recorded & will be available for replay Joe Leslie Senior Product Manager
  • 3.
    Copyright © 2017NuoDB - NuoDB Confidential MOVING TO THE CLOUD, RETHINKING EVERYTHING 3 Microservices and Containers Growing Data Workloads Developer Scarcity Dev-ops & Agile Data Security Purchasing Patterns
  • 4.
    Copyright © 2017NuoDB - NuoDB Confidential CLOUD DATABASE REQUIREMENTS 4 Required Capabilities SQL Business Database of Record ANSI-Standard SQL Migrate existing SQL apps In-memory performance Elastic Simple elastic scale-out/in (reads and writes) Zero downtime Hybrid cloud deployment
  • 5.
    Copyright © 2017NuoDB - NuoDB Confidential CLOUD DATABASE OPTIONS 5 Traditional RDBMS NoSQL Required Capabilities SQL Business Database of Record ✓ ✗ ✓ ANSI-Standard SQL ✓ ✗ ✓ Migrate existing SQL apps (✓) ✗ (✓) In-memory performance ✓ ✓ ✓ Elastic Simple elastic scale-out/in (reads and writes) ✗ ✓ (✗) Zero downtime ✗ ✓ ✗ Hybrid cloud deployment ✗ ✓ ✗
  • 6.
    Copyright © 2017NuoDB - NuoDB Confidential ELASTIC SQL: A BETTER CHOICE 6 Traditional RDBMS NoSQL Elastic SQL Required Capabilities SQL Business Database of Record ✓ ✗ ✓ ✓ ✓ ANSI-Standard SQL ✓ ✗ ✓ (✓) ✓ Migrate existing SQL apps (✓) ✗ (✓) ✗ (✓) In-memory performance ✓ ✓ ✓ ✓ ✓ Elastic Simple elastic scale-out/in (reads and writes) ✗ ✓ (✗) ✓ ✓ Zero downtime ✗ ✓ ✗ ✓ ✓ Hybrid cloud deployment ✗ ✓ ✗ ✗ ✓
  • 7.
    Copyright © 2017NuoDB - NuoDB Confidential NuoDB: THE ELASTIC SQL DATABASE NuoDB combines the scale-out simplicity, elasticity, and continuous availability that cloud applications require with the transactional consistency and durability that databases of record demand.
  • 8.
    Copyright © 2017NuoDB - NuoDB Confidential NuoDB: THE ELASTIC SQL DATABASE 8 Fast in-memory transaction processing with dynamic application data affinity Durable storage management with scale-out storage NuoDB Architecture Legacy RDBMS Architecture Modern distributed, peer-to-peer services acting as a single, logical database + New modern database architecture with independent database services + Services scale independently and elastically + Straightforward ANSI SQL interface + Deploy across containers, clouds & data centers + Continuous availability for planned (upgrades) and unplanned outages Storage
  • 9.
    Copyright © 2017NuoDB - NuoDB Confidential MIGRATING DATA INTO NuoDB: 4 EASY STEPS Once the Setup is complete you can run the Get, Dump, and Load from a single migration command. For this video we will migrate the MySQL “employees” ~ 3M rows star-schema demo database to NuoDB 9 Setup Get Schema Dump Data Load Data 1 1 1 1 1 1 1 1 1 2 1 1 1 1 3 1 1 1 1 4
  • 10.
    ON TO THEDEMO! 10 Let’s get started!
  • 11.
    Copyright © 2017NuoDB - NuoDB Confidential 11 Setting Value Source JDBC driver class name --source.driver=com.mysql.jdbc.Driver Source database connection URL --source.url=jdbc:mysql://localhost:3306/schema? NuoDB target database connection URL . --target.url=jdbc:com.nuodb://localhost/test CLASSPATH environment variable export CLASSPATH=/mysql-connector-java- 5.1.41/mysql-connector-java-5.1.41- bin.jar:$CLASSPATH Capture the source schema (e.g. tables, columns, data types, constraints, sequences, and indexes) and apply to the target schema Extract source data into a common file format (e.g. csv, xml, bson) Insert extracted data into NuoDB! Setup Get Schema Dump Data Load Data 1 1 1 1 1 1 1 1 1 2 1 1 1 1 3 1 1 1 1 4
  • 12.
    Copyright © 2017NuoDB - NuoDB Confidential Register for our December webinar with NayaTech to hear about the other steps! www.nuodb.com/mig ration-webinar …OBVIOUSLY THIS IS ONLY ONE PART OF THE MIGRATION 12
  • 13.
    READY TO TRYIT? Get the scripts nuodb.com/migrate-scripts Watch the NuoDB demo: nuodb.com/full-demo Download Community Edition: www.nuodb.com/download
  • 14.
    Copyright © 2017NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Get schema $ nuodb-migrator schema --source.driver=com.mysql.jdbc.Driver --source.url=jdbc:mysql://localhost:3306/employees? --source.username=dbuser --source.password=Dbuser:10 --identifier.quoting=minimal --output.path=schema.sql $ nuosql test --user dba --password goalie --file schema.sql 14
  • 15.
    Copyright © 2017NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Dump schema $ nuodb-migrator dump --source.driver=com.mysql.jdbc.Driver --source.url=jdbc:mysql://localhost:3306/employees? --source.username=dbuser --source.password=Dbuser:10 --output.type=csv --output.csv.delimiter=~ --output.path=/tmp/migrate_mysql_to_nuodb 15
  • 16.
    Copyright © 2017NuoDB - NuoDB Confidential MIGRATING DATA INTO NUODB Sample MySQL to NuoDB working run scripts Load schema $ nuodb-migrator load --target.url=jdbc:com.nuodb://localhost/test --target.schema=employees --target.username=dba --target.password=goalie --input.csv.delimiter=~ --input.path=/tmp/migrate_mysql_to_nuodb 16

Editor's Notes

  • #3 My name is… I’m joined by Tim Tadeo, a Solutions Architect with NuoDB. I’ll start by introducing the Elastic SQL database and why and how you should be thinking differently about databases as you move to a modern architecture. Tim will then do a live demonstration showing how to evaluate key aspects of an Elastic SQL database. I’ll wrap up with a pointer to resources available for you to conduct your own evaluation. At the end of the presentation we’ll be taking live questions from the audience. At any time during the webinar you can enter your questions using the Question box on the right side of your screen and we’ll address them at the end. We are also recording today’s webinar and will share the replay link with you should you wish to revisit it or share it with your colleagues. With that, let’s get started by exploring the database landscape
  • #4 To accomplish this, software organizations are rethinking everything about their computing environments - infrastructure, application development and operations. Data workloads are growing – everywhere – and storage and access for large datasets are a growing need as well Developers are a scare commodity Applications are being asked to do more and perform to more demanding requirements …while keeps costs to a minimum To accommodate these needs and transform in a service, customer-oriented business software orgs are moving their applications to the cloud As part of this modernization transformation, organizations must also consider how cloud applications will handle and store data
  • #5 Looking at the landscape today, we see that traditional relational databases are excellent at providing a business database of record with key capabilities around SQL and ACID properties, but as we’ve discussed, can’t scale out effectively. NoSQL technology can scale out elastically, but forgoes strict ACID compliance and full ANSI SQL support – making them less suitable for business-critical applications. Newer products – such as Google Cloud Platform Spanner, and CockroachDB promise the combination of these areas, with traditional relational designs that scale-out elastically. These first-generation products today provide limited SQL support making it harder to migrate existing SQL workloads to them, and rely on sophisticated clock synchronization limiting or constraining deployment flexibility. But they hold the promise of running business-critical applications in a cloud architecture. NuoDB, first introduced 4 years ago and proven in many production implementations, truly brings these worlds together with rich SQL support and ACID compliance in a scale-out architecture that can be deployed on-prem, across cloud providers, and even in a hybrid-cloud architecture.
  • #6 Looking at the landscape today, we see that traditional relational databases are excellent at providing a business database of record with key capabilities around SQL and ACID properties, but as we’ve discussed, can’t scale out effectively. NoSQL technology can scale out elastically, but forgoes strict ACID compliance and full ANSI SQL support – making them less suitable for business-critical applications. Newer products – such as Google Cloud Platform Spanner, and CockroachDB promise the combination of these areas, with traditional relational designs that scale-out elastically. These first-generation products today provide limited SQL support making it harder to migrate existing SQL workloads to them, and rely on sophisticated clock synchronization limiting or constraining deployment flexibility. But they hold the promise of running business-critical applications in a cloud architecture. NuoDB, first introduced 4 years ago and proven in many production implementations, truly brings these worlds together with rich SQL support and ACID compliance in a scale-out architecture that can be deployed on-prem, across cloud providers, and even in a hybrid-cloud architecture.
  • #7 Looking at the landscape today, we see that traditional relational databases are excellent at providing a business database of record with key capabilities around SQL and ACID properties, but as we’ve discussed, can’t scale out effectively. NoSQL technology can scale out elastically, but forgoes strict ACID compliance and full ANSI SQL support – making them less suitable for business-critical applications. Newer products – such as Google Cloud Platform Spanner, and CockroachDB promise the combination of these areas, with traditional relational designs that scale-out elastically. These first-generation products today provide limited SQL support making it harder to migrate existing SQL workloads to them, and rely on sophisticated clock synchronization limiting or constraining deployment flexibility. But they hold the promise of running business-critical applications in a cloud architecture. NuoDB, first introduced 4 years ago and proven in many production implementations, truly brings these worlds together with rich SQL support and ACID compliance in a scale-out architecture that can be deployed on-prem, across cloud providers, and even in a hybrid-cloud architecture.
  • #9 NuoDB has been built from the ground up to be an operational database that scales-out for cloud deployments. NuoDB appears as a single, logical, SQL database to the application, allowing developers to focus on building great applications, versus dealing with scale-out complexities. Under the hood, NuoDB has a peer-to-peer, two-layer, distributed architecture that can be deployed across multiple data centers and is optimized for in-memory speeds, continuous availability, and elastic scale-out. The transaction layer  consists of in-memory process nodes called transaction engines (TE). Transaction engines handle requests from applications, cache data for fast access, and coordinate transactions with other process nodes in both the transaction and storage layers. As an application makes requests of NuoDB, the transaction engines will naturally build in-memory caches with affinity for that application’s data, allowing NuoDB to maintain high performance.   The storage layer consists of process nodes called storage managers (SM). The storage manager ensures durability of data by writing it to disk, manages data on disk, handles requests from transaction engines (TEs), and sends asynchronous messages to other SMs to commit data to disk and to maintain copies of data in memory. These process nodes provide ACID-guarantees, data redundancy, and data persistence.   Within both layers, NuoDB can elastically scale out (and back) without any interruption to application service, simply by adding and removing TEs and SMs. This means developers can design applications to access a single logical database and not worry about handling scale out complexity related to dynamic operational workloads.  Database operators can scale out the database to accommodate dynamic workloads and not worry about adverse consequences to the application. The result is that developers and operations can focus on truly maximizing performance of both the application and the database.
  • #14 If you’d like to learn more about NuoDB and try it for yourself, you can check out our recorded demo, or download our community edition or evaluation guide from our website. At this point we’ll move into Q&A…