Migrating data to drupal 8

Nacho Sánchez CTO@ about.me @isholgueras nacho@letshackity.com drupal.org/u/isholgueras http://www.isholgueras.com

STEPS 1. Requirements 2. Anatomy of a migration 3. Migration Framework 4. D2D migrations

Two new modules ▸ Migrate Handles migrations. Framework. ▸ Migrate Drupal Contains migrations from D6 & D7. Migrate in core! 1. REQUIREMENTS

But… how can I execute that migration. UI is not ready? No Drush command? UI for D2D migration in progress: https://www.drupal.org/node/2257723 Migrate in core! 1. REQUIREMENTS

So... Contrib! 1. REQUIREMENTS

▸ Drupal 8.0.x ▸ Drush 8 ▸ Migrate tools (contrib) ▸ Migrate Plus (contrib) Needed 1. REQUIREMENTS Useful

2. ANATOMY OF A MIGRATION DESTINATIONSOURCE PROCESS PROCESS PROCESS Workflow

PLUGINS PHP Files Core files or custom files Types: - Source - Process - Destination - Builder - ID Map DEFINITIONS Yaml Files. Custom files 2. ANATOMY OF A MIGRATION In files

2. ANATOMY OF A MIGRATION The easiest example

DEFINITIONS config/install/migrate.migration.article_node.yml 2. ANATOMY OF A MIGRATIONid: article_node label: Migrate posts from CakePHP to Drupal 8 source: plugin: article_node key: legacy #target: legacy destination: plugin: entity:node process: type: plugin: default_value default_value: article nid: id title: title 'body/value': description uid: user_id status: plugin: default_value default_value: true created: plugin: callback source: created callable: strtotime

PLUGINS src/Plugin/migrate/source/ArticleNode.php 2. ANATOMY OF A MIGRATION<?php namespace .. use .. /** * Source plugin for beer content. * * @MigrateSource( * id = "article_node" * ) */ class ArticleNode extends SqlBase { public function query() { $query = $this->select('articles', 'a') ->fields('a', [ 'id', 'user_id', 'title', 'description', 'created', ]); return $query; } public function fields() { $fields = [ 'id' => $this->t("Article ID"), // ... ]; return $fields; } public function getIds() { return [ 'id' => [ 'type' => 'integer', 'alias' => 'a', ], ]; } }

CONFIGURATION sites/local/settings.php 2. ANATOMY OF A MIGRATION <?php $databases['legacy']['default'] = array( 'database' => 'old_app', 'username' => 'dev', 'password' => 'dev', 'prefix' => '', 'host' => 'localhost', 'port' => '3306', 'namespace' => 'DrupalCoreDatabaseDrivermysql', 'driver' => 'mysql', );

EXECUTION Only with Drush8 and migrate_plus enabled 2. ANATOMY OF A MIGRATION vagrant@dev $ drush8 migrate-status Group: my_group Status Total Imported Unprocessed Last imported article_node Idle 128 0 128 vagrant@dev $ drush8 migrate-import article_node; d8 ms Processed 128 item (128 created, 0 updated, 0 failed, 0 ignored) - done with 'article_node' [status] Group: my_group Status Total Imported Unprocessed Last imported article_node Idle 128 128 0 2016-02-22 12:34:38 vagrant@dev $ drush8 migrate-rollback article_node; d8 ms Rolled back 128 items - done with 'article_node' Group: my_group Status Total Imported Unprocessed Last imported article_node Idle 128 0 128 2016-02-22 12:34:38

Too easy! 2. ANATOMY OF A MIGRATION

0.- KNOWLEDGE 3. MIGRATION FRAMEWORKSource Plugin: - SqlBase Process Plugin: - ProcessPluginBase Destination Plugin: - DestinationBase More plugins in: core/modules/migrate/src/Plugin/migrate Most used

0.- KNOWLEDGE 3. MIGRATION FRAMEWORKprepareRow !== transform - extends SqlBase - Before execution - extends ProcessPluginBase - During execution

1.- SOURCE 3. MIGRATION FRAMEWORK We tell here to SqlBase: - Which Database is the Source. #migrate.migration.article_node.yml source: plugin: article_node key: legacy #target: legacy #settings.php $databases['key']['target'] = array(

1.- SOURCE 3. MIGRATION FRAMEWORK - And which Plugin will make the Query <?php namespace Drupalcm_migratePluginmigratesource; use ... /** * Source plugin for articles content. * * @MigrateSource( * id = "article_node" * ) */ class ArticleNode extends SqlBase { public function query() { $query = $this->select('articles', 'a') ->fields('a', [ 'id', 'user_id', 'title', 'description', 'created', ]); return $query; } public function fields() { $fields = [ 'id' => $this->t("Article ID"), // ... ]; return $fields; } public function getIds() { return [ 'id' => [ 'type' => 'integer', 'alias' => 'a', ], ]; } public function prepareRow(Row $row) { $id = $row->getSourceProperty('id'); //print $id . PHP_EOL; //print_r($row); $row->setSourceProperty('user_id', 1); return parent::prepareRow($row); } }

2.- DESTINATION 3. MIGRATION FRAMEWORK How and where to save the data - Where to save the data. #migrate.migration.article_node.yml destination: plugin: entity:node - entity:<place-here-an-entity>

2.- DESTINATION 3. MIGRATION FRAMEWORK More destination plugins? Search for “destination:” in core

3.- ID MAPPING 3. MIGRATION FRAMEWORKHow Migrate associate old row with new row. public function getIds() { return [ 'id' => [ 'type' => 'string', 'alias' => 'u', ], ]; }

4.- PROCESS 3. MIGRATION FRAMEWORKHow we transform each field, each file or data. - Map fields: Same value as origin. - Modify: Change o process the value - Add: Create new fields from other fields or calculate these fields.

4.- PROCESS 3. MIGRATION FRAMEWORKMap fields. Values are equal in both sides public function query() { $query = $this ->select('articles', 'a') ->fields('a', [ 'created', 'title', 'id', 'body', 'user_id', ]); return $query; } #common mapping process: title: title

4.- PROCESS 3. MIGRATION FRAMEWORKDefaultValue. Add a default value public function query() { $query = $this ->select('articles', 'a') ->fields('a', [ 'created', 'title', 'id', 'body', 'user_id', ]); return $query; } #default value process: type: plugin: default_value default_value: article 'body/format': plugin: default_value default_value: plain_text

4.- PROCESS 3. MIGRATION FRAMEWORKCallable. Values are similars but a transform is needed with a php function public function query() { $query = $this ->select('articles', 'a') ->fields('a', [ 'created_date', 'title', 'id', 'body', 'user_id', ]); return $query; } #'Callable.php' core plugin process: created: plugin: callback source: created_date callable: strtotime

4.- PROCESS 3. MIGRATION FRAMEWORKDedupeEntity. Values in destination cannot be equals, but in origin could be. public function query() { $query = $this ->select('users', 'u') ->fields('u', [ 'user_id', 'user_name', 'mail', ]); return $query; } # DedupeEntity.php' core # plugin process: name: plugin: dedupe_entity source: user_name entity_type: user field: name postfix: _ # admin_1, _2, ...

4.- PROCESS 3. MIGRATION FRAMEWORKMigration. Values from other migration. Magic! # ArticleNode.php public function query() { $query = $this ->select('articles', 'a') ->fields('a', [ 'created_date', 'title', 'id', 'body', 'user_id', ]); return $query; } # Migration as plugin process: field_tags: plugin: migration migration: tags_node source: terms migration_dependencies: required: - tags_node fields()

4.- PROCESS 3. MIGRATION FRAMEWORKMigration. Values from other migration. Magic! # ArticleNode.php public function fields() { $fields = [ 'terms' => $this->t ("New field terms"), // ... ]; return $fields; } # ArticleNode.php public function prepareRow(Row $row) { $terms = $this->select('terms') //... ->fetchCol(); //print_r($terms); $row->setSourceProperty('terms', $terms); return parent::prepareRow($row); } }

4.- PROCESS 3. MIGRATION FRAMEWORKMigration. Values from other migration. Magic! #migrate.migration.tags_node.yml source: plugin: tags_node destination: plugin: entity:taxonomy_term process: name: term parent: plugin: migration migration: tags_node source: parent_term # TagsNode.php public function query() { // code ->fields('terms', [ 'parent_term', 'term']); }

4.- PROCESS 3. MIGRATION FRAMEWORKCustomPlugins. Values are related but a custom transform for data is needed public function query() { $query = $this ->select('articles', 'a') ->fields('a', [ 'created_date', 'title', 'id', 'body', 'user_id', ]); return $query; } #CustomUUID.php' custom plugin process: uid: plugin: custom_uuid source: user_id

4.- PROCESS 3. MIGRATION FRAMEWORK<?php /** * @MigrateProcessPlugin(id = "custom_uuid") */ class CustomUUID extends ProcessPluginBase { public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { $uid = 0; $uids = Drupal::entityQuery('user') ->condition('field_user_old_uuid', $row->getSourceProperty("user_id")) ->execute(); if (count($uids) > 0) { $uid = array_values($uids)[0]; } return $uid; } }

4.- PROCESS 3. MIGRATION FRAMEWORKCustomPlugins. We need to copy some files public function query() { $query = $this ->select('files', 'f') ->fields('f', [ 'id', 'post_id', 'name', 'path', 'dir', ]); return $query; } #CustomFiles.php' custom plugin destination: plugin: entity:file process: fid: id filename: name uri: plugin: custom_file_uri source: - path - name

4.- PROCESS 3. MIGRATION FRAMEWORK// In CustomFiles.php. source plugin. public function prepareRow(Row $row) { // Set the complete external path to the image. $local_path = '/var/www/webroot/files/image/attachments/'; $attachment = $row->getSourceProperty('path'); $dir = $row->getSourceProperty('dir') . "/"; $filepath = $local_path . $dir . $attachment; $row->setSourceProperty('path', $filepath); $file_name = basename($attachment); // Set filename. Not OK in every origin row. $row->setSourceProperty('name', $file_name); return parent::prepareRow($row); }

4.- PROCESS 3. MIGRATION FRAMEWORK<?php /** * @MigrateProcessPlugin(id = "custom_file_uri") */ class CustomFileUri extends ProcessPluginBase { public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { list($filepath, $filename) = $value; $destination_base_uri = 'public://articles/'; print "TRANSFORM Filepath: $filepath" . PHP_EOL; print "TRANSFORM Destination: $destination_base_uri" . "$filename" . PHP_EOL; // public://articles/photo (2).jpg return $destination_base_uri . $filename; } }

4.- PROCESS 3. MIGRATION FRAMEWORKMore core plugins: - concat - extract - flatten - get - iterator - machine_name - menu_link_parent - route - skip_on_empty - skip_row_if_not_set - static_map Need an example? Core is full of examples! Search in core! - “plugin: concat” - “plugin: get” - …

5.- WANT MORE? 3. MIGRATION FRAMEWORKTake a look at: - core/modules/migrate/src/Plugin/migrate - core/modules/<any>/src/Plugin/migrate - core/modules/<any>/config/migration_templates Or search in core: - “@MigrateSource(“ - “destination:” - “migration_tags:”

Drupal 8 core is full of examples 3. MIGRATION FRAMEWORK Really!

THINK THAT MIGRATE IS COMPLEX? 4. D2D MIGRATIONMaybe. And sometimes is difficult to debug. So… Here we have: Migrate Drupal module And: - Migrate Upgrade - Migrate Plus - Migrate Tools *experimental

REQUIREMENTS 4. D2D MIGRATION- Fresh install of Drupal 8 - Modules enabled: - Migrate, - Drupal Migrate, - Migrate upgrade - Access to D6 or D7 database - Access to D6 or D7 files

SET UP 4. D2D MIGRATIONGo to: “/upgrade”.

4. D2D MIGRATION #migrate.migration.article_node.yml source: plugin: custom_user key: legacy #equal as standard migration 1.- SOURCE We tell here to DrupalSqlBase: - Which Database is the Source.

4. D2D MIGRATION 1.- SOURCE Class now extends from DrupalSqlBase class User extends DrupalSqlBase implements SourceEntityInterface { public function query() { return $this->select('users', 'u') ->fields('u', [//Fields]) ->condition('uid', 0, '>'); }

4. D2D MIGRATION 1.- SOURCE public function fields() { $fields = $this->baseFields(); $fields['first_name'] = $this->t('First Name'); $fields['last_name'] = $this->t('Last Name'); $fields['biography'] = $this->t('Biography'); return $fields; } - Can use baseFields and add more fields

4. D2D MIGRATION 1.- SOURCE public function prepareRow(Row $row) { $uid = $row->getSourceProperty('uid'); // first_name $result = $this->getDatabase()->query(' SELECT fld.field_first_name_value FROM {field_data_field_first_name} fld WHERE fld.entity_id = :uid ', array(':uid' => $uid)); foreach ($result as $record) { $row->setSourceProperty('first_name', $record->field_first_name_value ); } - Prepare the Rows

4. D2D MIGRATION Or give a try to ...

4. D2D MIGRATION drush migrate-manifest --legacy-db-url=... D6Manifest-User.yml migrate-manifest https://www.youtube.com/watch?v=l4m5msEY-Jg

4. D2D MIGRATION #D6Manifest-User.yml - d6_user - d6_user_profile_field - d6_user_profile_field_instance - d6_user_profile_entity_display - d6_user_profile_entity_form_display - d6_profile_values:user - d6_filter_format - d6_user_role - d6_user_picture_entity_display - d6_user_picture_entity_form_display - d6_user_picture_file - d6_user_picture_field - d6_user_picture_field_instance Manifest List of migrations

4. D2D MIGRATION Manifest Migrations that are in core.

4. D2D MIGRATION And something is coming to 8.1.x

THANKS! @isholgueras nacho@letshackity.com

Migrating data to drupal 8

More Related Content

What's hot

Viewers also liked

Similar to Migrating data to drupal 8

Recently uploaded

Migrating data to drupal 8