Drupal 8 migration - all the modules
Now that Drupal 8.0.0 is released, let's take a look at all the components of the Drupal migration ecosystem and where they stand now. To provide context to those familiar with migration under Drupal 7, I will show where the pieces came from. First, general migration support:
| Feature | D7 | D8 | D8 status |
| Basic migration framework | migrate (contrib) | migrate (core), plus destination plugins in core modules | Experimental |
| General-purpose drush commands | migrate (contrib) | migrate_tools (contrib) | The most common commands (ms, mi, mr, etc.) are working |
| General-purpose UI | migrate_ui submodule of migrate (contrib) | migrate_tools (contrib) | Just basic display of migrations and their status so far |
| CSV source plugin | migrate (contrib) | migrate_source_csv (contrib) | Working |
| XML source plugins | migrate (contrib) | migrate_source_xml (contrib) | A basic initial implementation exists, still needs work |
| JSON source plugin | migrate (contrib) | migrate_source_json (contrib) | A basic initial implementation exists, still needs work |
| List source plugin | migrate (contrib) | migrate_plus (contrib)? | Currently assessing how to architect this in D8 - most likely will be helper classes or traits in migrate_plus |
| Other source plugins | migrate (contrib) | Contrib | To be implemented as needed, by the people who need them |
| Migration groups | migrate (contrib) | migrate_plus (contrib) | Hopefully we'll finally get this into core for 8.1 |
| Basic programming examples | migrate_example submodule of migrate (contrib) | migrate_example submodule of migrate_plus (contrib) | Working but needs an image migration example |
| Advanced programming examples | migrate_example submodule of migrate (contrib) | migrate_example_advanced submodule of migrate_plus (contrib) | Skeletal at the moment |
| Migration support for contrib modules | Ideally in the contrib modules themselves - some legacy handlers are still in migrate_extras | Contrib | Each contrib module is responsible for implementing its own destination plugins (for general migrations), plus migrations and source plugins for migrations/upgrades from another Drupal site |
| Migration from manifest files (lists of migrations with configuration overrides) | N/A | migrate_manifest (contrib) | Only supports template-based migrations from SQL sources |
And then, features specific to migrating from another Drupal site:
| Feature | D7 | D8 | D8 status |
| Major Drupal version upgrade (drush command) | pm-update (in Drush itself) | migrate_upgrade (contrib) | Working - to be moved to Drush itself (hopefully by Drupal 8.1.0) |
| Major Drupal version upgrade (UI) | /update.php | migrate_upgrade (contrib) | Working - to be moved to Drupal core (hopefully by Drupal 8.1.0) |
| Framework for migration from Drupal 6 and 7. | migrate_d2d (contrib) | migrate_drupal (core), plus migrations and source/process plugins in core modules | Experimental. The D6 support is essentially complete; D7 is lacking support for some secondary core modules, file/image fields, etc. |
| Framework for migration from Drupal 5. | migrate_d2d (contrib) | migrate_drupal_d5 (contrib) | Very basic support for key core data (taxonomy, users, nodes). Not updated in a few months and probably needs rerolling for 8.0.0. |
| Framework for migration from Drupal 8 | N/A | migrate_drupal_d8? (not yet created) | Planned. Most likely will be initially implemented in contrib, to be submitted to core in a later minor release. |
| Support for custom migrations from Drupal | migrate_d2d (contrib) | migrate_d2d (contrib) | Not yet begun |
Notes on the statuses above:
- The migration support in core is currently marked "Experimental". This means there are still some holes to fill (particularly for upgrading from Drupal 7), and still some leeway to make backwards-compatibility-breaking changes for Drupal 8.1.
- I say "Working" rather than "Supported" for a few things above because A. All of these components are dependent on the core framework which is still subject to change, and B. To this point none of this has had substantial real-world testing, so there are likely significant issues still to be addressed.
So, you may be asking yourself "what can I do today, and what modules do I need to do it?"
I want to upgrade from Drupal 6 or 7 to Drupal 8
You can do this today with migrate_upgrade. As stated above, the Drupal 6 migration path for core modules is pretty much complete. If you're looking at upgrading from Drupal 7, review the meta issue for Drupal 7 migrations to see if any of the missing pieces are important to your site. See my previous blog posts for detailed run-throughs of performing the upgrade through the UI or with drush.
I want to implement an upgrade path for my contributed or custom module
Unfortunately, documentation is lacking at the moment to help module authors implement upgrade paths from previous Drupal versions - I hope to work with others on this (and also post to my blog) in the coming weeks. For today, the tl;dr is:
- Contributed modules are no different from core modules when it comes to the upgrade path (with one small caveat below), so
- Find the core module most similar to your module in terms of the kind of configuration and data it manages, and see how its upgrade path is implemented (specifically, look in its
migration_templatesandsrc/Plugin/migrate directories).
The one caveat is that for non-core modules, running the upgrade process through the UI will incorrectly report that no migration path is found for your module. This is due to the fact that currently there is no automated way to identify the source module for a migration, thus the UI is working from a hard-coded table of core migrations. This is a cosmetic issue only - your module's migration will still be run. I will also note that in terms of debugging your upgrade path, the drush migrate-upgrade command has a very handy --configure-only option, which runs the configuration portion of the upgrade (turning migration templates into migration configuration entities) without actually running the imports. When debugging a specific upgrade migration, I will use this option, then (with migrate_tools installed) import all the migrations previous to the one I'm testing. At this point I can do drush sql-dump >/tmp/backup.sql. It's much faster to test the migration, tweak it, and restore the backup for another try than it is to go all the way back to the beginning.
I want to perform a custom migration from Drupal
Ultimately, there should be tools like the migrate_d2d UI in Drupal 7 to enable you to customize what you're importing and how it's mapped into your Drupal 8 installation - but that's going to take some time. At this point, the simplest approach is to run the drush migrate-upgrade --configure-only command to generate the migration configuration for a straight upgrade, export that configuration to config/install in a custom module, and edit the migration .yml files to suit your needs.
I want to perform a custom migration from not-Drupal
You'll need the migrate_tools (drush commands and basic UI for running and managing migrations) and migrate_plus (provides group functionality migrate_tools needs, plus examples) modules. In migrate_plus, open up migrate_example/README.txt and it will guide you through the basics of implementing SQL-based migrations. It's simpler than you think! If you're importing from source data not in an SQL database, there are CSV, XML, and JSON source plugins available. It's still well worth going through migrate_example to understand the general structure of migrations - it's just that the configuration (and potential extension) of the source plugin will differ.
I want to help
Obviously, there's a lot left to do here, both in the core framework and in the contrib tools. Please, by all means, report any issues you find in the relevant issue queues, and feel free to submit patches to fix bugs or add additional features. At this stage, the best thing you can do (especially if you're not a developer) is test the upgrade path for your Drupal 6 or Drupal 7 site and report what you find. The diversity of Drupal sites in the wild is staggering - to get the upgrade path nailed down and fully supported in Drupal 8.1, we need a wide variety of sites to go through it.