Drupal file migrations: The s3fs module

A recent project gave me the opportunity to familiarize myself with the Drupal 8 version of the S3 File System (s3fs) module (having used the D7 version briefly in the distant past). This module provides an s3:// stream wrapper for files stored in an S3 bucket, allowing them to be used as seamlessly as locally stored public and private files. First we present the migrations and some of the plugins implemented to support import of files stored on S3 - below we will go into some of the challenges we faced.

Disabling functionality temporarily during migration

Continuing with techniques from the “Acme” project, the location content type had an address field and a geofield, with field_geofield configured to automatically determine latitude and longitude from the associated field_address - a fact I was initially unaware of. Our source data contained latitude and longitude already, which I mapped directly in the migration:

Configuring migrations via a form

Frequently, there may be parts of a migration configuration which shouldn’t be hard-coded into your YAML file - some configuration may need to be changed periodically, some may vary according to environment (for example, a dev environment may access a dev or test API endpoint, while prod needs to access a production endpoint), or you may need a password or other credentials to access a secure endpoint (or for a database source which you can’t put into settings.php). You may also need to upload a data file for input into your migration.

Back in the saddle again - at DrupalCon Nashville

Return to action

Those of you with an interest in Drupal migration may have noticed my absence in the last several months. A confluence of things led me to take a break from the Drupal community: a bout of physical exhaustion (initially diagnosed as Lyme disease, and then ¯\_(ツ)_/¯); professional exhaustion managing the D8 migration core initiative and maintaining a few contrib modules on top of paid contracts; and emotional exhaustion from the community drama of early last year.

Translation migration and more

I’m running a bit behind, but I want to make it a practice to blog about each migration project I’m involved with (with each client’s permission, of course). Constructed examples are all well and good, but there’s nothing like real-world scenarios to give the flavor of using the migration framework in practice.