Soong 0.6.0 released

The 0.6.0 release of the Soong ETL library is now available on Packagist. Key changes since 0.5.3 include:

  1. The Filter component has been introduced, which examines a Record and "approves" it for further processing, along with a filters configuration option added to extractors to limit what gets extracted. The Select filter and migrate command option --select have been added - extractor results may thus be filtered either in the base configuration for the extractor (representing the canonical set of data to be processed) or at runtime (for testing). An example of the latter, restricting input to records that have both a low id and a high foo value:

    bin/soong migrate arraytosql --select='id<8' --select='foo>g'
  2. Dependency injection has been improved - where previously class names were passed through to be instantiated by the components that needed them, responsibility for constructing all (almost, see below) components now belongs to the application (for now, the Symfony console commands being the single example) which will inject the class instances.
  3. Since components do need to dynamically generate Property and Record instances during migration, in those cases a PropertyFactory or RecordFactory instance is injected.
  4. The DataProperty and DataRecord classes have been renamed to Property and Record respectably.
  5. Error handling has been beefed up, with several exception classes added.
  6. We're now integrated with Scrutinizer for coverage and quality analysis. Unfortunately, Scrutinizer does not yet allow public access to analysis for Gitlab-based projects, so you can't look for yourself - but here's where we are now (having built out the tests further):Code quality/coverage of Soong

At the moment, the main emphases for 0.7.0 are:

  1. Build out the console commands to be more-or-less feature-complete.
  2. Refactor the transformation pipeline.
  3. Refactor our approach to tasks, in particular looking at using or gaining inspiration from other tools such as Robo or PortPHP.

Obligatory self-promotion - I'm available for data migration projects.


Use the Twitter thread below to comment on this post: