Soong 0.6.0 released
The 0.6.0 release of the Soong ETL library is now available on Packagist. Key changes since 0.5.3 include:
- The
Filter
component has been introduced, which examines aRecord
and "approves" it for further processing, along with afilters
configuration option added to extractors to limit what gets extracted. TheSelect
filter andmigrate
command option--select
have been added - extractor results may thus be filtered either in the base configuration for the extractor (representing the canonical set of data to be processed) or at runtime (for testing). An example of the latter, restricting input to records that have both a low id and a high foo value:bin/soong migrate arraytosql --select='id<8' --select='foo>g'
- Dependency injection has been improved - where previously class names were passed through to be instantiated by the components that needed them, responsibility for constructing all (almost, see below) components now belongs to the application (for now, the Symfony console commands being the single example) which will inject the class instances.
- Since components do need to dynamically generate
Property
andRecord
instances during migration, in those cases aPropertyFactory
orRecordFactory
instance is injected.
- The
DataProperty
andDataRecord
classes have been renamed to Property and Record respectably.
- Error handling has been beefed up, with several exception classes added.
- We're now integrated with Scrutinizer for coverage and quality analysis. Unfortunately, Scrutinizer does not yet allow public access to analysis for Gitlab-based projects, so you can't look for yourself - but here's where we are now (having built out the tests further):
At the moment, the main emphases for 0.7.0 are:
- Build out the console commands to be more-or-less feature-complete.
- Refactor the transformation pipeline.
- Refactor our approach to tasks, in particular looking at using or gaining inspiration from other tools such as Robo or PortPHP.
Obligatory self-promotion - I'm available for data migration projects.
Thanks!
Use the Twitter thread below to comment on this post: