VTL 1.1 (Validation and Transformation Language) in progress after public review
The Task force for the Validation and Transformation Language (VTL), created in 2013 under the initiative of the SDMX Secretariat, is working on version 1.1 of VTL. The public review version was released in October 2016 and was commented until January 2017. The consolidation of its final version is ongoing.
VTL is a standard language for defining validation and transformation rules (set of operators, their syntax and semantics) for any kind of statistical data. VTL 1.1 takes advantage of all the comments and proposals received after the publication of version 1.0, in March 2015. The new version contains several new features, together with an extended library of operators covering more use cases.
The initiative builds on the generic framework for defining mathematical expressions existing in the SDMX information model but the intention is to provide a language that is usable with other standards as well as for expressing logical validation rules and transformations on data.
The draft VTL 1.1 package includes:
- Part 1: the user manual, highlighting the main characteristics of VTL, its key assumptions and the information model the language is based on. Part 1 also includes a formal description of data validation objects against which information models can be mapped.
- Part 2: the reference manual, containing the full library of operators ordered by category, including examples; this version can support validation and basic compilation needs.
The eBNF (extended Backus-Naur Form), i.e. the technical notation for use in the implementation, will be added at the end of the revision process.
For more info, please send a mail to the SDMX Technical Working Group (SDMX-TWG) at firstname.lastname@example.org.
Published on: 8 February 2017
VTL 1.0 (March 2015)
VTL is a standard language for defining validation and transformation rules (set of operators, their syntax and semantics) for any kind of statistical data. VTL builds on the Transformation section of the SDMX information model, by taking the common parts of GSIM, SDMX and DDI as for the representation of concepts and data. The assumption is that such a logical formalization of validation and transformation rules will provide a “technology-neutral” expression at business level of the processing taking place, against which various implementations and specific programming languages can be mapped. The specifications for exchanging VTL validation rules in SDMX messages, for storing rules and for requesting validation rules from web services will be provided in a specific update to the SDMX Technical Standards on which the SDMX Technical Working Group is working on. The VTL 1.0 package includes:
- Part 1, highlighting the main characteristics of VTL, its core assumptions and the information model the language is based on;
- Part 2, containing the full library of operators ordered by category, including examples; this first version can support validation and basic compilation needs. Future versions, whose design is already in progress, will include more features related to transformation of data;
- EBNF notation (Extended Backus-Naur Form) which is the technical notation to be used as a test bed for all the examples throughout the document.