VTL 1.1 (Validation and Transformation Language) in public review until 17.12.2016
The Task force for the Validation and Transformation Language (VTL), created in 2013 under the initiative of the SDMX Secretariat, submits for public review the version 1.1 of VTL. The public review extends until 17 December 2016, in order to allow time for the consolidation of a final version at the beginning of 2017.
VTL is a standard language for defining validation and transformation rules (set of operators, their syntax and semantics) for any kind of statistical data. VTL 1.1 takes advantage of all the comments and proposals received after the publication of version 1.0, in March 2015. The new version contains several new features, together with an extended library of operators covering more use cases.
The initiative builds on the generic framework for defining mathematical expressions existing in the SDMX information model but the intention is to provide a language that is usable with other standards as well for expressing logical validation rules and transformations on data.
The draft package includes:
- Part 1: the user manual, highlighting the main characteristics of VTL, its key assumptions and the information model the language is based on. Part 1 also includes a formal description of data validation objects against which information models can be mapped.
- Part 2: the reference manual, containing the full library of operators ordered by category, including examples; this version can support validation and basic compilation needs.
The eBNF (extended Backus-Naur Form), i.e. the technical notation for use in the implementation, will be added as soon as possible, during the public review period.
Please send your comments to the SDMX Technical Working Group (SDMX-TWG) at email@example.com. Each email message should only contain a limited set of comments – possibly homogeneous – and all the elements indicated in the attached Comment Log.
Published on: 17 October 2016
VTL 1.0 (March 2015)
VTL is a standard language for defining validation and transformation rules (set of operators, their syntax and semantics) for any kind of statistical data. VTL builds on the Transformation section of the SDMX information model, by taking the common parts of GSIM, SDMX and DDI as for the representation of concepts and data. The assumption is that such a logical formalization of validation and transformation rules will provide a “technology-neutral” expression at business level of the processing taking place, against which various implementations and specific programming languages can be mapped. The specifications for exchanging VTL validation rules in SDMX messages, for storing rules and for requesting validation rules from web services will be provided in a specific update to the SDMX Technical Standards on which the SDMX Technical Working Group is working on. The VTL 1.0 package includes:
- Part 1, highlighting the main characteristics of VTL, its core assumptions and the information model the language is based on;
- Part 2, containing the full library of operators ordered by category, including examples; this first version can support validation and basic compilation needs. Future versions, whose design is already in progress, will include more features related to transformation of data;
- EBNF notation (Extended Backus-Naur Form) which is the technical notation to be used as a test bed for all the examples throughout the document.