A General Schema for Time Series Data Quality Guided by Real-World Use Cases and Based on International Standards

The World Wide Web Consortium (W3C) provides general best practices for including data quality information in data shared over the web. However, their implementation in practice often requires mapping or interpreting W3C concepts to the application domain. We report on a concrete approach to implementing the W3C Data Quality recommendations into time series data, thereby applying them to a wide range of scientific processes. 

To that end, we propose a general schema for modeling data quality control information for time series data. The schema is guided by prominent use cases from the Earth and environmental sciences. It incorporates data quality flags, as well as processing information from automated quality control procedures and data inspections by domain experts. We provide a concrete implementation of the schema in the SensorThings API data model. Additionally, we demonstrate how file-based time series data can be annotated using the proposed schema in RO-Crates and the NetCDF format. By deeply integrating the W3C standard, we obtain a practice-oriented, semantically sound schema. We demonstrate the schema’s implementation for its initial use cases and provide additional relevant examples.    The proposed schema realizes quality control in the SensorThings API data model and for file-based time series data. ... mehrOur approach preserves the original domain-specific structures while ensuring compliance with the W3C recommendations. Thus, we offer a straightforward plan to improve the readability and machine actionability of existing data quality information and corresponding workflows across domains. We even enable their interoperability on an international level.