An Interactive FAIR Data Publishing System for Time Series Data

An important point for the widespread dissemination of FAIR-data is the lowest possible entry barrier for preparing and providing data to other scientists according to the FAIR criteria. If scientists have to manually extract, transform and annotate the data according to the FAIR criteria and then export it to make it available to the public, this requires a significant investment of time that does not primarily reward the scientists who prepares and provides the data. The Energy-Lab at KIT is running a large cluster of an Influx database management system in which energy related time-series data being stored in a variety of individual databases for over 10 years. In order to increase the willingness to make data available to the scientific public, we have developed a web application that greatly supports and automates the publication and annotation process of time-series-data stored in Influx databases.
The web application consists of a backend service and an interactive frontend. The backend provides arbitrary, predefined and annotated time-series-data of a measurement (an Influx database structure that corresponds to a table in a relational database) via an URL without requiring any further information for access. ... mehrThe specification of the data is done via HTTP-GET parameters.
These include the desired time interval and specific conditions on the attributes as well as a configuration file in which the Influx-server access information are stored. The actual request is made by a series of REST API calls to the InfuxDB. In order to be able to extract arbitrarily large amounts of data, a stream-based approach was chosen. The data is returned as an ROCrate dataset, using the CSVW Namespace to describe tabular data. The column data-types are extracted from metadata calls to the Influx database. Further information about the attributes (like unit, quantity) can be specified in the configuration file.
The frontend implements the interactive construction of the URL for reading out the timeseries-data. The first step is to select the specific configuration file stored for a particular measurement, which contains the information (see above) for accessing a specific database. This information is used, to access the measurement and determine the time interval for which data is available. Meta information of the measurement is read out including the attributes with their data types. In addition, for attributes which act as tags (descriptive attributes), the existing tag-values are extracted. These attributes can be used to interactively formulate extraction conditions (e.g. only data of certain buildings, devices,...). Finally, the time interval of the data to be extracted must be specified. The result of this step is a URL, conforming to the backend API, to export the data in the appropriate format. Currently all attributes (tags & fields) are returned.
In a future version we also plan that it is possible to specify which attributes should be returned. Further extensions include a cache mechanism, where only dynamic data must be retrieved on every request while historical data requests can be fulfilled from a cache.