Zeitgeist : An Interactive FAIR Data Publishing System for Time Series Data

An important point for the widespread dissemination of FAIR-data is the lowest possible entry<br> barrier for preparing and providing data to other scientists according to the FAIR criteria. If<br> scientists have to manually extract, transform and annotate the data according to the FAIR<br> criteria and then export it to make it available to the public, this requires a significant investment<br> of time that does not primarily reward the scientists who prepares and provides the data.<br> The Energy-Lab at KIT is running a large cluster of an Influx database management system in<br> which energy related time-series data being stored in a variety of individual databases for over 10<br> years. In order to increase the willingness to make data available to the scientific public, we have<br> developed a web application that greatly supports and automates the publication and annotation<br> process of time-series-data stored in Influx databases.<br> The web application consists of a backend service and an interactive frontend. The backend<br> provides arbitrary, predefined and annotated time-series-data of a measurement (an Influx database<br> structure that corresponds to a table in a relational database) via an URL without requiring any<br> further information for access. ... mehrThe specification of the data is done via HTTP-GET parameters.<br> These include the desired time interval and specific conditions on the attributes as well as a<br> configuration file in which the Influx-server access information are stored. The actual request is<br> made by a series of REST API calls to the InfuxDB. In order to be able to extract arbitrarily<br> large amounts of data, a stream-based approach was chosen. The data is returned as an ROCrate<br> dataset, using the CSVW Namespace to describe tabular data. The column data-types are<br> extracted from metadata calls to the Influx database. Further information about the attributes<br> (like unit, quantity) can be specified in the configuration file.<br> The frontend implements the interactive construction of the URL for reading out the timeseries-<br> data. The first step is to select the specific configuration file stored for a particular measurement,<br> which contains the information (see above) for accessing a specific database. This<br> information is used, to access the measurement and determine the time interval for which data<br> is available. Meta information of the measurement is read out including the attributes with their<br> data types. In addition, for attributes which act as tags (descriptive attributes), the existing<br> tag-values are extracted. These attributes can be used to interactively formulate extraction conditions<br> (e.g. only data of certain buildings, devices,...). Finally, the time interval of the data to be<br> extracted must be specified. The result of this step is a URL, conforming to the backend API, to<br> export the data in the appropriate format. Currently all attributes (tags &amp; fields) are returned.<br> In a future version we also plan that it is possible to specify which attributes should be returned.<br> Further extensions include a cache mechanism, where only dynamic data must be retrieved on<br> every request while historical data requests can be fulfilled from a cache.