LSDMA-Driven Advances in Data Analysis

Besides data management, the analysis of research data is another important aspect covered by LSDMA. The overarching goal of all data efforts must be to start managing the scientist’s data right after the data left the data acquisition device as this is the only way to capture gapless provenance information. This is inevitable for transparent and reproducible science. However, this means also that the research data is at the very beginning of its lifecycle. In order to obtain publishable results the data often has to go through several processing steps until the final results are available. Such processing steps can reach from basic scripts to complex scientific workflows consisting of several dependent processing steps. In order to achieve the aforementioned reproducibility capturing the data provenance, e.g. what happened with the data and led to which results, should be an essential part of every processing step. This article describes the efforts taken by the LSDMA project in order to provide scientists with data analysis capabilities integrated seamlessly into data management workflows. For this, two approaches were chosen: The integration of data processing into an existing data repository system and the extension of an existing computing middleware by automated processing of data stored next to the computing environment. ... mehrThis article presents both approaches, realized use cases and finally gives a summary of the LSDMA efforts in term of data analysis.