Managing Large Data Sets in Super-resolution Optical Microscopy

Abstract Localization microscopy, especially SPDM (Spectral Position Determination Microscopy), can scale optical resolution down almost to the electron microscopy level in the 10 nm range, which is important for biological and medical research and diagnosis. But these techniques produce image data in the range of GB/s and require the handling, processing and evaluation of image stacks of up to thousands of frames per single cell. These data have to be stored and made accessible for the research community, in diagnostic connotation for 30 years by law. To this end, we have designed a system for transmitting the data, adding meta data for characterization and retrieval, storing the data, and also offering programs and processing procedures for fast evaluation, on individual machines or in clusters. A Generic Client Service (GCS) API for connecting disparate services is designed and implemented seamlessly integrating with the KIT Data Manager and the Large Scale Data Storage. A structured metadata model based on Core Scientific Metadata Model (CSMD) is established for describing the extremely large datasets of localization microscopy research. ... mehrStandardised descriptions of the workflow steps with an automated execution of the workflow, based on extended image analysis programs is achieved by a workflow management system (WfMS).