KIT | KIT-Bibliothek | Impressum | Datenschutz

DS2STAC: A Python package for harvesting and ingesting (meta)data into STAC-based catalog infrastructures

Hadizadeh, Mostafa 1; Lorenz, Christof ORCID iD icon 1; Barthlott, Sabine ORCID iD icon 2; Fösig, Romy ORCID iD icon 3; Çayoğlu, Uğur 4; Ulrich, Robert ORCID iD icon 5; Bach, Felix
1 Institut für Meteorologie und Klimaforschung – Atmosphärische Umweltforschung (IMK-IFU), Karlsruher Institut für Technologie (KIT)
2 Institut für Meteorologie und Klimaforschung – Atmosphärische Spurenstoffe und Fernerkundung (IMK-ASF), Karlsruher Institut für Technologie (KIT)
3 Institut für Meteorologie und Klimaforschung Atmosphärische Aerosolforschung (IMKAAF), Karlsruher Institut für Technologie (KIT)
4 Scientific Computing Center (SCC), Karlsruher Institut für Technologie (KIT)
5 KIT-Bibliothek (BIB), Karlsruher Institut für Technologie (KIT)

Abstract:

Despite the vast growth in accessible data from environmental sciences over the last decades, it remains difficult to make this data openly available according to the FAIR principles. A crucial requirement for this is the provision of metadata through standard catalog interfaces or data portals for indexing, searching, and exploring the stored data. With the release of the community-driven Spatio-Temporal Assets Catalog (STAC), this process has been substantially simplified as STAC is based on highly flexible and lightweight GeoJSONs instead of large XML-files. The number of STAC-users has hence rapidly increased and STAC now features a comprehensive ecosystem with numerous extensions. This is also why we have chosen STAC as our central catalog framework in our research project Cat4KIT, in which we develop an open-source software stack for the FAIRification of environmental research data. A central element of this project is the automatic (meta)data and service harvesting from different data servers, providers and services. This so-called DS2STAC-module hence contains tailormade harvesters for different data sources and services, a metadata validator and a database for storing the STAC items, collections and catalogs. ... mehr


Originalveröffentlichung
DOI: 10.5281/zenodo.8086566
Zugehörige Institution(en) am KIT Institut für Meteorologie und Klimaforschung – Atmosphärische Spurenstoffe und Fernerkundung (IMK-ASF)
Institut für Meteorologie und Klimaforschung – Atmosphärische Umweltforschung (IMK-IFU)
Institut für Meteorologie und Klimaforschung Atmosphärische Aerosolforschung (IMKAAF)
KIT-Bibliothek (BIB)
Scientific Computing Center (SCC)
Publikationstyp Vortrag
Publikationsdatum 09.06.2023
Sprache Englisch
Identifikator KITopen-ID: 1000162856
HGF-Programm 12.11.35 (POF IV, LK 01) Tailored information for users and stakeholders
Veranstaltung 8th Data Science Symposium (2023), Kiel, Deutschland, 08.06.2023 – 09.06.2023
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page