KIT | KIT-Bibliothek | Impressum | Datenschutz

Architectural Concept and Evaluation of a Framework for the Efficient Automation of Computational Scientific Workflows: An Energy Systems Analysis Example

Liu, Jianlei 1; Braun, Eric 1; Düpmeier, Clemens 1; Kuckertz, Patrick; Ryberg, D.; Robinius, Martin; Stolten, Detlef; Hagenmeyer, Veit 1
1 Institut für Automation und angewandte Informatik (IAI), Karlsruher Institut für Technologie (KIT)

Abstract (englisch):

Scientists and engineers involved in the design of complex system solutions use computational workflows for their evaluations. Along with growing system complexity, the complexity of these workflows also increases. Without integration tools, scientists and engineers are often highly concerned with how to integrate software tools and model sets, which hinders their original research or engineering aims. Therefore, a new framework for streamlining the creation and usage of automated computational workflows is introduced in the present article. It uses state-of-the-art technologies for automation (e.g., container-automation) and coordination (e.g., distributed message oriented middleware), and a microservice-based architecture for novel distributed process execution and coordination. It also supports co-simulations as part of larger workflows including additional auxiliary computational tasks, e.g., forecasting or data transformation. Using Apache NiFi, an easy-to-use web interface is provided to create, run and control workflows without the need to be concerned with the underlying computing infrastructure. Initial framework testing via the implementation of a real-world workflow underpins promising performance in the realms of parallelizability, low overheads and reliable coordination.


Verlagsausgabe §
DOI: 10.5445/IR/1000091407
Veröffentlicht am 21.02.2019
Originalveröffentlichung
DOI: 10.3390/app9040728
Scopus
Zitationen: 6
Web of Science
Zitationen: 5
Dimensions
Zitationen: 6
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Automation und angewandte Informatik (IAI)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2019
Sprache Englisch
Identifikator ISSN: 2076-3417
urn:nbn:de:swb:90-914075
KITopen-ID: 1000091407
HGF-Programm 37.98.11 (POF III, LK 01) ES 2050
Erschienen in Applied Sciences
Verlag MDPI
Band 9
Heft 4
Seiten Article no: 728
Bemerkung zur Veröffentlichung Gefördert durch den KIT-Publikationsfonds
Vorab online veröffentlicht am 20.02.2019
Schlagwörter automation; microservices; computing cluster; energy system; data processing; co-simulation; parallelization; coordination
Nachgewiesen in Web of Science
Scopus
Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page