KIT | KIT-Bibliothek | Impressum | Datenschutz
Open Access Logo
§
Verlagsausgabe
DOI: 10.5445/IR/1000087757
Veröffentlicht am 23.11.2018
Originalveröffentlichung
DOI: 10.1088/1742-6596/1085/3/032005

Provisioning of data locality for HEP analysis workflows

Heidecker, C.; Schnepf, M. J.; Kuehn, E.; Fischer, M.; Giffels, M.; Quast, G.

Abstract:
The heavily increasing amount of data produced by current experiments in high energy particle physics challenge both end users and providers of computing resources. The boosted data rates and the complexity of analyses require huge datasets being processed in short turnaround cycles. Usually, data storages and computing farms are deployed by different providers, which leads to data delocalization and a strong influence of the interconnection transfer rates. The CMS collaboration at KIT has developed a prototype enabling data locality for HEP analysis processing via two concepts. A coordinated and distributed caching approach that reduce the limiting factor of data transfers by joining local high performance devices with large background storages were tested. Thereby, a throughput optimization was reached by selecting and allocating critical data within user work-flows. A highly performant setup using these caching solutions enables fast processing of throughput dependent analysis workflows.


Zugehörige Institution(en) am KIT Institut für Experimentelle Teilchenphysik (ETP)
Steinbuch Centre for Computing (SCC)
Publikationstyp Zeitschriftenaufsatz
Jahr 2018
Sprache Englisch
Identifikator ISSN: 1742-6588, 1742-6596
URN: urn:nbn:de:swb:90-877578
KITopen-ID: 1000087757
HGF-Programm 51.01.01 (POF III, LK 01)
Erschienen in Journal of physics / Conference Series
Band 1085
Heft 3
Seiten Art. Nr.: 032005
Bemerkung zur Veröffentlichung 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, ACAT 2017; University of WashingtonSeattle; United States; 21 August 2017 through 25 August 2017
Nachgewiesen in Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft KITopen Landing Page