KIT | KIT-Bibliothek | Impressum | Datenschutz

Provisioning of data locality for HEP analysis workflows

Heidecker, C. 1; Schnepf, M. J. 1; Kuehn, E. ORCID iD icon 1; Fischer, M. ORCID iD icon 1; Giffels, M. ORCID iD icon 1; Quast, G. ORCID iD icon 1
1 Karlsruher Institut für Technologie (KIT)

Abstract:

The heavily increasing amount of data produced by current experiments in high energy particle physics challenge both end users and providers of computing resources. The boosted data rates and the complexity of analyses require huge datasets being processed in short turnaround cycles. Usually, data storages and computing farms are deployed by different providers, which leads to data delocalization and a strong influence of the interconnection transfer rates. The CMS collaboration at KIT has developed a prototype enabling data locality for HEP analysis processing via two concepts. A coordinated and distributed caching approach that reduce the limiting factor of data transfers by joining local high performance devices with large background storages were tested. Thereby, a throughput optimization was reached by selecting and allocating critical data within user work-flows. A highly performant setup using these caching solutions enables fast processing of throughput dependent analysis workflows.


Verlagsausgabe §
DOI: 10.5445/IR/1000087757
Veröffentlicht am 23.11.2018
Originalveröffentlichung
DOI: 10.1088/1742-6596/1085/3/032005
Dimensions
Zitationen: 1
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Experimentelle Teilchenphysik (ETP)
Scientific Computing Center (SCC)
Universität Karlsruhe (TH) – Zentrale Einrichtungen (Zentrale Einrichtungen)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2018
Sprache Englisch
Identifikator ISSN: 1742-6588, 1742-6596
urn:nbn:de:swb:90-877578
KITopen-ID: 1000087757
HGF-Programm 51.01.01 (POF III, LK 01) Teilchenphysik
Erschienen in Journal of physics / Conference Series
Verlag Institute of Physics Publishing Ltd (IOP Publishing Ltd)
Band 1085
Heft 3
Seiten Art. Nr.: 032005
Bemerkung zur Veröffentlichung 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, ACAT 2017; University of WashingtonSeattle; United States; 21 August 2017 through 25 August 2017
Nachgewiesen in Scopus
Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page