KIT | KIT-Bibliothek | Impressum | Datenschutz

Adoption of ARC-CE and HTCondor at GridKa Tier 1

Fischer, Max; Kuehn, Eileen; Schnepf, Matthias; Petzold, Andreas; Heiss, Andreas

Abstract (englisch):
The GridKa Tier 1 data and computing center hosts a significant share of WLCG processing resources. Providing these resources to all major LHC and other VOs requires efficient, scalable and reliable cluster management. To satisfy this, GridKa has recently migrated its batch resources from CREAM-CE and PBS to ARC-CE and HTCondor. This contribution discusses the key highlights of the adoption of this middleware at the scale of a European Tier 1 center: As the largest WLCG Tier 1 using the ARC-CE plus HTCondor stack, GridKa is exemplary for migrating more than 20 000 cores over the time span of only a few weeks. Supporting multiple VOs, we have extensively studied the constraints and possibilities of scheduling jobs of vastly different requirements. We present a robust and maintainable optimization of resource utilization which still respects constraints desired by VOs. Furthermore, we explore the dynamic extension of our batch system, integrating cloud resources with a lightweight configuration mechanism.

Open Access Logo

Verlagsausgabe §
DOI: 10.5445/IR/1000105787
Veröffentlicht am 12.02.2020
DOI: 10.1051/epjconf/201921403053
Cover der Publikation
Zugehörige Institution(en) am KIT Grid Computing Centre Karlsruhe (GridKa)
Steinbuch Centre for Computing (SCC)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2019
Sprache Englisch
Identifikator ISSN: 2100-014X
KITopen-ID: 1000105787
HGF-Programm 53.52.02 (POF III, LK 02) GridKa
Erschienen in The European physical journal / Web of Conferences
Verlag EDP Sciences
Band 214
Seiten Art.-Nr.: 03053
Bemerkung zur Veröffentlichung 23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2018)
Vorab online veröffentlicht am 17.09.2019
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page