Transparent expansion of a WLCG compute site using HPC resources

High energy physics (HEP) experiments generate vast amounts of data, which need to be processed and analyzed. For this, experiments located at the Large Hadron Collider (LHC) rely on the Worldwide LHC Computing Grid (WLCG). This distributed computing infrastructure is composed of several computing sites around the world. Operated in close contact with the local HEP groups, these sites provide very specialized environments for computing workloads of the collaborations. The expected increase in volume of data collected by the experiments and also a shift towards the use of non-HEP computing sites, such as university and commercial computing sites and high performance computing (HPC) centers, demand for solutions to easily access and use those sites.

In this contribution we present “COBalD– the Opportunistic Balancing Daemon” and the “Transparent Adaptive Resource Dynamic Integration System” (TARDIS), a scalable solution to dynamically and transparently integrate heterogeneous resources in existing infrastructure, as e.g. the WLCG. Two setups are showcased: First a setup with several sites throughout Germany consolidated through dedicated points-of-entry for multiple experiments, and second the transparent expansion of an existing WLCG site with a local HPC site.