eCoSimHPC: Scaling up sector-coupled energy system analysis with RSE support

As the world continues to warm due to global warming, the reliance on fossil fuels for energy production must be significantly reduced to limit overall warming to 1.5C, as required by the Paris Agreement. This requires global emissions of greenhouse gases such as CO to be reduced to half the 2019 level by 2030 and to preindustrial levels by 2050. Despite this, carbon emissions have continued to increase above the 2019 level. Countries are now shifting towards renewable energies to fulfill their contribution to the Paris Agreement, requiring a significant understanding of their populations' energy consumption to ensure a continuous supply during this transition. Models and simulations of coupled energy systems are therefore needed to analyze the impact of the transition to renewable energy sources on the environment and population. The main challenge in simulating sector-coupled energy systems is the diversity of models, not only across buildings and neighborhoods, but also across energy suppliers and grids. This is where the eCoSim framework (Energy System Co-Simulation) can be used. eCoSim is a Python-based co-simulation framework that enables communication between independent simulators. ... mehrEach simulator is treated as a black box that runs in parallel whenever possible at each time step, or in a sequential manner if data dependencies with other simulators are present. These simulators can be written in pure Python, MATLAB, or as Functional Mock-up Units (FMUs) that contain precompiled executable C-code with an embedded associated model. By assembling a system of connected models, a larger co-simulation model can be created. For example, weather, occupancy, and heating models can be connected to simulate the gas consumption in a house. As more and more simulators are added to the co-simulation, the computational demand of eCoSim also increases. In its current state, eCoSim is parallelized via a server-client-based protocol across a given network. The infrastructure allows users to run their own models as clients and access a remote server to synchronize with the larger co-simulation. However, with thousands of models needed to simulate a city, the current framework is not sufficient to handle that much data, and a switch to HPC systems becomes essential. Here we present eCoSimHPC, an extended version of eCoSim with an MPI-based communication framework enabled by Dask, suitable for running on HPC systems. With the aid of the distributive memory capabilities of Dask-MPI, we are able to conduct large-scale co-simulations of sector-coupled energy systems with energy grids, energy producers, and consumers of neighborhoods, thus forming the foundation for running city-scale co-simulations in the future. This talk will detail the steps that were taken to introduce MPI into the existing infrastructure of eCoSim whilst maintaining core functionality. We will present the outcomes of the first test co-simulations with the new infrastructure, including performance tests and scaling tests between eCoSim and eCoSimHPC. Lastly, we will discuss how the collaboration between researchers and research software engineers throughout eCoSim/eCoSimHPC's development has allowed the code to become more accessible for users.