KIT | KIT-Bibliothek | Impressum | Datenschutz

HeAT – a Distributed and GPU-accelerated TensorFramework for Data Analytics

Götz, Markus; Debus, Charlotte; Coquelin, Daniel ORCID iD icon; Krajsek, Kai; Comito, Claudia; Knechtges, Philipp; Hagemeier, Björn; Tarnawa, Michael; Hanselmann, Simon; Siggel, Martin; Basermann, Achim; Streit, Achim ORCID iD icon


To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude.

Zugehörige Institution(en) am KIT Steinbuch Centre for Computing (SCC)
Universität Karlsruhe (TH) – Zentrale Einrichtungen (Zentrale Einrichtungen)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2020
Sprache Englisch
Identifikator ISBN: 978-1-72816-251-5
KITopen-ID: 1000127706
HGF-Programm 46.12.02 (POF III, LK 01) Data Activities
Erschienen in 2020 IEEE International Conference on Big Data (Big Data)
Veranstaltung IEEE International Conference on Big Data (IEEE BigData 2020), Online, 10.12.2020 – 13.12.2020
Verlag Institute of Electrical and Electronics Engineers (IEEE)
Seiten 276-287
Projektinformation HAF (HGF, HGF IVF2016 STRATZUK, ZT-I-0003)
Schlagwörter HeAT, Tensor Framework, High-performance Computing, PyTorch, NumPy, Message Passing Interface, GPU, Big Data Analytics, Machine Learning, Dask, Model Parallelism, Parallel Application Frameworks
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page