KIT | KIT-Bibliothek | Impressum | Datenschutz

HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics

Götz, Markus; Coquelin, Daniel ORCID iD icon; Debus, Charlotte; Krajsek, Kai; Comito, Claudia; Knechtges, Philipp; Hagemeier, Björn; Tarnawa, Michael; Hanselmann, Simon; Siggel, Martin; Basermann, Achim; Streit, Achim ORCID iD icon


In order to cope with the exponential growth in available data, the efficiency of data analysis and machine learning libraries have recently received increased attention. Although corresponding array-based numerical kernels have been significantly improved, most are limited by the resources available on a single computational node. Consequently, kernels must exploit distributed resources, e.g., distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload via MPI on arbitrarily large high-performance computing systems. It provides both low-level array-based computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take advantage of their available resources, significantly lowering the barrier to distributed data analysis. Compared with applications written in similar frameworks, HeAT achieves speedups of up to two orders of magnitude.

Volltext §
DOI: 10.5445/IR/1000123473
Veröffentlicht am 10.09.2020
Cover der Publikation
Zugehörige Institution(en) am KIT Scientific Computing Center (SCC)
Universität Karlsruhe (TH) – Zentrale Einrichtungen (Zentrale Einrichtungen)
Publikationstyp Forschungsbericht/Preprint
Publikationsdatum 27.07.2020
Sprache Englisch
Identifikator KITopen-ID: 1000123473
HGF-Programm 46.12.02 (POF III, LK 01) Data Activities
Projektinformation HAF (HGF, HGF IVF2016 STRATZUK, ZT-I-0003)
Schlagwörter HeAT, Tensor Framework, High-performance Computing, PyTorch, NumPy, Message Passing Interface, GPU, Data Analysis, Machine Learning, Dask, Neural Networks
Nachgewiesen in arXiv
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page