KIT | KIT-Bibliothek | Impressum | Datenschutz

HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics

Götz, Markus; Coquelin, Daniel; Debus, Charlotte; Krajsek, Kai; Comito, Claudia; Knechtges, Philipp; Hagemeier, Björn; Tarnawa, Michael; Hanselmann, Simon; Siggel, Martin; Basermann, Achim; Streit, Achim

Abstract:
In order to cope with the exponential growth in available data, the efficiency of data analysis and machine learning libraries have recently received increased attention. Although corresponding array-based numerical kernels have been significantly improved, most are limited by the resources available on a single computational node. Consequently, kernels must exploit distributed resources, e.g., distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload via MPI on arbitrarily large high-performance computing systems. It provides both low-level array-based computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take advantage of their available resources, significantly lowering the barrier to distributed data analysis. Compared with applications written in similar frameworks, HeAT achieves speedups of up to two orders of magnitude.

Open Access Logo


Volltext §
DOI: 10.5445/IR/1000123473
Veröffentlicht am 10.09.2020
Cover der Publikation
Zugehörige Institution(en) am KIT Steinbuch Centre for Computing (SCC)
Publikationstyp Forschungsbericht/Preprint
Publikationsdatum 27.07.2020
Sprache Englisch
Identifikator KITopen-ID: 1000123473
Projektinformation HAF (HGF, HGF IVF2016 STRATZUK, ZT-I-0003)
Schlagwörter HeAT, Tensor Framework, High-performance Computing, PyTorch, NumPy, Message Passing Interface, GPU, Data Analysis, Machine Learning, Dask, Neural Networks
Nachgewiesen in arXiv
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page