Hybrid Approaches towards Adaptive Heterogeneous Computing Platforms for Exchangeable High-Demanding Workloads

The entire history of computer technology has been characterized by one persistent objective: the continuous enhancement of computational performance. This trend is currently driven by Machine Learning (ML) algorithms, whose immense computational demands push modern systems to their operational limits. Traditional methods for performance scaling, such as increasing clock frequency and shrinking transistors, are increasingly reaching physical limitations, making alternative approaches necessary. Alongside microarchitectural improvements and spatial parallelism, there is a growing trend towards highly specialized computing systems. These systems feature application-specific extensions and optimizations, such as instruction-set extensions or discrete hardware accelerators. This specialization, however, comes at a cost: the system may not be suitable for algorithms that were unknown at design time. In the field of neural networks, for instance, novel models and methods are developed continuously, and hardware development constantly lags behind the state of the art due to the long hardware design cycles. Consequently, the very specialization that is essential to achieve the required performance can cause a computing platform to become obsolete quickly. ... mehrThe challenge for computer architects is to find a balance between specialization and flexibility to maximize the operational lifetime of computing platforms.

FPGAs represent one approach to achieve greater flexibility compared to inherently immutable ICs. They allow the reconfiguration of their implemented logic functions, making them adaptable to new use cases. However, they are inferior to ASICs in terms of maximum clock frequency and area efficiency. This raises a critical question: How can the advantages of ASICs–high performance and energy efficiency–be combined with the adaptability of FPGAs? To answer that question, this thesis investigates hybrid architectures that combine fixed logic with FPGAs. In contrast to existing FPGA-SoCs, which feature large, monolithic processor and FPGA blocks, this work proposes a fine-grained interleave of ASIC and FPGA logic. Components such as processors and memories are implemented as fixed logic for maximum performance. Conversely, FPGA fabric is used for components that are less performance-critical but significantly influence the system’s behavior. This approach provides the performance of an ASIC in the datapath while retaining the flexibility of an FPGA for defining the algorithm.

Architectures based on this hybrid approach are divided into two categories: primarily control-flow-dominant systems and dataflow-driven systems. A methodology based on the Roofline Model is used to estimate whether an existing algorithm predominantly focuses on control-flow or dataflow. For control-flow-dominant applications, run-time reconfigurable processors are investigated. These combine a processor core implemented in hard logic with FPGA units for special instructions (SIs). The application benefits from both high processor pipeline throughput and hardware acceleration of compute-intensive operations. Using the i-Core as an example, the application of such a processor for Near-Memory Computing (NMC) is examined. Here, the reconfigurable units benefit from a discrete, high-bandwidth memory interface, which allows for faster execution of memory-intensive operations. Furthermore, approximate computing is analyzed as an optimization method for reconfigurable processors. Certain algorithms, particularly in media processing and ML, can tolerate a certain degree of inaccuracies in their calculations while still delivering an acceptable result. It is shown that this method is particularly effective here, enabling a 75% reduction in hardware resources with a reduction of accuracy of only 3.24%.

For data-flow-driven applications, an accelerator architecture based on a Network-on-Chip (NoC) is presented. A heterogeneous, multi-level memory hierarchy provides the bandwidth required for modern algorithms like ML. The memory system, the NoC, and the host computer interface are realized as fixed logic. The accelerator tiles, however, are implemented using embedded FPGAs, allowing them to be reconfigured at run time. This partitioning into ASIC and FPGA domains achieves high memory bandwidth while allowing the algorithm in the accelerator tiles to be flexibly adapted. As an example application, accelerators for Convolutional Neural Networks (CNNs) are presented and implemented in the tiles. Individual tiles achieve a throughput of 6.28 GOp/s, corresponding to an efficiency of 89.7%. The high throughput of the memory hierarchy enables nearly linear performance scaling across multiple tiles. For instance, with 20 tiles in a 5×5 NoC, the throughput can be increased by a factor of 10.3×.