ePrints

Accelerating Deep Learning Inference in Constrained Embedded Devices Using Hardware Loops and a Dot Product Unit

Lookup NU author(s): Dr Farhad Merchant

Downloads

• Published version [.pdf]

Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

Abstract

Deep learning algorithms have seen success in a wide variety of applications, such as machine translation, image and speech recognition, and self-driving cars. However, these algorithms have only recently gained a foothold in the embedded systems domain. Most embedded systems are based on cheap microcontrollers with limited memory capacity, and, thus, are typically seen as not capable of running deep learning algorithms. Nevertheless, we consider that advancements in compression of neural networks and neural network architecture, coupled with an optimized instruction set architecture, could make microcontroller-grade processors suitable for specific low-intensity deep learning applications. We propose a simple instruction set extension with two main components-hardware loops and dot product instructions. To evaluate the effectiveness of the extension, we developed optimized assembly functions for the fully connected and convolutional neural network layers. When using the extensions and the optimized assembly functions, we achieve an average clock cycle count decrease of 73% for a small scale convolutional neural network. On a per layer base, our optimizations decrease the clock cycle count for fully connected layers and convolutional layers by 72% and 78%, respectively. The average energy consumption per inference decreases by 73%. We have shown that adding just hardware loops and dot product instructions has a significant positive effect on processor efficiency in computing neural network functions.

Publication metadata

Author(s): Vreca J, Sturm K, Gungl E, Merchant F, Bientinesi P, Leupers R, Brezocnik Z

Publication type: Article

Publication status: Published

Journal: IEEE Access

Year: 2020

Volume: 8

Pages: 165913-165926

Online publication date: 08/09/2020

Acceptance date: 01/05/2020

Date deposited: 25/08/2023

ISSN (electronic): 2169-3536

Publisher: IEEE

URL: https://doi.org/10.1109/ACCESS.2020.3022824

DOI: 10.1109/ACCESS.2020.3022824

Altmetrics

Funding

Share