On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier Access content directly
Conference Papers Year : 2020

On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression

Abstract

Deep neural networks demonstrate impressive levels of performance, particularly in computer vision and speech recognition. However, the computational workload and associated storage inhibit their potential in resource-limited embedded systems. The approximate computing paradigm has been widely explored in the literature. It improves performance and energyefficiency by relaxing the need for fully accurate operations. There are a large number of implementation options with very different approximation strategies (such as pruning, quantization, low-rank factorization, knowledge distillation, etc.). To the best of our knowledge, no automated approach exists to explore, select and generate the best approximate versions of a given convolutional neural network (CNN) according to the design objectives. The goal of this work in progress is to demonstrate that the design space exploration phase can enable significant network compression without noticeable accuracy loss. We demonstrate this via an example based on weight sharing and show that our method can obtain a 4x compression rate in an int-16 version of LeNet-5 (5-layer 1,720-kbit CNNs) without retraining and without any accuracy loss.
Fichier principal
Vignette du fichier
DATE_2020_submission_paper (1).pdf (301.77 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

lirmm-03054114 , version 1 (11-12-2020)

Identifiers

Cite

Etienne Dupuis, David Novo, Ian O'Connor, Alberto Bosio. On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression. DATE 2020 - 23rd Design, Automation and Test in Europe Conference and Exhibition, Mar 2020, Grenoble, France. pp.1319-1322, ⟨10.23919/DATE48585.2020.9116350⟩. ⟨lirmm-03054114⟩
111 View
230 Download

Altmetric

Share

Gmail Facebook X LinkedIn More