Skip to Main content Skip to Navigation
Conference papers

On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression

Etienne Dupuis 1 David Novo 2 Ian O'Connor 1 Alberto Bosio 1
2 ADAC - ADAptive Computing
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
Abstract : Deep neural networks demonstrate impressive levels of performance, particularly in computer vision and speech recognition. However, the computational workload and associated storage inhibit their potential in resource-limited embedded systems. The approximate computing paradigm has been widely explored in the literature. It improves performance and energyefficiency by relaxing the need for fully accurate operations. There are a large number of implementation options with very different approximation strategies (such as pruning, quantization, low-rank factorization, knowledge distillation, etc.). To the best of our knowledge, no automated approach exists to explore, select and generate the best approximate versions of a given convolutional neural network (CNN) according to the design objectives. The goal of this work in progress is to demonstrate that the design space exploration phase can enable significant network compression without noticeable accuracy loss. We demonstrate this via an example based on weight sharing and show that our method can obtain a 4x compression rate in an int-16 version of LeNet-5 (5-layer 1,720-kbit CNNs) without retraining and without any accuracy loss.
Document type :
Conference papers
Complete list of metadata

https://hal-lirmm.ccsd.cnrs.fr/lirmm-03054114
Contributor : David Novo <>
Submitted on : Friday, December 11, 2020 - 12:04:17 PM
Last modification on : Tuesday, December 15, 2020 - 3:32:38 AM
Long-term archiving on: : Friday, March 12, 2021 - 7:25:31 PM

File

DATE_2020_submission_paper (1)...
Files produced by the author(s)

Identifiers

Collections

Citation

Etienne Dupuis, David Novo, Ian O'Connor, Alberto Bosio. On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression. Design, Automation & Test in Europe Conference & Exhibition (DATE), Mar 2020, Grenoble, France. ⟨10.23919/DATE48585.2020.9116350⟩. ⟨lirmm-03054114⟩

Share

Metrics

Record views

53

Files downloads

72