On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier
Communication Dans Un Congrès Année : 2020

On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression

Résumé

Deep neural networks demonstrate impressive levels of performance, particularly in computer vision and speech recognition. However, the computational workload and associated storage inhibit their potential in resource-limited embedded systems. The approximate computing paradigm has been widely explored in the literature. It improves performance and energyefficiency by relaxing the need for fully accurate operations. There are a large number of implementation options with very different approximation strategies (such as pruning, quantization, low-rank factorization, knowledge distillation, etc.). To the best of our knowledge, no automated approach exists to explore, select and generate the best approximate versions of a given convolutional neural network (CNN) according to the design objectives. The goal of this work in progress is to demonstrate that the design space exploration phase can enable significant network compression without noticeable accuracy loss. We demonstrate this via an example based on weight sharing and show that our method can obtain a 4x compression rate in an int-16 version of LeNet-5 (5-layer 1,720-kbit CNNs) without retraining and without any accuracy loss.
Fichier principal
Vignette du fichier
DATE_2020_submission_paper (1).pdf (301.77 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

lirmm-03054114 , version 1 (11-12-2020)

Identifiants

Citer

Etienne Dupuis, David Novo, Ian O'Connor, Alberto Bosio. On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression. DATE 2020 - 23rd Design, Automation and Test in Europe Conference and Exhibition, Mar 2020, Grenoble, France. pp.1319-1322, ⟨10.23919/DATE48585.2020.9116350⟩. ⟨lirmm-03054114⟩
153 Consultations
275 Téléchargements

Altmetric

Partager

More