An Energy-Efficient Stochastic Computational Deep Belief Network
Yidong Liu1,a, Yanzhi Wang2, Fabrizio Lombardi3 and Jie Han1,b
1University of Alberta Edmonton, AB, Canada
ayidong1@ualberta.ca
bjhan8@ualberta.ca
2Syracuse University Syracuse, NY, USA
ywang393@syr.edu
3Northeastern University Boston, MA, USA
lombardi@ece.neu.edu
ABSTRACT
Deep neural networks (DNNs) are effective machine learning models to solve a large class of recognition problems, including the classification of nonlinearly separable patterns. The applications of DNNs are, however, limited by the large size and high energy consumption of the networks. Recently, stochastic computation (SC) has been considered to implement DNNs to reduce the hardware cost. However, it requires a large number of random number generators (RNGs) that lower the energy efficiency of the network. To overcome these limitations, we propose the design of an energy‐efficient deep belief network (DBN) based on stochastic computation. An approximate SC activation unit (A‐SCAU) is designed to implement different types of activation functions in the neurons. The A‐SCAU is immune to signal correlations, so the RNGs can be shared among all neurons in the same layer with no accuracy loss. The area and energy of the proposed design are 5.27% and 3.31% (or 26.55% and 29.89%) of a 32‐bit floating‐point (or an 8‐bit fixed‐point) implementation. It is shown that the proposed SC-DBN design achieves a higher classification accuracy compared to the fixedpoint implementation. The accuracy is only lower by 0.12% than the floating‐point design at a similar computation speed, but with a significantly lower energy consumption.
Keywords: Stochastic computing, deep belief network, rectifier linear unit, cognitive computing.
