Lightweight memristive gated recurrent unit networks

Feng Wang; Houji Zhou; Yi Li; Xiangshui Miao

doi:10.1117/12.2625117

27 March 2022 Lightweight memristive gated recurrent unit networks

Feng Wang, Houji Zhou, Yi Li, Xiangshui Miao

Proceedings Volume 12169, Eighth Symposium on Novel Photoelectronic Detection Technology and Applications; 121698F (2022) https://doi.org/10.1117/12.2625117
Event: Eighth Symposium on Novel Photoelectronic Detection Technology and Applications, 2021, Kunming, China

Abstract

Memristor arrays have been widely used to accelerate neural network algorithms for edge intelligence. Although previous study has revealed the potential of memristive Long Short-Term Memory (LSTM) neural network for sequential information processing, it is difficult to map its large weights to existing memristor arrays due to the limitation of the fabrication. Here we demonstrate several gate-variants that further simplify the original Gated Recurrent Unit (GRU) neural network, a simplified version of LSTM, by removing its specific weight matrix, which is used to lower the requirement of weights mapping in memristor arrays. We then performed software validation and offline inference device validation of these new networks using a variety of different datasets based on Mixed National Institute of Standards and Technology (MNIST) dataset segmentation, testing their effectiveness for classifying sequential datasets of different sizes and their robustness for device imperfection. Ultimately, we illustrate that these gate-variants of GRU networks can maintain the performance of the original network in different segmented MNIST datasets, while reducing the memristor array size for more than 50% on most datasets. This work will further advance the edge application of memristive Recurrent Neural Networks (RNN) with limited resources.

Citation Download Citation

Feng Wang, Houji Zhou, Yi Li, and Xiangshui Miao "Lightweight memristive gated recurrent unit networks", Proc. SPIE 12169, Eighth Symposium on Novel Photoelectronic Detection Technology and Applications, 121698F (27 March 2022); https://doi.org/10.1117/12.2625117

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available