KEYWORDS: Satellites, Satellite imaging, Clouds, RGB color model, Earth observing sensors, Data transmission, Power consumption, Landsat, Data modeling, Remote sensing
To address these issues, this paper introduces a real-time on-board satellite cloud cover detection system based on a lightweight neural network. By discarding excessively cloudy images, the proposed approach can lead to an improvement in the efficiency and accuracy of satellite image-based systems. At the same time, it allows to minimize the data to be transmitted to the ground, consequently mitigating bandwidth problems and reducing transmission power. The proposed CNN shows a compact architecture, requiring fewer than 9 thousand parameters, while maintaining a detection accuracy of 89% when evaluated using the Landsat 8 dataset. An optimized hardware accelerator is designed to meet the on-board nanosatellites constraints. Post-implementation simulations on a Xilinx Artix 7 FPGA demonstrate state-of-the-art results with a utilization of about 12 thousand and 7 thousand of mapped LUTs and FFs, respectively, with a power consumption of 116 mW.
KEYWORDS: Real time image processing, Field programmable gate arrays, Digital signal processing, Embedded systems, Power consumption, Image enhancement, Digital filtering, Design, Convolutional neural networks, Prototyping
Multiply-Accumulate (MAC) operation is widely used in various real-time image processing tasks, ranging from Convolutional Neural Networks to digital filtering, significantly impacting overall system performance. In this work the Self-Adapting Reconfigurable Multiply-Accumulate (SR-MAC) is proposed as a new instrument to find the optimal trade-off between operation throughput, power consumption and physical resources utilization in real-time image processing applications. Operations of the proposed system rely on the dynamic reconfiguration of the hardware resources on the basis of the current computational requirements. This is achieved by monitoring overflow and over-representation occurrences at each accumulation cycle, and properly considering the relevant portion of the accumulation result. A custom architecture of the proposed algorithm has been designed and implemented on an AMD Xilinx Artix-7 FPGA through a Verilog description and compared to the AMD Xilinx fixed-point macro (floating-point fused multiply-accumulate). The SR-MAC achieves reductions of 83% (82%), 79% (93%) and 87.2% (94%) in the number of LUTs, FFs, and the power dissipation, PdynN, respectively. The SR-MAC has also been used to replace arithmetic units in typical real-time image processing applications. In these cases, its employment has allowed the reduction up to 6% and 14% of FFs and PdynN, respectively, while increasing up to 14% the fMax. These results highlight the significant performance enhancement achieved with respect to both single operators and entire systems, making SR-MAC an excellent design choice in real-time image processing applications.
Driver posture and micro movements are main indicators of his attention and situation awareness, as well as of his capability to suddenly take control if necessary. Therefore, the real-time detection of wrong postures is essential to mitigate the risk of accidents. In this work we want to show that, by using a custom Convolutional Neural Network (CNN) for image processing, a very accurate driver posture recognition system can be realized by using a limited number of pressure sensors, grouped in a small carpet placed only on the seat of the driver, regardless of its shape. Data from the sensor carpet are converted in images reproducing the different pressure regions of the driver’s body, so that the CNN can extract features and classify 8 postures with an average accuracy of 98.81 % in real-time. According to the edge computing paradigm, the CNN implements an end-to-end classification by exploiting a quantization scheme for weights and binarized activations to reduce the number of required resources and allow a compact and low-power HW implementation on a small FPGA. When implemented with a Xilinx Artix 7 FPGA, the CNN consumes less than 7 mW of dynamic power at an operation frequency of 47.64 MHz. Such frequency is compatible with a sensor Output Data Rate (ODR) of 16.50 kHz, fundamental in critical applications, requiring a continuous monitoring and real-time action. Results of a 130 nm CMOS standard cells synthesis have also been reported.
KEYWORDS: Performance modeling, Image processing, Neural networks, Signal processing, Neurons, Tunable filters, Feature extraction, Digital signal processing, Systems modeling, Sensors
KeyWord Spotting (KWS), i.e. the capability to identify vocal commands as they are pronounced, is becoming one of the most important features of Human-Machine Interface (HMI), also thanks to the pervasive diffusion of high-performance MEMS audio sensors with very reduced dimensions. In-Sensor Computing (ISC) appears the most viable solution to get the maximum advantage of KWS, since the dimensions of MEMS microphones remain reduced and minimally invasive. ISC, indeed, represents the extreme evolution of the edge computing paradigm, where the processing circuits are moved close to the audio sensor, integrated into its auxiliary circuitry or in the same package. However, ISC introduces severe area and power constraints and must trade off with processing speed to meet real-time operations naturally required by KWS. In this work, we want to show a neural network-based KWS suitable for ISC contexts, when audio sensor data are converted into MEL spectrogram images and a Depthwise Separable Convolutional Neural Network (DSCNN) with feature extraction capabilities is designed. To show the advantages of the above approach, the DSCNN is compared with an alternative Fully Connected Neural Network (FCNN), operating on audio signals not converted into images. The considered models have been profiled on a microcontroller and implemented on an FPGA. Their performances are compared in terms of classification accuracy and HW resources. Comparisons show that the FCNN is very far from meeting the ISC real-time processing requirements, showing a number of parameters and a frame latency respectively of 3 and 1 orders of magnitude higher than required by the DSCNN alternative when mapped to a Xilinx Zynq Ultrascale+ MPSoC.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.