1.

INTRODUCTION

Fast growing space economy pushes for an increased availability of high-resolution Earth Observation (EO) data that can be subsequently used both for scientific and for commercial exploitation. On one hand, there is a growing interest towards data with higher information content thanks to the deployment of hyperspectral sensors (e.g. the Italian Hyperspectral Precursor of the Applicative Mission (PRISMA), the German Environmental Mapping and Analysis Program (EnMAP) satellite mission or the future Copernicus Hyperspectral Imaging Mission for the Environment (CHIME)). On the other hand, there is a need to reduce payload budgets and make them suitable for small platforms in order to reduce costs and also pave the way to the deployment of satellite constellations, thus also improving revisit time to a large extent.

In the whole, this yields to an increasing need to handle large amount of data on board, yet with limited (or very limited, in the case of small platforms) resources, with the consequence that this can often prevent the full exploitation of the acquired data. Additional challenges for the development of EO imagers with high spatial resolution are posed by the limited availability of large focal plane arrays in some spectral regions, like for example the medium infrared spectral range.

Compressive Sensing (CS) has been proposed as disruptive approach that can mitigate some issues related both to handle large amount of data and to use large detector bidimensional arrays for generating high resolution images. The basic idea consists in using a Spatial Light Modulator (SLM) to apply a series of suitable binary coding masks to the image collected by the foreoptics and acquiring a limited number of integrated measurements – each corresponding to a given coding mask applied to the SLM - by using a single pixel detector [1]-[2]. The image is subsequently reconstructed at the ground segment from the series of integrated measurements by means of suitable reconstruction algorithms [3]-[4]. In other words, CS – leveraging on the concepts of sparsity, which can be referred to the characteristics of ‘compressibility’ of many natural signals, and of incoherence, which implies the use of proper coding masks - merges the sampling and compression phases into a single step, providing the possibility to efficiently reconstruct the image from a smaller number of samples than that dictated by the Shannon-Nyquist theorem [5]. Following the development of the first CS based instrument, there were several other studies that addressed the CS architecture for the implementation of instruments in different several application domains [6]-[7], including space applications [8]-[10]. The latter included not only Earth Observation, but also Planetary Exploration and Space Science [11]. At the same time, SLM technology was further developed – yet with the market still dominated by the commercial DLP® models manufactured by Texas Instruments – and also tested for space environmental conditions [13]-[14]. Another interesting feature of CS data is the possibility to perform information extraction before reconstructing the images: this feature can be exploited for a screening and selection of the the acquired data and the generation of early-warning alarms with very low latency since in this way we can bypass image transmission and full processing. Native encryption is also an additional feature of compressive sensed data that can be addressed at low computational cost. All these aspects, although fascinating for their application to spaceborne instrumentation in which available onboard resources are very limited, require to be investigated in detail and tested for a sound evaluation of actual advantages and drawbacks.

In this paper, we present a demonstrator of a super-resolved compressive imager operating in whiskbroom mode in the Visible-Near Infrared (VIS-NIR) and Medium Infrared (MIR) spectral ranges. The instrument – which is under construction in the frame of the EU H2020 funded SURPRISE project – uses a Digital MicroMirror Device (DMD) as a core element of its architecture and it is inspired by a single-pixel camera in order to overcome the use of large focal plane arrays. The demonstrator has 10 channels in the VNIR and two channels in the MIR and it can reach a superresolution factor from 4 x 4 to 32 x 32. The super-resolution factor represents the ratio between the number of pixels of the image reconstructed at the end of the process and the number of pixels of the detector. Besides, on the grounds of the results obtained by image reconstruction tests on simulated datasets by using Deep Learning based algorithms, data are expected to be natively compressed with a Compression Ratio up to 50%. The study is expected to provide valuable insight for the future development of a novel class of EO instruments with improved performances in terms of ground sampling distance, native compression and on-board processing capabilities.

2.

THE SURPRISE DEMONSTRATOR

The demonstrator under development in the frame of the SURPRISE project is a CS instrument using SLM technology with the aim to shown an enhancement of its performances in terms of number of pixels in the reconstructed image (with respect to the number granted natively by the detector), and also with respect to instrument future capabilities in terms of on-board data processing and native encryption for a novel generation of EO super-resolved CS-based payloads in the VIS-NIR and MIR spectral regions. Although the technical specifications of the demonstrator are simpler than the actual technical features of a possible instrumental counterpart for EO applications, the working principle of the demonstrator and its implementation were conceived to be inspirational for an EO payload counterpart – definitely more complex and costly - working in whiskbroom mode from geostationary platform. In the following sections we describe the working principle of the demonstrator, its main functional blocks and the algorithms used for the reconstruction of the CS images.

2.2.

Overall architecture

The overall architecture of the SURPRISE demonstrator is shown in the block diagram of Figure 1, in which mechanical, optical and electronic parts are highlighted in different colors. The three main sub-systems can be identified:

• – Target Scanning System (TSS): this system allows for the control of the targets and mimics, by means of several movement stages applied to the target holder to move the scene, the whiskbroom operation mode of the demonstrator.

• – Optical section: this is the main unit of the demonstrator, and it includes the foreoptics, the DMD, all the optical components and detectors.

• – Master Unit: this is the unit that controls all the sensors and actuators of the demonstrator, including the TSS, and provides the synchronization of all the parts.

Figure 1.

Overall architecture of the SURPRISE demonstrator: optical, mechanical and electronics parts of the demonstrator.

Different types of scenes can be arranged at the TTS, and these are captured by the demonstrator. Movement stages - controlled by the TTS - provide the scan of the scene. The optical section of the demonstrator is arranged on an optical bench: the fore-optics provides the image of the observed portion (target) of the scene on the DMD image plane field stop. Following the coding step applied at the DMD plane by means of suitable binary coding masks, the signal is split by means of dichroic mirrors. There are three different optical sub-sections: the VIS-NIR sub-section with ten channels, the MIR sub-section with two channels and the High- Resolution (Hi-Res) VIS sub-section that is used for validation and calibration procedures. The Master Unit provides overall control and synchronization as well as data handling.

2.3.

Optical section

The optical section (Figure 2) is the core of the demonstrator: this is where the image of the target is encoded by the modulation masks and the signal is spatially integrated and finally measured.

Figure 2.

Optical section of the SURPRISE demonstrator: (a) overview of the demonstrator’s optical section arranged on the optical bench with a detail of the collection optics and the collimators; (b) optical layout of the demonstrator, with the optical path to the three main sections: (1) Hi-Res camera, (2) VIS-NIR channels, and (3) MIR channels.

A view of the optical section during preliminary tests in the laboratory is shown in Figure 2a: the components are arranged on an optical bench. In the inset, there is a detail of the collection optics, with its primary and secondary mirrors. Figure 2b shows a block diagram with detail of the optical layout of the demonstrator. The collection optics provides the image of the target - placed at the TTS - on the image plane field stop where the DMD is placed and the image is coded. Spectral splitting is applied after the coding stage by means of dichroic mirrors. The next step is the spatial integration by means of optical condensers. The signal is further spectrally filtered (or dispersed by the spectrometer in the VIS-NIR section) and finally measured by the corresponding detectors. An additional sub-section is represented by the Hi-Res panchromatic camera that is used for alignment and for validation procedures.

2.4.

Target scanning system

The TSS is a motorized mechanical structure for the control and fine movement of several modules representing the scenes to be observed by SURPRISE demonstrator. This structure holds a set of 30x30 mm models that are used as targets. The TTS and its main parts are shown in Figure 3, together with their main functions. The TTS main parts are:

• – Control Unit (CU) - this subsystem is an assembled rack 19” which houses several subunits: the Power Distribution Unit (PDU), the Power Supply Unit (PSU), provided with Power Supply for linear and rotary actuators (motion controls) and Power Supply for local Front-End electronics (Auxiliary controls); Processing Unit (PU); PC service interface (SIF): this module provides interface for the connection to external monitor, keyboard, mouse, and interface for connection to the Master Unit; Front End Unit (FEU), that is the interface for the connection to Mechanical Bench and provides on one hand the front-end electronics for the RTDs, infrared sources, halogen illuminators, interlock switch, on the other hand power and control lines for linear and rotary stages; Power Input Unit (PIU).

• – Test Bench (TB) - this subsystem is composed by: a mechanical structure allowing for positioning and replacing of standard-shape targets (240 x 240 mm) of different types; two Precision Motorized Linear Stages allowing for the correct setting X-Y translation of a single target; precision Motorized Rotary stage allowing for the correct setting of the rotation of a single target; protection cover with interlock switch.

• – SW Package - this package includes: an application software installed on the CU workstation used to receive commands and to control the TSS from the Master Unit; Man-Machine Interface (MMI) which reports the target’s positions (x, y and angle), the internal illuminator status, the temperature sensor status, power supply state, warnings and alarms of the motors; a simulation SW that can be also used to test and interact with the TSS without the needs of the Master Unit.

Figure 3.

Target Scanning System and its main parts: Control Unit, Test Bench and SW package.

3.

RECONSTRUCTION ALGORITHMS

The recovery of the image by means of CS reconstruction algorithms can be performed given some conditions [15]: nonetheless, the problem is NP-hard and an exhaustive search of the sparsest solution cannot, in fact, be done. Traditional algorithms used to solve the problem rely on different approaches, and include: greedy algorithms, iterative thresholding algorithms, convex relaxation algorithms, non-convex relaxation algorithms [16]-[19].

Recently, Deep Learning (DL) algorithms have shown very good generalization ability, considerably improving the performance of previous cutting-edge technologies in many sectors, including image processing. As a consequence, CS reconstruction has been investigated also by using DL methods: the basic idea is to make Deep Learning (DL) learn a suitable reconstruction algorithm - optimizing the signal representation - instead of solving a complex numerical problem as that posed by the CS reconstruction. In general, traditional reconstruction algorithm are slower than DL-based ones. Even Total Variation (TV) [20] and its optimized versions - which can be considered among the fastest traditional algorithms - performs worse than most Deep Neural Network (DNN)-based algorithms, both in reconstruction precision (PSNR) and computational time.

Among different DL methods, we identified (Iterative Shrinkage/Thresholding Algorithm) ISTA-Net [21] - and its improved version ISTA-Net+ - as particularly suitable for images representing natural scenes. Its framework consists of mapping each classic ISTA algorithm update step into a deep network architecture in which there is a fixed number of phases that correspond to iteration in the traditional algorithm.

Both TV and DL methods were applied to a dataset of simulated images – that already included the main optical parameters of the SURPRISE demonstrator like the Point Spread Function (PSF) - of natural targets and the results compared. In Figure 4, we show the results we obtained in the reconstruction of one of the images of the dataset by applying both traditional and DL methods. Figure 4a shows the original simulated image of a natural target (Travertino stone image), while Figure 4b and Figure 4c show, respectively, its reconstruction by using TV and its reconstruction by using ISTA-Net+. Both reconstructions refer to the case of a 32x32 super-resolution factor and 75% compression ratio. It is apparent that the reconstruction obtained by using the Total Variation method exhibits less detail, and occasionally have significant ‘blockiness’ artifacts, whereas the DL method is successful at reconstructing images of suitable quality (Figure 4c). In general, by evaluating the reconstruction results in terms of Peak Signal-to-Noise Ratio (PSNR), ISTA-Net+ significantly outperformed the TV algorithm (PSNR (dB) with 32x32 super-resolution factor and 75% compression ratio is 33.63 for DL and 29.45 for TV on the ‘Travertino’ sample image). The gain is significant at 25% and 50% compression ratios and becomes even larger at high quality levels. If compression is not applied, the DP method provides an almost perfect image; on the contrary, the TV method shows a quality plateau.

Figure 4.

Reconstruction of the image by using simulated dataset: (a) Original image (Travertino sample image), (b) image reconstructed with Total Variation method, and (c) image reconstructed with Deep Learning method. The applied CR was 75%. The SR was 32 x 32.

4.

CONCLUSIONS

In this paper, we presented a demonstrator of a super-resolved CS instrument working in the VIS-NIR and in the MIR, which is under construction at CNR-IFAC labs in the frame of the EU H2020 SURPRISE project. All the sub-systems of the demonstrator were already assembled and tested whereas their integration is ongoing. The main features of the demonstrator include the availability of 10 channels in the VIS-NIR and 2 channels in the MIR. The demonstrator has a super-resolution factor that can be set up to 32 x 32 to achieve final images reconstructed at the end of the process with an enhanced number of pixels. Its CS-based architecture allows also for native compression of the data at the acquisition stage and intrinsic encryption. Deep learning- based algorithm tests on simulated dataset of natural targets demonstrated an effective reconstruction with Compression Ratio between 50% and 70%. EO optical payloads based on the same approach could benefit not only in terms of native compression and encryption capabilities, but also in terms of increased Ground Sampling Distance with respect to that granted natively by the number of pixels of the detector used in the payload architecture.