|
|
1.INTRODUCTIONThe international scientific community has long highlighted the importance of observations in the mid and thermal infrared spectral region (MIR-TIR) for various application sectors: from agriculture and food safety to meteorological and climatic studies, monitoring of coastal areas and natural and anthropogenic hazards. Some of these aspects have been widely discussed at European and international level during workshops focused on user needs, from which the relevance of multi-spectral observations (at least three bands) with high spatial resolution in this spectral region emerged[1]. In particular, many applications concerning surface temperature, events due to high temperatures or the characterization of gases still cannot be described satisfactorily, also due to the fact that a better understanding of them requires a very high spatial resolution in the SWIR-MIR-TIR region. For example, the ability to quantify the CO2 emitted during a volcanic eruption or from the combustion of biomass from agricultural fires and/or landfills, as well as the mitigation of volcanic risks by identifying eruptions in progress, require sensors operating in the MIR-TIR region with a high spatial resolution (from 50 to 100 meters) and a suitable number of spectral bands[2]-[4]. This study proposes an innovative concept of optical payload operating in the medium infrared, based on two emerging technological approaches - super-resolution and Compressive Sensing (CS) - to obtain particularly interesting performances in terms of spatial sampling on the ground. (Ground Sampling Distance - GSD) and mitigation of some effects, such as saturation and blooming, which often constitute a limit to obtaining high quality products in various application sectors, such as detection and monitoring of fires, lava and in general hotspots whose observed surfaces have very high temperatures (> 1200 K). Both approaches are based on the use of a spatial light modulator (SLM), a top optoelectronic device typically consisting of an array of electronically actuated micro-mirrors (MMA - MicroMirror Array). The proposed concept offers various advantages: first, it permits to increase the ground spatial resolution of the payload. At the same time, thanks to the CS technology, the proposed approach will result in a mitigation of the blooming and saturation effects of the single pixel, frequent in traditional sensors when a high temperature hot spot (for example a fire) is observed. In addition, compressed-format capture typical of CS eliminates the need for a separate compression card, saving mass, memory and energy consumption. It is also worth mentioning that the CS approach can natively provide a data encryption feature, which protects acquired information at no additional cost. In this way, the instrument has advanced features and functionality in the section on board the satellite, but its computational complexity is moved to the ground segment, where the calculation constraints are much more relaxed. The possibility to construct instruments with less stringent requirements in terms of power, mass and memory, can pave to a new family of EO payloads working in the MIR spectral region. On the other hand, the possibility of acquiring a data that is already compressed and with a high number of pixels (in principle greater than that reached by COTS detectors) permits to increase the ground spatial resolution and to observe a greater number of details with a considerable expected impact on the related application products. 2.BACKGROUND2.1Super-resolution and compressive sensingThe concept of super-resolution is based on the possibility of obtaining a final image with a higher resolution than the nominal resolution of the chosen detector, thanks to the use of an SLM with a number of elements NxN times greater than the elements of the detector. A single pixel on the detector is associated with each NxN group of SLM elements. The image focused on the SLM is modulated through appropriate binary coding masks. Each pixel of the detector thus integrates the light coming from groups of NxN elements (therefore with N times higher resolution). The samples thus acquired allow to obtain, once transmitted to the ground and with appropriate reconstruction algorithms, an image at the same resolution as that on the focal plane at the SLM. If NxN modulations are carried out during the acquisition, there is no data compression, but if the number of modulations is lower, a decrease in the number of samples to be sent to the ground is inherently obtained (Compressive Sensing). The theory of Compressive Sensing (CS) exploits the a priori knowledge on the sparsity characteristics of the data in order to digitally acquire it using much fewer samples of the ones envisaged Nyquist-Shannon[5]. In general, it is always possible to find a domain where the data is sparse, the one of acquisition or eventually a transformed domain (DCt, wavelet). Following CS theory, several algorithms have been developed to obtain good quality images starting from acquisitions performed in compressive mode. Moreover, instruments have been implemented as well, by exploiting CS theory: an example of these is the single Pixel camera implemented by the Rice Institute[6]. Following the example of such prototype, the interest in such kind of technology grew exponentially, leading to the development of several prototypes that demonstrated both the CS and the possibility of obtaining super-resolution in various application sectors[6]. Nevertheless, up to now very few examples of prototypes addressed the aerospace sector[7]. On the other hand, in recent years ESA has funded some studies to investigate the potential of CS for the construction of optical payloads based on this approach[7][7]-[10]. Regarding super-resolution, some prototypes have been developed to demonstrate its feasibility[11]-[13]. In Mahalanobis, R., Shilling, R., Murphy, R. and Muise, R., “Recent results of medium wave infrared compressive sensing,” Appl. Opt. 53, 8060-8070 (2014).[7], the set-up is sensitive in the MIR frequencies, while in Dumas, J. P., Lodhi, M. A., Bajwa, W. U. and Pierce, M. C., “From modeling to hardware: an experimental evaluation of image plane and Fourier plane coded compressive optical imaging,” Opt. Express 25, 29472-29491 (2017).[13] the prototype operates in the visible and in Chen, H., Asif, M. S., Sankaranarayanan, A. C. and Veeraraghavan, A., “FPA-CS: Focal plane array-based compressive imaging in short-wave infrared,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2358-2366 (2015).[11] and Mahalanobis, R., Shilling, R., Muise, R. and Neifeld, M. A., “High-resolution imaging using a translating coded aperture,” Opt. Eng. 56(8), 084106 (2017).[12] in the SWIR. 2.2Scientific and technological challengesThe study of a multispectral imaging spectrometer operating in the MIR region (3-5 µm) with a high GSD represents an opportunity of great interest for the scientific community, both from an application and a technological point of view. Indeed, this study will allow to evaluate the potential of spectroscopic methodologies for a better characterization and understanding of high temperature phenomena, helping to answer the crucial questions related to the changes occurring on Earth. In particular, the SISSI study may bring significant contributions to the following Scientific Challenges[14]: Challenge A2 - Interaction between the atmosphere and Earth’s surface involving natural and anthropogenic feedback processes for weather, energy and atmospheric composition. Plumes are produced by the continuous degassing of magma and represent an indicator of volcanic activity. As the magma rises up the pipe, the pressure decreases and the gases are released into the atmosphere, dragging fragments of magma. Plumes are a turbulent mixture of gases, solid particles and drops, continuously emitted at high temperatures from summit craters, fumarolic fields or during eruptive episodes. The spectral range in the MIR allows the recognition of spectral absorptions related to volcanic activity; in particular, the SO2 band at 4.0 µm in the MIR could be studied by SISSI also during the non-eruptive degassing phases thanks to the high GSD. For similar reasons (high GSD of the bands in the range 3-5 µm) SISSI would allow the study of the absorption characteristics of CO2, CO and CH4 and therefore to observe small emissions during different volcanic phases, with particular reference to the bands (4.8 µm, 4.6 µm and 3.3 µm). Challenge L1 - Natural processes and human activities and their interactions on the land surface. Thanks to the high GSD, SISSI can lead to the recognition of active fires and FRP (Fire Radiative Power) with a spatial precision level of the order of ten meters, maintaining the radiometric dynamics high compared to most space sensors available today. Regarding the effects of fire on atmospheric chemistry and climate, real-time FRP data provided by e.g. MODIS, or geostationary systems are already used in many scientific studies on fire emissions, and in pre-operational systems such as GMES Atmospheric Core Service prototype However, due to the large number of small fires and low FRP values typical of the most common fires, the aerosol and tracking gas emissions resulting from these calculations will almost certainly be significantly underestimated. Challenge G1 - Physical processes associated with volcanoes, earthquakes, tsunamis and landslides. In order to better assess the natural hazards, Volcanic thermal modelling and precursor analysis. The gas emissions from the fumarole fields or degassing plumes emitted from the summit craters could be studied with high resolution multispectral images acquired in the MIR. At the moment only ASTER with 0.3 K NEDT on 5 channels TIR only allows temperature/emissivity separation and, especially with time series data, allows the detection of low temperature geothermal anomalies of ~ 6-10 K (SNR ~ 20-30) on areas of about 100 m in diameter. The loss of heat during the placement of a lava flow is its most salient feature and it modulates the whole process of setting up. The spectral radiation emitted by hot volcanic sources such as molten lava or high-temperature gas emissions reaches its maximum in the MIR region (approximately 3 µm). MIR is adequate where the High Temperature Events (HTE) radiation is much higher than both the reflected solar radiation and the background. In order to separate small HTEs from the thermal background and estimate HTE parameters, a mission such as SISSI would produce MIR data capable of filling this observation gap and improving understanding of the thermal properties of melted materials. As far as the technological aspects are concerned, both the European Community and ESA and other European national agencies have funded various studies for thermal imaging missions with resolution <100 m over the past decade. These include the FUEGO program, aimed primarily at monitoring fires, with the aim of converging into a low-cost operational concept suitable for a microsatellite mission or as a secondary payload of a larger system. MISTIGRI (MicroSatellite for Thermal InfraRed Ground Surface Imaging) and THIRSTY are studies carried out nationally by CNES for applications in the agricultural sector. To date, however, the operational capabilities of European Earth Observation do not include a high-resolution space mission in the MIR-TIR. Ultimately, the present study is in synergy with many of the studies planned both at national, European and international level - yet maintaining a completely complementary character – and it focuses attention on a particularly critical aspect of the missions in the thermal infrared, the ground spatial resolution. 3.THE SISSI INSTRUMENTIn the following sections, we describe the instrument’s basic concept and architecture, its acquisition mode, the preliminary design of the optical section and the reconstruction algorithms. 3.1The conceptThe operating principle of the SISSI instrument is inspired by that of the single pixel camera[6]. A CS instrument’s architecture is implemented - in its simplest form – by using a light modulation element (Spatial Light Modulator - SLM) and a single pixel detector on which the light is concentrated through a lens (condenser). Each acquisition (measurement) is given by the product - element by element - between each element of the encoding binary pattern applied to the modulator and the corresponding element of the image focused on it, and the subsequent integration carried out by the condenser on the single-pixel detector. Based on the sparseness of the data to be acquired, it is possible to reconstruct the initial image with sufficient quality by making a number of measurements equal to or less than half of the pixels of the image to be reconstructed, thus merging the acquisition and compression phases into a single step. Similarly, the key elements (Figure 1) of the optical module of the SISSI instrument are:
With respect to the single pixel camera architecture, in this case there is a parallel acquisition of the images of MxM groups of adjacent micro-mirrors of the SLM. Each group (macro-pixel) consists of NxN micro-mirrors suitably encoded by a binary mask, different for each frame. Each of the MxM elements of the detector acquires the signal coming from one of the MxM groups of NxN micro-mirrors. Each detector pixel as a whole subtends an area of the scene on the ground corresponding to a macropixel on the SLM (Figure 2). Each SLM macropixel is made up of NxN micropixels, each of which corresponds to a micromirror of the SLM. The NxN factor represents the super-resolution factor of the instrument. Each micropixel subtends to the ground a portion of the scene corresponding to the super-resolute GSD. The acquisition of the same scene in several spectral bands (data cube) is obtained thanks to the presence of spectral filters on the detector, arranged along the across-track dimension of the scene on several contiguous lines of the detector, and to the apparent movement of the scene in the direction along-track through the field of view (push-broom acquisition without slit). In this way, thanks to the movement of the platform, the spatially resolved scene (with a GSD equal to that determined by the micro-pixel) crosses each spectral band on the detector, allowing - line by line - the reconstruction of the spatially resolved image in each spectral band. In the SISSI instrument, thanks to the super-resolution concept, we aim at achieving a super-resolution factor equal to 4x4. A single pixel of the detector will acquire the portion of the scene corresponding to a macropixel on the SLM, that is, an area of the DMD consisting of 4x4 micro-mirrors (each micromirror corresponds to a micro-pixel). The super-resolution of the scene (resolution at the micropixel level) is achieved by acquiring a series of measurements with different spatial encodings of each macropixel, spatially integrated by each element of the detector. The series of measurements acquired by applying a different encoding mask for each measurement at the macropixel level is finally used to reconstruct - using an ad-hoc algorithm - an image of the scene with a GSD equal to that of the projection on the ground of the dimensions of the single micropixel with a super-resolution factor equal to 4x4. 3.2Acquisition modeThe SISSI instrument is a spectrometer that operates in push-broom acquisition mode without slit (Figure 3), i.e. the light that reaches the 2-D detector comes from the entire scene framed by the instrument’s field of view. Thanks to a series of filters, deposited in correspondence with a contiguous set of detector lines, at each acquisition each portion of the scene will be acquired exclusively in the corresponding spectral band. The SISSI instrument then acquires spatially resolved images both across and along-track, with groups of lines detected in different spectral bands, depending on their position on the sensor (Figure 3b). Each spectral band is made up of r lines. The scene seen by the sensor, during the forward movement of the platform, slides on the filters, allowing the acquisition of the same in all the expected bands. The 3D data (spectrally and spatially resolved) is no longer comparable to a cube as in the case of a traditional push-broom instrument (Figure 3a). Instead, to reconstruct an entire scene in each spectral band (as in the push-broom traditional imagers), the data must be reorganized by extracting from each acquisition the data referring to a number r of rows for each spectral band and that are relative to the same scene (Figure 3b). 3.3Observational requirementsIn this first stage of the project, we outlined the SISSI payload’s observation requirements starting from the analysis of the end user requirements and a first iteration on the observation constraints posed by the instrument’s acquisition method. The observation requirements for the SISSI payload are summarized in Table 1. Table 1.Main requirements of the SISSI payload.
3.4Optical section of the payloadBased on the observational requirements outlined from the analysis of the end user requirements and a market survey aimed at identifying the COTS elements available, the most suitable optical elements were identified (Table 2). Table 2.Main technical specifications of the SISSI payload.
Starting from the observational requirements and the devised main technical specifications, we did a preliminary optical design of the instrument’s optical section (Figure 4). The study outlined as, in both collection and focusing optics, the use of reflective optical elements was a priority. This was due to the need to minimize the effect of chromatic aberration and, at the same time, to maximize the transmittance of the optical system. Actually, one of the main critical aspects of achromatic optical systems in the MIR spectral region is the absorption due to glass. Consequently, decreasing the thickness of any dielectrics maximizes the transmittance of the optical system. The design of the optical section in its preliminary version as a whole has provided good results in terms of enclosed energy per detector pixel (equal to about 85%) and a total transmittance of about 30% (DMD efficiency excluded). In the next steps, the optical design will be further refined in order to increase the overall transmittance of the system. 3.5Electronic section of the payloadThe study aims also at outlining the architecture of the electronic section of the payload. This will be made of the following main logic modules:
These, however, should be regarded as logic modules rather than physical ones: for example, it has been verified that an ideal physical subdivision would lead to physically separated sections between the power section, the digital one and the analogue one. Under this respect, the HCC and HDP modules will be provided on a single board. HPU module must supply the main power lines starting from the primary 28Vdc provided by the spacecraft. We are only talking about main lines because the very low voltage power supplies necessary for the various subassemblies, both to limit voltage drops and to decouple the various components as much as possible, should be generated close to the devices themselves that will use them. For this reason, each module must in any case have its power management section. HPE module has the function of managing the interfaces towards the DMD and the detector with the relative cooler. The transmission of the binary configuration patterns to the DMD will be carried out by the FPGA through high frequency (400 MHz) LVDS buses. The interface with the detector, on the other hand, will be based on a Camera Link bus. HDP module has the task of acquiring data from the sensor and sending configuration information to the DMD. Presently, after a market survey, it is under consideration to implement the module starting from a Xilinx radiation tolerant FPGA. HCC module is in charge of the management of the communication towards the spacecraft and the general control of the instrument. Presently, we are considering to use a DSP from Texas Instruments to play this role. The choice of the component was based above all on the objective of allocating a substantial availability of processing capability to support that available at the FPGA level. The activity carried out up to now made it possible to define the logical architecture of the section and to identify the main problems and construction risks. The physical architecture will be defined in the next project’s phase, in accordance with the results of the detailed design of the other sections of the payload. 3.6Reconstruction algorithmsThere are different types of reconstruction algorithms that can be used for CS[15]-[17]. We can mainly distinguish them into two categories: traditional and based on deep learning. For this project, we have chosen as the representative of the former the algorithm that minimizes the Total Variation (TV) pseudo-norm, while for the latter the convolutional neural network called ISTA-Net (ISTA-Net + in its improved version); the latter implements at each phase the Iterative Shrinkage Thresholding Algorithm, which is a traditional method. A traditional solution to solve the image reconstruction problem consists in minimizing the Total Variation (TV) pseudo-norm[18]. This approach is particularly useful when dealing with natural images that have compressible gradients, or images such that most of the energy is enclosed in very few gradients. In the TV formulation, the reconstruction problem can be formulated as: where X represents the two-dimensional image while x its vectorized version and the isotropic TV is defined as: Reconstruction by TV is fast and widely used and thus it can be regarded as a typical traditional method. A different approach is offered by the methods based on deep learning, such as the algorithm known as ISTA-Net. This method, and its improved version ISTA-Net +, are particularly suitable for working with natural images. The authors’ idea is to map the updating steps of the traditional ISTA algorithm on a deep neural network architecture in which there is a fixed number of phases that correspond to the iteration in the classical algorithm. In the ISTA-Net algorithm, a general nonlinear transformation is used to sparsify the images. The optimization in ISTA-Net + is due to the fact that natural images are more compressible than other and this feature is here fully exploited. The results obtained up to now showed that ISTA-Net significantly exceeds the performances achieved by the other algorithms, both traditional and based on deep learning, for each CS ratio considered, where CS ratio means the ratio between the number of measurements acquired and the number of total pixels n. Also, with regard to computational time, the performances of ISTA-Net and ISTA-Net + are definitely better[19]. Since the ISTA-Net neural network provides an excellent compromise between reconstruction accuracy and flexibility, the framework, in its improved version ISTA-Net +, was chosen as the starting point for the development of deep learning-based reconstruction algorithms for this study. 4.CONCLUSIONSIn this paper, we have addressed the on-going study of a super-resolved compressive imager for Earth Observation in the MIR spectral region, which can pay the way to a novel class of EO instruments featuring enhanced capabilities in terms of spatial resolution, on-board data processing and encryption. The use of an SLM as a core element of the instrument’s architecture allows to acquire a data that is already compressed and with a larger number of pixels with respect to that enabled by the 2D detector used. Besides an increased ground spatial resolution with respect to the number of pixels of the detector, the expected advantages of the proposed concept are the mitigation of the blooming and saturation effects of the single pixel when high temperature hotspots are observed, which can have a beneficial impact on the application products. Finally, the compressed-format capture typical of compressive sensing eliminates the need for a separate compression card, yielding saving – although to a limited extent - in terms of payload’s budgets. 5.ACKNOWLEDGEMENTSThis study was funded by the Italian Space Agency under the contract ASI N. 2020-3-U.0 within the programme on novel Earth Observation payloads. REFERENCES(ASI workshop for MWIR-LWIR mission study,
(2019) https://www.asi.it/event/mwir-lwir-mission-study/ Google Scholar
Romaniello, V., Spinetti, C., Silvestri, M. and Buongiorno, M. F.,
“A Sensitivity Study of the 4.8 μm Carbon Dioxide Absorption Band in the MWIR Spectral Range,”
Remote Sensing, 12
(1), 172
(2020). https://doi.org/10.3390/rs12010172 Google Scholar
Wooster M.J., Roberts, G. and Perry, G.L.W.,
“Retrieval of biomass combustion rate and totals from fire radiative power observations: FRP derivation and calibration relationships between biomass consumption and fire radiative energy release,”
Journal of Geophysical Research, 110 D24311
(2005). https://doi.org/10.1029/2005JD006318 Google Scholar
Lombardo V., Musacchio, M. and Buongiorno, M. F.,
“Error Analysis of Subpixel Lava Temperature Measurements Using Infrared Remotely Sensed Data,”
Geophysical Journal International, 191
(1), 112
–125
(2012). https://doi.org/10.1111/j.1365-246X.2012.05632.x Google Scholar
Candes, E.J., Wakin, M.,
“An introduction to compressive sampling,”
IEEE Signal Processing Magazine, 21
–30
(2008). https://doi.org/10.1109/MSP.2007.914731 Google Scholar
Digital Signal Processing (DSP), at Rice University,Google Scholar
Mahalanobis, R., Shilling, R., Murphy, R. and Muise, R.,
“Recent results of medium wave infrared compressive sensing,”
Appl. Opt., 53 8060
–8070
(2014). https://doi.org/10.1364/AO.53.008060 Google Scholar
Barducci, A., Guzzi, D., Lastri, C., Nardino, V., Pippi, I. and Raimondi, V.,
“Compressive sensing for hyperspectral earth observation from space,”
in Proc. SPIE 10563, International Conference on Space Optics — ICSO 2014,
1056353
(2014). Google Scholar
Raimondi, V., Magli, E., Coltuc, D., Labate, D., Barducci, A., Baronti, S., Coluccia, G., Garoi, F., Guzzi, D., Iordache, I., Lastri, C., Nardino, V., Palombi, L., Pippi, I., Ravazzi, C., Zuccaro-Marchi, A. and Miranda, M.,
“Optical compressive sensing technologies for space applications: a pros and cons analysis of application-driven instrumental concepts,”
in 5th International Workshop on On-Board Payload Data Compression OBPDC,
(2016). Google Scholar
Guzzi, D., Coluccia, G., Labate, D., Lastri, C., Magli, E., Nardino, V., Palombi, L., Pippi, I., Coltuc, D., Barducci, A., Zuccaro Marchi, A., and Raimondi, V.,
“Optical compressive sensing technologies for space applications: instrumental concepts and performance analysis,”
in Proc. SPIE 11180, International Conference on Space Optics — ICSO 2018,
111806B
(2018). Google Scholar
Chen, H., Asif, M. S., Sankaranarayanan, A. C. and Veeraraghavan, A.,
“FPA-CS: Focal plane array-based compressive imaging in short-wave infrared,”
in IEEE Conference on Computer Vision and Pattern Recognition,
2358
–2366
(2015). Google Scholar
Mahalanobis, R., Shilling, R., Muise, R. and Neifeld, M. A.,
“High-resolution imaging using a translating coded aperture,”
Opt. Eng, 56
(8), 084106
(2017). https://doi.org/10.1117/1.OE.56.8.084106 Google Scholar
Dumas, J. P., Lodhi, M. A., Bajwa, W. U. and Pierce, M. C.,
“From modeling to hardware: an experimental evaluation of image plane and Fourier plane coded compressive optical imaging,”
Opt. Express, 25 29472
–29491
(2017). https://doi.org/10.1364/OE.25.029472 Google Scholar
O’Neill, A & Barber, D & Bauer, P & Dahlin, H & Diament, Michel & Hauglustaine, D & Traon, Pierre-Yves & Mattia, Francesco & Mauser, Wolfram & Merchant, C. & Pulliainen, J & Schaepman, Michael & Visser, Pieter & Antoine, David & Bojinski, Stephan & Carli, Bruno & Chapron, B & Crevoisier, C & Ebbing, Jörg & Shepherd, Andrew,
“ESA’s Living Planet Programme: Scientific Achievements and Future Challenges – Scientific Context of the Earth Observation Science Strategy for ESA,”
(2015). Google Scholar
Candes, E. and Tao, T.,
“Decoding by Linear Programming,”
IEEE Transactions on Information Theory, 51
(12), 4203
–4215
(2005). https://doi.org/10.1109/TIT.2005.858979 Google Scholar
Wang, J., Kwon, S., Li, P. and Shim, B.,
“Recovery of sparse signals via generalized orthogonal matching pursuit: a new analysis,”
IEEE Transaction on Signal Processing, 64
(4), 1076
–1089
(2016). https://doi.org/10.1109/TSP.2015.2498132 Google Scholar
Needell, D. and Tropp, J.,
“CoSaMP: iterative signal recovery from incomplete and inaccurate samples,”
Communications of the ACM, 12 93
–100
(2010). https://doi.org/10.1145/1859204.1859229 Google Scholar
Wang, Y., Yang, J., Yin, W. and Zhang, Y.,
“A new alternating minimization algorithm for total variation image reconstruction,”
SIAM Journal on Imaging Sciences, 1
(3), 248
–272
(2008). https://doi.org/10.1137/080724265 Google Scholar
Zhang, J. and Ghanem, B.,
“ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing,”
in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition,
1828
–1837
(2018). Google Scholar
|