1.

INTRODUCTION

Remote sensing observations often require to couple the information acquired by an imaging system with the one acquired by a spectroscopic instrument: “patching” the spectral characteristics of an observed region to the corresponding imaging features provides a much larger and more complete set of information than the single instruments themselves. For this, it is very common to find both these types of instruments on board of a satellite, in particular for planetary/moon exploration (see for example [1] and [2], or [3]). Often, to get still more information, the on-board imaging system has stereoscopic capabilities, that is it provides data with which it is possible to reconstruct a digital terrain model (DTM) of the observed features. Stereo capabilities are now more and more common on remote sensing satellites (e.g. [4],[5]), even if the data reduction is much more complicated than simple imaging: in fact, to realize a DTM it is necessary to acquire at least a couple of images of the same region from different perspectives, and all the geometrical acquisition parameters (i.e. satellite attitude and position with respect a common reference system) have to be known with great accuracy.

The datasets acquired by merging the products of these two types of instruments have 4-dimensions, three spatial and one spectral, and they provide an essentially complete information about the observed regions, at least for the large majority of scientific cases of interest. Unfortunately, the possibility of obtaining homogeneous datasets is often limited by the different characteristics of the involved instruments, in terms of sensitivity, of spatial resolution on ground, of field of view (FoV), of coordinated observations, of overlapping spectral ranges, and so on. To solve this issue, a very accurate instrument cross-calibration would be needed, but we all know that it is practically very difficult to obtain it, mainly if the instruments are realized by different teams with different on ground support equipment and different calibration procedures. To our knowledge, these problems exist with all remote sensing satellites, also those which have full attitude control and are presently used for Earth Observation.

To overcome this problem, we thought about the possibility of developing an instrument which has simultaneously both stereoscopic and spectroscopic capabilities, in practice an instrument able to provide a spectral DTM (SDTM) of the observed region as final product. Such an instrument, clearly more complicated than either a stereoscopic imaging system or a spectroscopic one, would intrinsically solve all the problems related to instrument cross-calibration, being a single one summarizing all the capabilities of the merged two: it would be sufficient to properly calibrate this instrument to be able to fully characterize the observed spectral and spatial surface features in an optimal way.

The possible application we had in mind for such an instrument is planetary/moon exploration. Since we are thinking to a potentially complex interplanetary mission, we preferred to consider a rather consolidated mission scenario, in which the satellite is put in polar orbit around the target. In such a way, the most typical way of observing the surface is by pushbroom/pushframe techniques with a nadir pointing satellite. With this type of orbit, the easiest way to get a stereoscopic system is to have two fixed cameras, one looking forward and one looking backward ([4],[5], but also the High Resolution Stereoscopic (HRS) camera on board of SPOT5 [6]): this allows a simple control of the spacecraft and a rather simple instrument operational mode. Actually, with a fully controllable satellite attitude it is possible to get stereo information also with a single camera: however, we consider this as a rather complex operation to be implemented in the next years on a planetary mission, thus we preferred a more conventional approach. Also, in the hypothesis to be able to maintain rather limited the resources for our instrument, in terms of mass and power, we assume that such a simpler orbit profile allows the possible application of this instrument also to low-cost Earth 4D-observing systems, like nano-sats or Unmanned Aerial Vehicles (UAVs), for civilian applications, as agriculture or geology.

Under these assumptions we started to design HYPSOS (HYPerspectral Stereo Observing System), that is a novel remote sensing instrument able to extract the spectral information from the two channels of a pushbroom stereo camera; this instrument will be able to return data allowing to reconstruct a 4D information, three spatial and one spectral, of the observed features. HYPSOS design has been conceived as a small-class instrument, for being compatible with small satellites, as for planetary exploration or terrestrial environmental monitoring from nano-sats. An instrument with such global capabilities, both in terms of scientific return and needed resources, is optimal for fully characterizing the observed surface of investigation.

In this paper we are reporting on the development of such an instrument prototype. After a brief description of the instrument optical design, we report on the realization of the instrument breadboard prototype and on the laboratory equipment we are using for checking the instrument performance; finally, the preliminary results of the instrument alignment are provided. These measurements are finalized to verify the goodness of the optical performance of HYPSOS; the reduction of the obtained data to reconstruct the SDTM of the analyzed surface will be reported in a future paper.

2.

HYPSOS OPTICAL DESIGN

An instrument able to provide a SDTM as final product has to be able to get two images of the same target from two different perspectives, and then to obtain the spectrum of each image pixel. From the analysis of all the spectra with a suitable software, it will be possible to finally obtain the DTM’s for each portion of the spectrum. In terms of optical design, this corresponds to have two coupled optical systems: first, the “telescope”, that is a pushbroom stereocamera, which images the target on the telescope focal plane; then the “imaging spectrometer”, with its entrance slit located on the telescope focal plane, that provides the spectrum of each projected slit element on a bidimensional sensor.

To design HYPSOS, we took advantage of our experience in the realization of the STereo Camera (STC) [5] of SIMBIO-SYS [3][7]. SIMBIO-SYS is the imaging system of the ESA BepiColombo mission [8][9] to Mercury, and it consists of the High Resolution Imaging Channel (HRIC), the Visible and near-Infrared Hyperspectral Imaging channel (VIHI) [10] and the STereo imaging Channel. STC is a very compact camera system, in which light is collected from two fixed channels tilted by ±20° with respect to nadir, sent to a common optical path, and finally collected by a single bidimensional sensor [11]. To minimize mass and envelope, we designed the HYPSOS telescope with a concept similar to the STC one, that is light is collected from two independent channels and then is sent along a common optical path. There are however some differences with respect to STC: first, the telescope configuration, which is no longer based on a modified Schmidt camera but on a three mirror anastigmat (TMA) configuration; then, on the telescope focal plane there is not the focal plane assembly, but the entrance slit of the HYPSOS spectrometer; in addition, to have a single entrance slit spectrograph, we added two 90° field rotators in front of the HYPSOS apertures.

To have a better understanding of the main motivations of all these substantial changes in the HYPSOS design with respect to the STC one, let us start with explaining the need of the two rotators at the channel apertures. Figure 1 shows a schematic view of the ground projected FoV’s for a standard two-cameras pushbroom stereo system with a nadir pointing satellite: in addition to Channel 1 (forward) and Channel 2 (backward) FoV’s, we can see the spacecraft orbit ground projection and motion orientation. If using standard camera systems, in the hypothesis of a common focal plane, these two FoV’s would be imaged as shown in Figure 2 left: this is exactly what happens for example on the High-Resolution Stereo Camera (HRSC) experiment on Mars Express [4], where two CCD linear arrays collect the light projected on the focal plane by the camera objective. Unfortunately, such a configuration would make much more complex the realization of the spectrometer, that in this case would need to have two separate entrances. Thus, by adding two field rotators, in our case two 45° tilted Schmidt-Pechan prisms, one per channel, we could rotate by 90° the FoV’s obtaining on the telescope focal plane a projected image like the one shown in Figure 2 right. It is clear that, thanks to this rotation, we are projecting on a single spectrometer entrance slit, which has also the function of field stop, both the channel FoV’s. In addition, by suitably selecting the two FoV’s, they enter the spectrograph in different portions of the entrance slit: so, thanks to the stigmatic properties of the spectrograph, the spectra of the two FoV’s will be obtained independently and without any crosstalk.

Figure 1.

Ground projection of the FoV of a two-camera pushbroom stereoscopic system (SSP: Sub Satellite Point).

Figure 2

On the left, the FoV’s on the stereo camera focal plane is shown in case there is no rotator; on the right, the same FoV’s are shown on the camera focal plane, including the 90° rotation, together with the HYPSOS spectrometer entrance slit.

The choice of a different optical design for HYPSOS with respect to STC is also due to the possibility of making HYPSOS a rather versatile instrument. In fact, STC is operative in a limited portion of the visible spectrum, while with HYPSOS we had in mind to potentially extend it to larger portion of the spectrum, potentially from ultraviolet to mid infrared, as a function of the scientific targets under investigation. For this, we preferred a telescope with a catoptric configuration, as a TMA, with a forth flat folding mirror. In Figure 3 it is possible to see the schematic of the optical design of HYPSOS prototype, where all the optical elements are shown: for what concern HYPSOS telescope, after the Schmidt-Pechan prisms (SPP) there are the three mirrors M1, M2, M3 and the folding mirror FM. The latter could actually be replaced by a diopter, to have also a transmitted beam in addition to the reflected one: in this way, by adding a second spectrometer, the operational spectral range of HYPSOS could be extended in an additional portion of the spectrum. Because of the limited available resources, we realized an instrument working in visible spectrum only, but with this configuration there is enough room to add a second spectrometer and extend the instrument spectral range. The only potential concern for this are the SPP’s: being them reflecting elements working in a collimated portion of the beam, there is no significant issue in extending the spectral range in the infrared if a suitable low absorption glass is used; however, for UV extension this could be more a problem, and in case a catoptric rotator should be adopted, at the price of a more complex and bulky instrument.

Figure 3.

Optical layout of the HYPSOS prototype. For the acronym description, see the text.

As shown in Figure 3, light entering the two channels is folded by ±20° with respect to nadir by two entrance flat mirrors (EFM). The aperture stops of HYPSOS, two 35 mm diameter circular masks, one per channel, are located at the input SPP face’s. Light is 90° rotated by SPP, and illuminates two sub-apertures of the TMA mirrors; in order to limit possible stray light, suitable masks are set in front of the telescope mirrors. As mentioned before, the FoV is limited by the spectrograph entrance slit (S) on the telescope focal plane. The “slit” is actually a double-long-slit, 22 μm wide and 8 mm long each channel, with a 1 mm obscuration in between (see Figure 2 right); given the telescope focal length of 245 mm, the nominal instantaneous FoV of HYPSOS is 18.5 arcsec × 1.87°; because of the folding of the two EFM, the actual across track FoV is ±(20.125 - 22)° with respect to nadir, respectively, for the two channels. More details about the HYPSOS telescope optical configuration can be found in [12]; here we limit to say that the nominal RMS spot size at the telescope focus is well below 10 μm over all the useful FoV.

Light passing through the slit enters a double-pass imaging spectrometer, composed by four lenses (L1, L2, L3, L4) and a concave reflection grating (G). Finally, the two separate spectra are collected by a bidimensional detector (D). Some of the optical parameters of the HYPSOS spectrometer (slightly different from those reported in [12] because of a design update) are summarized in Table 1. The spectrograph practically has a 1:1 magnification. Some characteristics spot diagrams as a function of the field incident on the TMA and of the wavelength are shown in Figure 4; the represented square boxes have a size of 22 μm, corresponding to the entrance slit width. Apart from some residual aberration at the longest wavelength, more than 80% of the monochromatic energy is always included within the box. The nominal plate scale factor on the spectrometer focal plane is 34.7 nm/mm: assuming a 22 μm sampling, we conservatively obtain a resolving spectral element (double sampling) of 1.53 nm.

Figure 4.

Spot diagrams on the focal plane of the HYPSOS spectrograph. Wavelengths are in μm; the square box size is 22 μm. Three fields are shown, representative of the whole spectral range covered by HYPSOS prototype.

Table 1.

Some parameters of the HYPSOS prototype spectrometer. Acronyms are described in the text.

3.

HYPSOS PROTOTYPE AND LABORATORY SETUP

To check HYPSOS actual performance, we realized an instrument prototype to be operated in a laboratory environment. We realized a rather solid mechanical structure holding all the optical elements, and allowing all the degrees of freedom needed to perform a proper system alignment. All HYPSOS optical elements have been realized by Italian companies. A schematic open view of the HYPSOS mechanical structure is shown in Figure 5a, while the actual instrument is shown in Figure 5b.

Figure 5.

In (a) an open view of the rendered HYPSOS structure. In (b) the view of HYPSOS prototype (the SPP’s are not yet mounted). The approximate envelope is 310×310×320 mm³.

More technical details about the prototype realization can be found in [12], here we just mention the main characteristics of the focal plane sensor. For HYPSOS, we selected an active CMOS 6480×4856 pixels, 3.45×3.45 μm² size; this sensor has a 12 bits ADC, it allows pixel binning, it is possible to make windowing, and readout rates up to 12 full frames/s. The latter is an important characteristic to be able to reproduce the instrument push-broom acquisition mode in the lab environment.

The basic idea for reproducing in the laboratory the HYPSOS operational mode is to look at a known 3D target with one channel at the time under the same nominal geometrical condition as in-orbit [12], in a similar way to what was done for characterizing STC [13][14]. As target, we use stones with known spectral variegations whose 3D surface profile has been fully characterized by a range camera with an accuracy of 20 μm; the stones can be illuminated from different orientations by suitable halogen lamps to reproduce different Sun illumination conditions. The targets are mounted on a linear stage on top of a rotator, for reproducing the relative motion ground-satellite at the right observation angle. Light diffused by the target is collected by a fixed collimating lens (1 m focal length) and directed towards HYPSOS. Light from the collimator is collected by a couple of folding mirrors which substitute, for this calibration setup only, the entrance flat mirrors, and enters HYPSOS apertures. HYPSOS and the folding mirrors, to properly align the channel boresight directions with the entrance beam, can be rotated by a suitable stage (see Figure 6).

Figure 6.

HYPOS acquisition configuration scheme. On top, the configuration for reproducing the acquisition with the forward channel is shown; on bottom, the same for the backward channel.

4.

HYPSOS ALIGNMENT: PRELIMINARY RESULTS AND FUTURE ACTIVITIES

Presently, we are in the process of aligning HYPSOS. For this, we first independently align the TMA and the spectrograph, then we joint the two subsystems.

For the telescope alignment, we did not use any foreoptics in front of the telescope apertures, nor the SPP’s: in such a way, we could illuminate the two telescope apertures with a collimated on-axis beam. Actually, all the telescope mirrors have a front mask limiting their sub-apertures, and we foresaw also a central smaller aperture for alignment purposes on them; so, for the alignment activities, we illuminated the telescope with three parallel on-axis collimated light beams, generated by expanding and splitting a He-Ne laser, filling the two HYPSOS entrance apertures and the central auxiliary one. An important point to take under consideration is that the telescope optical parameters have been optimized to have the best focus over the FoV’s of the two channels, and not when illuminated with an on-axis light beam (that corresponds to the alignment condition). The ray-tracing in Figure 7 shows that the image on the focal plane when illuminating the two channels with on-axis beams consists of two small spots separated by about 60 μm (and not a single focused spot as would be the case with an on-axis optimized telescope). In addition, this is just a geometrical ray-tracing which does not consider diffraction: actually, because of the limited size of the auxiliary optics used for the TMA alignment, the focal plane image is diffraction limited and the actual spot has a theoretical Airy radius of the order of 14 μm (at 633 nm). Figure 8 shows the image quality at the telescope focal plane, acquired with a 5.5 μm pixel-size auxiliary camera. This figure shows on the left the image of a USAF target acquired with one of the two channels, and on the right the corresponding measured contrast transfer function (CTF) from which the modulation transfer function (MTF) has been derived . The obtained MTF is slightly lower than the expected one, but it shows in any case a relatively good telescope alignment.

Figure 7.

Ray tracing simulation of the spots on the telescope focal plane when illuminating HYPSOS with two on-axis collimated light beams, filling the two telescope apertures. The box has a size of 80 μm.

Figure 8.

Estimate of the contrast (both CTF and MTF) on the HYPSOS telescope focal plane (right) by imaging a USAF target (left).

Regarding the spectrometer, which is essentially an on-axis optical system, after having mounted within mechanical tolerances all the lenses in a suitable barrel and verified their correct alignment, the grating was also mounted and its on-axis alignment checked. At this point we closed the barrel front aperture with a flange holding both the slit and the detector. For checking the complete spectrometer alignment and to characterize its performance, we mounted a set of 22 μm square pinholes at the same nominal position of the spectrometer entrance slit. Illuminating the pinholes with a spectral lamp, we could check both the quality of the spots and the spectrometer dispersion. Again it was necessary to consider the peculiar illumination path of HYPSOS, by which just two small lateral portions of the grating are hit by light. Thus, a suitable illumination setup was realized, with which the pinholes have been illuminated using a series of spectral lamps. In Figure 9 we can see an example of a typical spot on the spectrometer focal plane: this image shows the spectral image obtained at the 587.56 nm HeI line acquired with the nominal sensor. The spatial FWHM, from Figure 9b, is about 6 pixels (21 μm), corresponding to the projected pinhole size; a weak ghost is present on the left side of the spectral image and we are presently investigating it. Figure 9c shows the spectral width of the line: a preliminary fit shows an almost Voigt line profile, as expected due to the type of used lamp, and a FWHM spectral width of 1.4 nm. The latter value has been found by measuring the spectrometer dispersion, obtained by acquiring spectra of several spectral lamps. Figure 10 shows the linear fit of the measured wavelengths as a function of the pixel number. The fit slope, given the camera pixel size of 3.45 μm, is ∆λ/∆x = 34.1 nm/mm which is reasonably in line with the expected value of 34.7 nm/mm.

Figure 9.

Example of color image (a) and x (spatial) (b) and y (spectral) (c) plots of an image spot at the HYPSOS spectrometer focal plane. This is the spectral projection of the 587.56 nm HeI line.

Figure 10.

Measured positions of the spectral line centroids on the HYPSOS focal plane, and relative linear fit. This plot combines the measurements done using the following spectral lamps: Ar, Cd, Hg, Hg-Cd-Zn, K, Na, Ne.

Next step will be the co-alignment of the two subsystems, telescope and spectrometer. For this, we foresee to use again the He-Ne expanded laser beams, but in a different way with respect to what has been done for the alignment of the telescope only: in fact, HYPSOS FoV does not include the on-axis direction, which is blocked by the spectrometer slit. Thus, we will use the HYPSOS rotating stage to rotate it with respect to the laser beam fixed on the optical bench by the suitable angle to send the focused beam on the spectrometer slit (with the field rotators SPP’s not yet mounted, such a rotation moves the spot along the slit direction). At this point we will be able to check the telescope-spectrometer co-alignment over the whole HYPSOS FoV. Once this step will be concluded, we will proceed with the integration and alignment of the two SPP’s and of the foreoptics.

With the completely aligned instrument, we will finally start the measurement phase. With the setup shown in Figure 6 to reproduce the in-flight geometry, and a proper system control which suitably synchronizes the speed of the translation stage with the sensor acquisition frame rate, we will acquire a stereo pair of target spectra simulating the spacecraft pushbroom acquisition mode. Then, the collected data will allow to generate the SDTM for each spectral sampling, providing the 4D information of each resolved element of the observed surface: this will be done by using an ad hoc software, derived by 3DPD [15][12], a photogrammetric pipeline we developed for processing the stereo images from the Colour and Stereo Imaging System (CaSSIS) [16] on board of ExoMars Trace Gas Orbiter (TGO). Then the obtained SDTM will be compared with the known DTM of the target and with its known spectral features.

5.

PHOTOGRAMMETRIC CALIBRATION

Another activity on which we are working in parallel to the instrument alignment is the development of a suitable procedure to realize its photogrammetric calibration. In fact, HYPSOS operates as a double pushbroom spectrometer in which the acquisitions are associated to a unique set of internal and external parameters whose knowledge is necessary to generate the final SDTM [18]. Thus, the geometric camera calibration is essential to estimate the value of these parameters.

We implemented a code in Python to calibrate linear pushbroom cameras [19]. This code uses a geometry-camera model [20] able to transform the object space coordinates into image coordinates as acquired by a pushbroom sensor traveling in a straight line with constant speed. The method foresees the use of a suitable known gauge with which acquiring some images. To this end, we realized a bi-planar laminated chessboard that provides a high number of easily detectable corners, homogeneously distributed over the HYPSOS FoV (see Figure 11). Both the planes are printed with a regular square pattern, 0.2×0.2 cm² size: the bottom and upper planes have size of 4×4 cm² and 2×2 cm² respectively, for a total of 400 black and white squares. The basic concept to realize this calibration is to acquire images of the gauge with the lab HYPSOS setup previously described; then, from the analysis of these images, and in particular of the corner positions, and with the knowledge of the gauge geometry, the Python code determines the camera parameters by means of a suitable algorithm.

Figure 11.

Left: Design specifications of the reference gauge. Right: the bi-planar gauge with the regular chessboard pattern.

To verify the goodness of the implemented code, as a first step, we provided the code with “perfect” data, finding essentially nominal camera parameters [19]. Then, a standard chessboard corner detection algorithm was introduced in the software, showing that it was necessary to improve its accuracy to obtain a robust and stable estimation of internal and external camera parameters. Presently, we have acquired the first calibration images with the lab setup, in static condition, obtaining the expected performance. We are now working on the control software to automatically reproduce in the lab the pushbroom acquisition mode using both the rotation stages and the translation one. As soon as the acquisition process will be automated, the final photogrammetric calibration will be done.

6.

CONCLUSIONS

In this paper we described the status of the on-going activities for the realization of a prototype of a novel remote sensing instrument named HYPSOS. HYPSOS is a rather compact instrument which joins the capabilities of a stereo pushbroom camera with those of a spectrometer, to provide a spectral DTM of the observation target. Because of the limited available resources, the HYPSOS prototype has been realized to operate in the visible portion of the spectrum, but its design allows the possibility to extend the spectral range in the infrared or UV. To our knowledge, HYPSOS is the first instrument able to simultaneously provide such a complete information dataset.

Presently, we are in the process of aligning the instrument: we already independently checked the TMA telescope and the reflection grating spectrograph; the next step will be putting the two together to determine actual HYPSOS performance. The implemented laboratory setup will allow to reproduce the operational mode of HYPSOS, that is a pushbroom acquisition mode. For this, a suitable control system has also been realized that will allow to automatically operate the system and store the acquired spectra. In parallel we are proceeding with the implementation of the photogrammetric calibration, to obtain the most accurate internal and external camera parameter. We expect to get the first SDTM for the end of the year.

This project will demonstrate the possibility of providing a complete 4D information, three spatial and one spectral, of an observed target by using a single instrument: this will be really advantageous with respect to the present state of the art in which more instruments need to be used for getting the same information, at the price of a lot of cross-correlation issues. We realized this prototype having in mind space exploration missions, but definitely such an instrument can be adapted to Earth observation. Also, HYPSOS can be made more compact [17], to be mounted on a cubesat, increasing the possible range of applications.