Many optical systems are used for specific tasks such as classification. Of these systems, the majority are designed to maximize image quality for human observers. However, machine learning classification algorithms do not require the same data representation used by humans. We investigate the compressive optical systems optimized for a specific machine sensing task. Two compressive optical architectures are examined: an array of prisms and neutral density filters where each prism and neutral density filter pair realizes one datum from an optimized compressive sensing matrix, and another architecture using conventional optics to image the aperture onto the detector, a prism array to divide the aperture, and a pixelated attenuation mask in the intermediate image plane. We discuss the design, simulation, and trade-offs of these systems built for compressed classification of the Modified National Institute of Standards and Technology dataset. Both architectures achieve classification accuracies within 3% of the optimized sensing matrix for compression ranging from 98.85% to 99.87%. The performance of the systems with 98.85% compression were between an F / 2 and F / 4 imaging system in the presence of noise.
We investigate the feasibility of additively manufacturing optical components to accomplish task-specific classification in a computational imaging device. We report on the design, fabrication, and characterization of a non-traditional optical element that physically realizes an extremely compressed, optimized sensing matrix. The compression is achieved by designing an optical element that only samples the regions of object space most relevant to the classification algorithms, as determined by machine learning algorithms. The design process for the proposed optical element converts the optimal sensing matrix to a refractive surface composed of a minimized set of non-repeating, unique prisms. The optical elements are 3D printed using a Nanoscribe, which uses two-photon polymerization for high-precision printing. We describe the design of several computational imaging prototype elements. We characterize these components, including surface topography, surface roughness, and angle of prism facets of the as-fabricated elements.
In this effort, random noise data augmentation is compared to phenomenologically-inspired data augmentation for a target detection task, evaluated on the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model “MegaScene” simulated hyperspectral dataset. Random data augmentation is commonly used in the machine learning literature to improve model generalization. While random perturbations of an input may work well in certain fields such as image classification, they can be unhelpful in other applications such as hyperspectral target detection. For instance, random noise augmentation may not be beneficial when the applied noise distribution does not match underlying physical signal processes or sensor noise. In the context of a low-noise sensor, augmentation mimicking material mixing and other practical spectral modulations is likely to be more effective when used to train a target detector. It is therefore important to utilize a data augmentation strategy that emulates the natural variability in observed spectra. To validate this claim, a small fully connected neural network architecture is trained using an ideal hemispheric reflectance materials dataset as a trivial baseline. That dataset is then augmented using Gaussian random noise and the model is retrained and again applied to MegaScene. Finally, augmentation is instead performed using phenomenological insight and used to retrain and reevaluate the model. In this work, the phenomenological augmentation implements only simple and commonly encountered spectral permutations, namely linear mixing and shadowing. Comparison is made between the augmented models and the baseline model in terms of low constant false alarm rate (CFAR) performance.
Optical remote sensing has become a valuable tool in many application spaces because it can be unobtrusive, search large areas efficiently, and is increasingly accessible through commercially available products and systems. In the application space of chemical, biological, radiological, nuclear, and explosives (CBRNE) sensing, optical remote sensing can be an especially valuable tool because it enables data to be collected from a safe standoff distance. Data products and results from remote sensing collections can be combined with results from other methods to offer an integrated understanding of the nature of activities in an area of interest and may be used to inform in-situ verification techniques. This work will overview several independent research efforts focused on developing and leveraging spectral and polarimetric sensing techniques for CBRNE applications, including system development efforts, field deployment campaigns, and data exploitation and analysis results. While this body of work has primarily focused on the application spaces of chemical and underground nuclear explosion detection and characterization, the developed tools and techniques may have applicability to the broader CBRNE domain.
Fog is a commonly occurring degraded visual environment which disrupts air traffic, ground traffic, and security imaging systems. For many application of interest, spatial resolution is required to identify elements of the scene. However, studying the effects of fog on resolution degradation is difficult because the composition of naturally occurring fogs is variable, and data collection is reliant on changing weather conditions. For our study, we used the Sandia National Laboratories fog facility to generate repeatable characterized fog conditions. Sandia’s well characterized fog generation allowed us to relate the resolution degradation of active and passive long-wave infrared (LWIR) imagers to the properties of fog. Additionally, the fogs we generated were denser than naturally occurring fogs. This allowed for testing of long range imaging in the shorter optical pathlengths obtainable in a laboratory environment.
In this presentation, we experimentally investigate the resolution degradation of LWIR wavelengths in realistic fog droplet sizes. Transmission of LWIR wavelengths has been studied extensively in literature. To date however, there are few experimental results quantifying the resolution degradation for LWIR imagery in fog. We present experimental results on resolution degradation for both passive and active LWIR systems. The degradation of passive imaging was measured using 37˚C blackbody with a slant edge resolution targets. The active imaging resolution degradation was measured using a polarized CO2 laser reflecting off a set of bar targets. We found that the relationship between meteorological optical range and resolution degradation was more complicated than described purely by attenuation.
Many optical systems are used for specific tasks such as classification. Of these systems, the majority are designed to maximize image quality for human observers; however, machine learning classification algorithms do not require the same data representation used by humans. In this work we investigate compressive optical systems optimized for a specific machine sensing task. Two compressive optical architectures are examined: an array of prisms and neutral density filters where each prism and neutral density filter pair realizes one datum from an optimized compressive sensing matrix, and another architecture using conventional optics to image the aperture onto the detector, a prism array to divide the aperture, and a pixelated attenuation mask in the intermediate image plane. We discuss the design, simulation, and tradeoffs of these compressive imaging systems built for compressed classification of the MNSIT data set. To evaluate the tradeoffs of the two architectures, we present radiometric and raytrace models for each system. Additionally, we investigate the impact of system aberrations on classification accuracy of the system. We compare the performance of these systems over a range of compression. Classification performance, radiometric throughput, and optical design manufacturability are discussed.
The scattering of light in fog is a complex problem that affects imaging in many ways. Typically, imaging device performance in fog is attributed solely to reduced visibility measured as light extinction from scattering events. We present a quantitative analysis of resolution degradation in the long-wave infrared regime. Our analysis is based on the calculation of the modulation transfer function from the edge response of a slant edge blackbody target in known fog conditions. We show higher spatial frequencies attenuate more than low spatial frequencies with increasing fog thickness. These results demonstrate that image blurring, in addition to extinction, contributes to degraded performance of imaging devices in fog environments.
Advancements in machine learning (ML) and deep learning (DL) have enabled imaging systems to perform complex classification tasks, opening numerous problem domains to solutions driven by high quality imagers coupled with algorithmic elements. However, current ML and DL methods for target classification typically rely upon algorithms applied to data measured by traditional imagers. This design paradigm fails to enable the ML and DL algorithms to influence the sensing device itself, and treats the optimization of the sensor and algorithm as separate sequential elements. Additionally, this current paradigm narrowly investigates traditional images, and therefore traditional imaging hardware, as the primary means of data collection. We investigate alternative architectures for computational imaging systems optimized for specific classification tasks, such as digit classification. This involves a holistic approach to the design of the system from the imaging hardware to algorithms. Techniques to find optimal compressive representations of training data are discussed, and most-useful object-space information is evaluated. Methods to translate task-specific compressed data representations into non-traditional computational imaging hardware are described, followed by simulations of such imaging devices coupled with algorithmic classification using ML and DL techniques. Our approach allows for inexpensive, efficient sensing systems. Reduced storage and bandwidth are achievable as well since data representations are compressed measurements which is especially important for high data volume systems.
We report on the design, modeling, calibration, and experimental results of a LWIR, spectrally and temporally resolved broad band bi-directional reflectance distribution function measuring device. The system is built using a commercial Fourier transform infrared spectrometer, which presents challenges due to relatively low power output compared to laser based methods. The instrument is designed with a sample area that is oriented normal to gravity, making the device suitable for measuring loose powder materials, liquids, or other samples that can be difficult to measure in a vertical orientation. The team built a radiometric model designed to understand the trade space available for various design choices as well as to predict instrument success at measuring the target materials. The radiometric model was built by using the output of commercial non sequential raytracing tools combined with a scripted simulation of the interferometer. The trade space identified in this analysis will be presented.
The design was based on moving periscopes with custom off axis parabolas to focus the light onto the sample. The system assembly and alignment will be discussed. The calibration method used for the sensor will be detailed, and preliminary measurements from this research sensor will be presented.
The modeling and simulation of non-traditional imaging systems require holistic consideration of the end-to-end system. We demonstrate this approach through a tolerance analysis of a random scattering lensless imaging system.
Computational imagers fundamentally enable new optical hardware through the use of both physical and algorithmic elements. We report on the creation of a static lensless computational imaging system enabled by this paradigm.
We report on the design of a refracting prism array for use in a computational lensless imaging system. The technique discussed enables creation of a refracting element that maximizes signal on a detector region. Examples of pseudo-random prism arrays for the generation of images are provided. The pseudo-random prism array is compared to a randomly oriented prism array and the advantages of the optimal scattering element are highlighted.
Hyperspectral imaging polarimetry enables both the spectrum and its spectrally resolved state of polarization to be measured. This information is important for identifying material properties for various applications in remote sensing and agricultural monitoring. We describe the design and performance of a ruggedized, field deployable hyperspectral imaging polarimeter, designed for wavelengths spanning the visible to near-infrared (450 to 800 nm). An entrance slit was used to sample the scene in a pushbroom scanning mode across a 30 deg vertical by 110 deg horizontal field-of-view. Furthermore, athermalized achromatic retarders were implemented in a channel spectrum generator to measure the linear Stokes parameters. The mechanical and optical layout of the system and its peripherals, in addition to the results of the sensor’s spectral and polarimetric calibration, are provided. Finally, field measurements are also provided and an error analysis is conducted. With its present calibration, the sensor has an absolute polarimetric error of 2.5% RMS and a relative spectral error of 2.3% RMS.
Channeled linear imaging polarimeters measure the two-dimensional distribution of the linear Stokes parameters. A key aspect of this technique is to accurately reconstruct the Stokes parameters from a snapshot, modulated measurement of the channeled linear imaging polarimeter. The state-of-the-art reconstruction takes the Fourier transform of the measurement to separate the Stokes parameters into channels. While straightforward, this approach is sensitive to channel cross-talk and imposes bandwidth limitations that cut off high frequency details. To overcome these drawbacks, we present a reconstruction method called compressed channeled linear imaging polarimetry. In this framework, reconstruction in channeled linear imaging polarimetry is an underdetermined problem, where we measure N pixels and recover 3N Stokes parameters. We formulate an optimization problem by creating a mathematical model of the channeled linear imaging polarimeter with inspiration from compressed sensing. Through simulations, we show that our approach mitigates artifacts seen in Fourier reconstruction, including image blurring and degradation and ringing artifacts caused by windowing and channel cross-talk. By demonstrating more accurate reconstructions, we push performance to the native resolution of the sensor, allowing more information to be recovered from a single measurement of a channeled linear imaging polarimeter.
Lensless imaging systems have the potential to provide new capabilities for lower size and weight configuration than traditional imaging systems. Lensless imagers frequently utilize computational imaging techniques, which moves the complexity of the system away from optical subcomponents and into a calibration process whereby the measurement matrix is estimated.
We report on the design, simulation, and prototyping of a lensless imaging system that utilizes a 3D printed optically transparent random scattering element. Development of end-to-end system simulations, which includes simulations of the calibration process, as well as the data processing algorithm used to generate an image from the raw data are presented. These simulations utilize GPU-based raytracing software, and parallelized minimization algorithms to bring complete system simulation times down to the order of seconds.
Hardware prototype results are presented, and practical lessons such as the effect of sensor noise on reconstructed image quality are discussed. System performance metrics are proposed and evaluated to discuss image quality in a manner that is relatable to traditional image quality metrics. Various hardware instantiations are discussed.
This paper describes measurements being made on a series of material systems for the purpose of developing a radiative-transfer model that describes the reflectance of light by granular solids. It is well recognized that the reflectance spectra of granular materials depend on their intrinsic (n(λ) and k(λ)) and extrinsic (morphological) properties. There is, however, a lack of robust and proven models to relate spectra to these parameters. The described work is being conducted in parallel with a modeling effort1 to address this need. Each follows a common developmental spiral in which material properties are varied and the ability of the model to calculate the effects of the changes are tested. The parameters being varied include particle size/shape, packing density, material birefringence, optical thickness, and spectral contribution of a substrate. It is expected that the outcome of this work will be useful in interpreting reflectance data for hyperspectral imaging (HSI), and for a variety of other areas that rely on it.
Compact snapshot imaging polarimeters have been demonstrated in literature to provide Stokes parameter estimations for spatially varying scenes using polarization gratings. However, the demonstrated system does not employ aggressive modulation frequencies to take full advantage of the bandwidth available to the focal plane array. A snapshot imaging Stokes polarimeter is described and demonstrated through results. The simulation studies the challenges of using a maximum bandwidth configuration for a snapshot polarization grating based polarimeter, such as the fringe contrast attenuation that results from higher modulation frequencies. Similar simulation results are generated and compared for a microgrid polarimeter. Microgrid polarimeters are instruments where pixelated polarizers are superimposed onto a focal plan array, and this is another type of spatially modulated polarimeter, and the most common design uses a 2x2 super pixel of polarizers which maximally uses the available bandwidth of the focal plane array.
Channeled spectropolarimeters (CSP) measure the polarization state of light as a function of wavelength. Conventional Fourier reconstruction suffers from noise, assumes the channels are band-limited, and requires uniformly spaced samples. To address these problems, we propose an iterative reconstruction algorithm. We develop a mathematical model of CSP measurements and minimize a cost function based on this model. We simulate a measured spectrum using example Stokes parameters, from which we compare conventional Fourier reconstruction and iterative reconstruction. Importantly, our iterative approach can reconstruct signals that contain more bandwidth, an advancement over Fourier reconstruction. Our results also show that iterative reconstruction mitigates noise effects, processes non-uniformly spaced samples without interpolation, and more faithfully recovers the ground truth Stokes parameters. This work offers a significant improvement to Fourier reconstruction for channeled spectropolarimetry.
KEYWORDS: Polarization, Polarimetry, Signal to noise ratio, Prototyping, Sensors, Chemical analysis, Humidity, Chemical detection, Coating, Titanium dioxide
We report on the development of a prototype polarization tag based system for detecting chemical vapors. The system primarily consists of two components, a chemically sensitive tag that experiences a change in its optical polarization properties when exposed to a specific chemical of interest, and an optical imaging polarimeter that is used to measure the polarization properties of the tags. Although the system concept could be extended to other chemicals, for the initial system prototype presented here the tags were developed to be sensitive to hydrogen fluoride (HF) vapors. HF is used in many industrial processes but is highly toxic and thus monitoring for its presence and concentration is often of interest for personnel and environmental safety. The tags are periodic multilayer structures that are produced using standard photolithographic processes. The polarimetric imager has been designed to measure the degree of linear polarization reflected from the tags in the short wave infrared. By monitoring the change in the reflected polarization signature from the tags, the polarimeter can be used to determine if the tag was exposed to HF gas. In this paper, a review of the system development effort and preliminary test results are presented and discussed, as well as our plan for future work.
Modulated imaging Stokes polarimeters require processing of acquired data to produce an estimate of the Stokes
parameters from the scene. The total polarimeter operator describes the estimation of the Stokes parameters from
the incident fields from the scene through reconstruction. In this discussion will shall consider the polarimeter
being applied to an application where the spectral density matrix of the scene Stokes parameters and detector
noise are known. The spectral density matrix of the estimated Stokes parameters is found using the known
spectral density matrix of the scene to find the response of the operator to signal fluctuations. This analysis
grants the ability to optimize the operator for a given application. We demonstrate an optimization of system
processing algorithm that takes inspiration from the classical Wiener filter.
KEYWORDS: Polarimetry, Polarization, Spatial frequencies, Cameras, Calibration, Scene based nonuniformity corrections, Nonuniformity corrections, Long wavelength infrared, Algorithm development, Video
Non-uniformity noise is common in infrared imagers, and is usually corrected through calibration, often by
momentarily blocking the optical system with a relatively uniform temperature plate. The non-uniformity
patterns also tend to drift and require periodic recalibration, necessitating occasional loss of video from the imager
during the recalibration process. Microgrid polarimeters are especially sensitive to fixed-pattern noise because
the polarization signal is acquired by differentiation of neighboring pixels. Scene-based algorithms attempt to
alleviate the need for recalibration of the imager through image processing techniques. We introduce a new
frequency-domain scene-based non-uniformity estimation and correction technique, and apply the technique to
infrared and microgrid polarimeter imagery. The technique demonstrates promising results for shutter-assisted
(recalibration) video, for microgrid polarization systems as well as most spatially modulated sensor systems.
Recently, a polarimetric data reduction technique has been developed that in the presence of a time varying
signals and noise free measurement process can achieve an error free reconstruction provided that the signal
was band limited. Error free reconstruction for such a signal is not possible using conventional data reduction
methods. The new approach provides insight for processing arbitrary modulation schemes in space, time, and
wavelength. Theory predicts that a polarimeter that employs a spatio-temporal modulation scheme may be able
to use the high temporal resolution of a spatially modulated device combined with the high spatial resolution
of a temporally modulated system to attain greater combined resolution capabilities than either modulation on
scheme can produce alone. A polarimeter that contains both spatial and temporal modulation can be constructed
(for example) by placing a rotating retarder in front of a micropolarizer array (microgrid). This study develops
theory and analysis for the rotating retarder microgrid polarimeter to show how the available bandwidth for
each channel is affected by additional dimensions of modulation and demonstrates a working polarimeter with
a simulation of Stokes parameters that are band limited in both space and time with a noisy measurement
process.
For the past several years we have been working on strategies to mitigate the effects of IFOV errors on
LWIR microgrid polarimeters. In this paper we present a detailed, theoretical analysis of the source of
IFOV error in the frequency domain, and show a frequency domain strategy to mitigate those effects.
Microgrid polarimeters are a type of division of focal plane (DoFP) imaging polarimeter that contains a mosaic
of pixel-wise micropolarizing elements superimposed upon an FPA sensor. Such a device measures a slightly
different polarized state at each pixel. These measurements are combined to estimate the Stokes vector at each
pixel in the image. DoFP devices have the advantage that they can obtain Stokes vector image estimates for
an entire scene from a single frame capture. However, they suffer from the disadvantage that the neighboring
measurements that are used to estimate the Stokes vector images are acquired at differing instantaneous fields of
view (IFOV). This IFOV issue leads to false polarization signatures that significantly degrade the Stokes vector
images. Interpolation and other image processing strategies can be employed to reduce IFOV artifacts; however
these techniques have a limit to the amount of enhancement they can provide on a single microgrid image.
Here we investigate algorithms that use multiple microgrid images that contain frame-to-frame global motion
to further enhance the Stokes vector image estimates. Motion-based imagery provides additional redundancy
that can be exploited to recover information that is "missing" from a single microgrid frame capture. We have
found that IFOV and aliasing artifacts can be defeated entirely when these types of algorithms are applied to the
data prior to Stokes vector estimation. We demonstrate results on real LWIR microgrid data using a particular
resolution enhancement technique from the literature.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.