Inpainting sparse scenes through physics aware transformers for single-photon LiDAR

Luke McEvoy; Daniel Tafone; Yong Meng Sua; Yuping Huang

doi:10.1117/12.3014641

18 June 2024 Inpainting sparse scenes through physics aware transformers for single-photon LiDAR

Luke McEvoy, Daniel Tafone, Yong Meng Sua, Yuping Huang

Proceedings Volume 12996, Unconventional Optical Imaging IV; 129960K (2024) https://doi.org/10.1117/12.3014641
Event: SPIE Photonics Europe, 2024, Strasbourg, France

Abstract

We increase single-photon LiDAR capabilities via a hardware-accelerating inpainting transformer model. This model reconstructs all non-observed information within the image plane as it communicates with the beam steering hardware. We apply this to 3D time-of-flight (ToF) reconstruction, where objects obstruct each other’s line of sight. We use ToF histograms to distinguish objects within either the foreground and background, and their overlap will be treated as the dynamic mask for the model to reconstruct. We also employ this to unorthodox scanning patterns such as Lissajous and spiral, which are riddled with sparsity. Lastly, we are developing an AI MEMs system, which intelligently downsamples the image plane based off foreground masks, combating sampling redundancy. We believe that our approach will be useful in applications for imaging and sensing dynamic targets with sparse single-photon data across all domains.

Conference Presentation

Citation Download Citation

Luke McEvoy, Daniel Tafone, Yong Meng Sua, and Yuping Huang "Inpainting sparse scenes through physics aware transformers for single-photon LiDAR", Proc. SPIE 12996, Unconventional Optical Imaging IV, 129960K (18 June 2024); https://doi.org/10.1117/12.3014641

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available