RGB pixel-block point-cloud fusion for object detection

Timothy Foster; Ajaya Dalal; John E. Ball

doi:10.1117/12.2587510

12 April 2021 RGB pixel-block point-cloud fusion for object detection

Timothy Foster, Ajaya Dalal, John E. Ball

Proceedings Volume 11748, Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021; 1174804 (2021) https://doi.org/10.1117/12.2587510
Event: SPIE Defense + Commercial Sensing, 2021, Online Only

Abstract

This paper explores a multimodal deep learning network based on SqueezeSeg. We extend the standard SqueezeSeg architecture to enable camera and lidar fusion. The sensor processing method is termed pixel-block point-cloud fusion. Using co-registered camera and lidar sensors, the input section of the proposed network creates a feature vector by extracting information from a block of RGB pixels from each point in the point-cloud that is also in the camera’s field of view. Essentially, each lidar point is paired with neighboring RGB data so the feature extractor has more meaningful information from the image. This fusion method adds rich information on object color and texture from the camera data to enhance the overall performance. The image pixel blocks will not only add color information to the lidar data, but it will also add information about texture. The proposed pixel-block point-cloud fusion method yields better results than single-pixel fusion.

Conference Presentation

Citation Download Citation

Timothy Foster, Ajaya Dalal, and John E. Ball "RGB pixel-block point-cloud fusion for object detection", Proc. SPIE 11748, Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021, 1174804 (12 April 2021); https://doi.org/10.1117/12.2587510

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available