Multiple caterpillar tracking and 3D positioning in orchard via YOLO plus DeepSORT

Sumesh Nair; Chai-Wei Hsu; Yvonne Yuling Hu; Shean-Jen Chen

doi:10.1117/12.2673773

9 August 2023 Multiple caterpillar tracking and 3D positioning in orchard via YOLO plus DeepSORT

Sumesh Nair, Chai-Wei Hsu, Yvonne Yuling Hu, Shean-Jen Chen

Proceedings Volume 12621, Multimodal Sensing and Artificial Intelligence: Technologies and Applications III; 126210J (2023) https://doi.org/10.1117/12.2673773
Event: SPIE Optical Metrology, 2023, Munich, Germany

Abstract

Caterpillars have been causing havoc to agriculture due to feeding aggressively on the foliage of the crops. The current methods of pest control like sticky traps or pheromone traps work on adult moths, but not on foraging caterpillars. Chemical means like pesticides are effective, but chemical residues on crops are concerning. Therefore, this study aims to primarily track and estimate the 3D position of the caterpillars in orchards in real-time. To this end, we have employed the state-of-the-art object detector YOLOv7, combined with Simple Online and Realtime Tracking with a Deep Association Metric (DeepSORT) algorithm. This combined approach, when compared to merely YOLOv7, has improved detection up to 25%, courtesy of the SORT embedded tracker. The RGB-D camera is utilized for this work is Intel Realsense D405. For the training data, 2,000 images captured in a jujube orchard with varying exposure, occlusion, and wind conditions were used. Inference was done from completely new images in real time. In the experiments, the YOLOv7+SORT approach makes detections within 17 ms per frame, with an average detection rate of 85%, indicative of is real-time applicability in orchards. The smallest object (around 2-cm length caterpillar) is recognized around 21×12 pixels, which is at a distance of 35 cm from the camera. Thus, this development of YOLOv7+SORT approach can be integrated with technologies like robot arms, that can pick the caterpillars, or even into stand-off techniques like laser pest targeting, which can help eradicate the pest problems in a physical manner efficiently.

Conference Presentation

Citation Download Citation

Sumesh Nair, Chai-Wei Hsu, Yvonne Yuling Hu, and Shean-Jen Chen "Multiple caterpillar tracking and 3D positioning in orchard via YOLO plus DeepSORT", Proc. SPIE 12621, Multimodal Sensing and Artificial Intelligence: Technologies and Applications III, 126210J (9 August 2023); https://doi.org/10.1117/12.2673773

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available