Large-scale visual search and similarity for e-commerce

Gaurav Anand; Siyun Wang; Karl Ni

doi:10.1117/12.2594924

1 August 2021 Large-scale visual search and similarity for e-commerce

Gaurav Anand, Siyun Wang, Karl Ni

Proceedings Volume 11843, Applications of Machine Learning 2021; 118430S (2021) https://doi.org/10.1117/12.2594924
Event: SPIE Optical Engineering + Applications, 2021, San Diego, California, United States

Abstract

Visual search and similarity can aid an e-commerce platform by providing appropriate recommendations where semantic labeling and associated metadata does not always exist. In this work, we detail the specifics of our system that powers visually similar recommendations. While a common approach leverages learned representation from common classification tasks using DNN’s, the crux of the problem are the labels and ontology that is applied in order for the DNN to functionally perform. Our proposed approach in production for a variety of products is to supply these recommendations based on a defined taxonomy through a hierarchy that has been carefully curated while additionally scaled up through our platform's natural crowd-sourcing interface. To scale the use of these taxonomies in production, we quantization schemes for retrieving approximate nearest neighbors after applying base transformations on the images using Apache Beam and Tensorflow Transform. The nearest neighbor retrievals are based on using a ResNet model architecture, trained from scratch on 3000+ classes. These are trained daily in a distributed fashion and optimizing data throughput. Finally, in order to verify the appropriateness, we use an extensive human evaluation pipeline and quality control. In this work, we share our product design learnings from the various attempts/experiments we conducted for a successful launch.

Conference Presentation

Citation Download Citation

Gaurav Anand, Siyun Wang, and Karl Ni "Large-scale visual search and similarity for e-commerce", Proc. SPIE 11843, Applications of Machine Learning 2021, 118430S (1 August 2021); https://doi.org/10.1117/12.2594924

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available