AI-based telepresence for broadcast applications

Henrique Piñeiro Monteagudo; Rayan Daod Nathoo; Laurent Deillon; Changsheng Gao; Touradj Ebrahimi

doi:10.1117/12.2638229

3 October 2022 AI-based telepresence for broadcast applications

Henrique Piñeiro Monteagudo, Rayan Daod Nathoo, Laurent Deillon, Changsheng Gao, Touradj Ebrahimi

Proceedings Volume 12226, Applications of Digital Image Processing XLV; 1222610 (2022) https://doi.org/10.1117/12.2638229
Event: SPIE Optical Engineering + Applications, 2022, San Diego, California, United States

Abstract

In this paper, we introduce a new solution and underlying architecture that allows remote participants to interact with hosts in a broadcast scenario. To achieve this, background extraction is first applied to video received from remote participants to extract their faces and bodies. Considering that the video from remote participants are usually of lower resolutions when compared to content produced by professional cameras in production, we propose to scale the extracted video with a super-resolution module. Finally, the processed video from remote participants are merged with studio video and streamed to audiences. Given the real-time and high-quality requirements, both background extraction and super-resolution modules are learning-based solutions and run on GPUs. The proposed solution has been deployed in the Advance Mixed Reality (AdMiRe) project. The objective and subjective assessment results show that the proposed solution works well in real world applications.

Conference Presentation

Citation Download Citation

Henrique Piñeiro Monteagudo, Rayan Daod Nathoo, Laurent Deillon, Changsheng Gao, and Touradj Ebrahimi "AI-based telepresence for broadcast applications", Proc. SPIE 12226, Applications of Digital Image Processing XLV, 1222610 (3 October 2022); https://doi.org/10.1117/12.2638229

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available