|
High dynamic range (HDR) photography is an emerging technology that has the potential to dramatically enhance the
visual quality and realism of digital photos. One of the key technical challenges of HDR photography is displaying HDR
photos on conventional devices through tone mapping or dynamic range compression. Although many different tone
mapping techniques have been developed in recent years, evaluating tone mapping operators prove to be extremely
difficult. Web2.0, social media and crowd-sourcing are emerging Internet technologies which can be harnessed to
harvest the brain power of the mass to solve difficult problems in science, engineering and businesses. Paired
comparison is used in the scientific study of preferences and attitudes and has been shown to be capable of obtaining an
interval-scale ordering of items along a psychometric dimension such as preference or importance. In this paper, we
exploit these technologies for evaluating HDR tone mapping algorithms. We have developed a Web2.0 style system that
enables Internet users from anywhere to evaluate tone mapped HDR photos at any time. We adopt a simple paired
comparison protocol, Internet users are presented a pair of tone mapped images and are simply asked to select the one
that they think is better or click a "no difference" button. These user inputs are collected in the web server and analyzed
by a rank aggregation algorithm which ranks the tone mapped photos according to the votes they received. We present
experimental results which demonstrate that the emerging Internet technologies can be exploited as a new paradigm for
evaluating HDR tone mapping algorithms. The advantages of this approach include the potential of collecting large user
inputs under a variety of viewing environments rather than limited user participation under controlled laboratory
environments thus enabling more robust and reliable quality assessment. We also present data analysis to correlate user
generated qualitative indices with quantitative image statistics which may provide useful guidance for developing better
tone mapping operators.
|
