Computational Imaging and Analysis of Scene Appearance

Appearance modeling has a rich history in the fields of optics, computer graphics and vision. The emergent cross-disciplinary area of computational photography has led to recent advances in measurement based appearance modeling. This fellowship project aims to further extend the boundaries of appearance modeling by investigating novel techniques in computational imaging for measurement of scene appearance and its analysis for novel graphics, vision and imaging applications.

The goal is to advance the state of the art in appearance measurements in controlled conditions as well as detailed appearance modeling in uncontrolled environments. While traditional reflectometry techniques have mostly accounted for geometric optics, this project proposes novel imaging techniques that extensively consider wave properties of polarization and spectra of incident and reflected light. The project targets novel applications of such imaging in estimation of surface reflectance of complex materials, photometric reconstruction of transparent objects, and measurement of layered scattering in translucent materials. Besides having significant applications in graphics and vision, the proposed approach has broader applications in applied optics and medical imaging.

The project proposes to further generalize such appearance measurements to uncontrolled outdoor settings in this ambitious endeavor. This will include measurement and modeling of sky polarization fields and spectral modeling of natural scene reflectance. Furthermore, the project proposes novel statistical analysis of such imagery including both spatial and angular domain appearance statistics for appearance estimation from sparse uncontrolled measurements. This has varied applications in acquisition and rendering of outdoor scenes for graphics and virtual reality applications, and in large scale scene recognition and analysis for vision, robotics and remote sensing applications.

The proposed fellowship project will extend the boundaries of measurement based appearance modeling in both controlled laboratory conditions as well as in uncontrolled outdoor environments and hence has many applications in graphics, vision and applied optics. A major application area is digitization of shape and reflectance of material samples for realistic modeling of digital props in the entertainment industry for visual effects and video games. The proposed computational imaging techniques for appearance acquisition also have the potential to benefit applications outside the entertainment industry in areas such as imaging and digitization of cultural heritage artifacts in museums and art galleries for posterity, surface and subsurface imaging in diagnostic medical imaging. There are other possible applications in computer aided manufacturing for visualization and analysis/inspection of surface appearance of manufactured materials. The proposed reflectometry approaches for outdoor environments have the potential to impact the realism and quality of virtual reality and scene visualization systems and consumer applications such as Google StreetView/Earth, as well as 3D sensing and large scale scene analysis applications. Beyond graphics and vision, the study of sky polarization fields and spectral modeling of natural scenes have the potential for impact in remote sensing and atmospheric studies.

Industrial impact will be made with active engagement with industrial partners and connections in various domains including visual effects, games, cosmetics, architecture, etc. Engagement will also be done with local museums and art galleries for applications of the research in cultural heritage preservation. Applications of the research in medical imaging areas will also be explored in collaboration with respective departments at Imperial.

Academic impact will be made through research dissemination at leading venues of publications, invited talks to research groups and presentations at focused workshops on the topic of appearance acquisition and modeling.

Broader societal impact will be made in the form of public communication of the research at Imperial festivals and outreach events. A cross-disciplinary international scientific seminar on imaging will also be organized to exchange ideas between researchers from various communities.