Data-Driven Methods for Compression and Editing of Spatially Varying Appearance

Abstract

The optical reflectance behavior of materials and surfaces is of great importance for the perception of a scene. The specific gloss and the texture of a surface give valuable hints on properties of the material it is made of; like if it is smooth or porous, soft or firm or even, in the case of a textile, if it is made from cotton- or synthetic fibers. During the early years of Computer Graphics these aspects have been slightly overlooked and materials were modeled based on quite simple assumptions about reflectance and texture appearance. Maybe this can be appointed to the fact that at that time the community had not realized the importance of surface appearance in its full extent. Another reason certainly was that materials are just particularly difficult to model due the involved micro- and mesoscopic structures which have non-trivial influence on the reflectance behavior and appearance. The standard models that have been used for surface appearance during the first two decades of computer graphics were clearly not sufficient to model this complexity in an effective and intuitive way.<br /> At this point the so-called data-driven methods step in: instead of using a parametric model the idea is to base the scene representation on measurements of the object one wants to visualize. Common examples for this data-driven way of image generation are laser-scanned geometry data or animation data, acquired by motion-capture systems. In terms of spatially varying material appearance data-driven methods are already common practice in the form of textures which are usually special photographs of a material.<br /> In this thesis we will deal with an extended definition of classical image textures: the Bi-directional Texture Function (BTF). In contrast to ordinary textures, which usually consist only of a single image, a sampled BTF comprises thousands of images which have been captured under various light situations and from different view points. This way the BTF represents the appearance of a material under varying view- and light direction which explains its defining term bi-directional. Due to the large number of images required to capture this behavior with sufficient accuracy the BTF is rarely used in today's practical applications. This dissertation aims at improving the practical applicability of BTFs by introducing novel methods for effective BTF data reduction and editing. Consequently, the thesis consists of two parts:<br /> The first part introduces and compares techniques for reducing the effective size of the data that needs to be stored on disk down to less than one percent of the original size while preserving the visual quality. Special emphasis is laid on efficient reconstruction from the compressed representation which is indispensable for interactive display in rendering applications. The main idea of the presented techniques is to transfer algorithms known from the field of matrix factorization to BTF data. Another substantial contribution of this work is an efficient algorithm for the computation of local coordinate systems from the raw measured reflectance data alone. In contrast to previous methods it does not rely on assumptions about a specific reflection model.<br /> In the second part we will present algorithms which enable editing of the spatial structure and reflectance of the BTF without the need to touch the thousands of images individually. Our technique is characterized by the consistent and interactive transfer of simple editing operations performed on a single image to the whole data. For this purpose, we generalize techniques known from texture synthesis and transfer to BTFs. Furthermore, we introduce meaningful interpolation of different BTFs based on our generalized appearance transfer technique. It allows to intuitively edit their reflectance in a completely data-driven way.