Multiplexing and Multi-Mode Super-Resolution for Digital Material Appearance

Abstract

The visual appearance of materials on the surface of objects contributes greatly to the overall perception of virtually every scene we encounter every day. However, complex material appearance can be neigh impossible to model mathematically. Consequently, it can be difficult to reproduce in computer-generated imagery. As a work-around, image-based digital representations such as the bidirectional texture function (BTF) treat material appearance as a "black box'' which reproduce a given material's appearance under the desired conditions by appropriately interpolating between actual images of the material at hand. Effectively, the problem of physically-based modelling is replaced with the problem of measuring: in order to ensure high-quality, photo-realistic renderings, thousands to tens of thousands images of a given material under varying viewing and lighting conditions are required. The design of appearance acquisition devices inevitably involves choices with negative impact on acquisition speed and accuracy. <br /> The present thesis introduces a number of methods which alleviate this impact and therefore make appearance acquisition much more practical. It is demonstrated that a generalization of phenomenological models usually used for the purpose of BTF compression allows for reconstruction of a full BTF measurement from a sparse measurement, greatly enhancing measurement speed and enabling cheaper and more compact acquisition devices. <br /> As another way of reducing acquisition time, illumination patterns of multiple instead of single light sources are used, which significantly brightens materials with low albedo or lots of self-shadowing, allowing for shorter shutter times. It is shown that the shot noise present in the reconstruction of the desired images under single-light illumination can be effectively reduced using the same or similar phenomenological models. <br /> Both methods seem orthogonal and thus applicable simultaneously. However, due to the dynamic range of BTF data, the employed models involve non-linear transformations which make a combination unfeasible. A novel model is introduced, along with an alternating least-squares algorithm to compute it efficiently, which deals with dynamic range in a different way and allows for simultaneously sparse and illumination-multiplexed acquisition. <br /> As a means to improve the optical resolution of material measurements, or allow for larger material samples to be measured, a texture-space super-resolution algorithm based on deep-learning techniques is introduced. It is shown that despite working on an image-by-image basis, the resulting BTF does not exhibit obvious artifacts, and that data available realistically is sufficient ground truth for the purpose of training