Progressive refinement imaging by variable-resolution image and range fusion

In the past years, various algorithmic approaches have been proposed that address the fusion of multiple camera observations, enabling the acquisition of scenes that cannot be captured with a single photograph. Despite various improvements in seamless image blending, a key challenge to creating a convincing composite remains in compensating for geometric and photometric discrepancies (due to, for example, changes in viewpoint and illumination conditions). While previous methods mitigate these inconsistencies mainly through global optimization, any kind of computationally intensive post-processing prevents an acquisition in an interactive, online fashion.
In this thesis, novel methods for fusing a stream of camera observations into a progressively refined, consistent image representation are proposed. By enriching a low-resolution image with high-resolution details from close-ups, the user is allowed to interactively increase resolution locally where added image detail is desired.
First, a method is proposed to fuse an RGB image sequence with substantial geometric and photometric discrepancies into a single consistent output image. It can handle large sets of images, acquired from a nearly planar or far-distant scene at variable object-space resolutions and under varying local or global illumination conditions. At its core, a dynamically extendable multi-scale representation allows for variable-resolution image fusion. Details from the incoming image data are selectively merged in a way that removes artifacts such as lens distortions, lighting changes, or varying exposure and color balance.
Second, by bridging between 2D and 3D approaches, a disparity-corrected method is proposed that allows adaptive image refinement for general 3D scenes, even in the presence of silhouettes and strong scene parallax. It features the fusion of handheld RGB-D camera streams into a high-quality, variable-resolution 2.5-D reconstruction (color and range data). This is enabled by a parallax-aware image warping, assisted by adaptively refined depth values to compensate for parallax effects due to depth disparities. All pipeline modules are designed for resilience against low-resolution, artifact-prone depth readings while refining the high-resolution color data.