Upsampling Range Data in Dynamic Environments

Jennifer Dolson Jongmin Baek Christian Plagemann Sebastian Thrun

CVPR 2010


We present a flexible method for fusing information from optical and range sensors based on an accelerated high-dimensional filtering approach. Our system takes as input a sequence of monocular camera images as well as a stream of sparse range measurements as obtained from a laser or other sensor system. In contrast with existing approaches, we do not assume that the depth and color data streams have the same data rates or that the observed scene is fully static. Our method produces a dense, high-resolution depth map of the scene, automatically generating confidence values for every interpolated depth point. We describe how to integrate priors on object shape, motion and appearance and how to achieve an efficient implementation using parallel processing hardware such as GPUs.


Adobe Acrobat PDF (1.4 MB)


Example of Processing the highway1 dataset: AVI Video (4.1 MB)

Example of Processing the synthetic1 dataset: AVI Video (9.8 MB)

Data Sets

One download:    All 8 data sets
Individual data sets:
    highway1     highway2      avenue1      avenue2

    synthetic1     synthetic2     synthetic3     synthetic4

(Note: all analysis in the paper uses the GPU version of the algorithm)

Here is the GPU implementation, as described in the paper, which runs on linux with NVIDIA's cuda installed (in theory it should also run on Windows, but that theory has not been tested). There's a README file and sample Makefile included: download

Here is the CPU implementation, also with a README and sample Makefile: download.

For image processing operations and data visualization, we recommend ImageStack. ImageStack also includes a more general joint bilateral filter implmentation for the CPU, written by Andrew Adams.

For more information about the original versions of the data structures used in our GPU and CPU code, see:
Fast High-Dimensional Filtering Using the Permutohedral Lattice and
Gaussian Kd-Trees for Fast High Dimensional Filtering