WYSIWYG Computational Photography via Viewfinder Editing

Jongmin
Baek
Dawid
Pająk
Kihwan
Kim
Kari
Pulli
Marc
Levoy

Proc. ACM SIGGRAPH Asia, November 2013

Figure. Viewfinder editing and appearance-based metering. (a) In the proposed system, the user interacts with the camera viewfinder as if it were a live canvas, by making sparse strokes to select regions and to specify edits. The edits propagate spatiotemporally as the camera and the scene objects move, and the edits are applied in the subsequent frames of the viewfinder, which are tonemapped HDR images, to provide WYSIWYG experience. (b) After making a local tonal edit, the user presses the shutter to trigger a high-resolution capture. Here we show the resulting tonemapped HDR composite without any edits applied, for reference. The edit mask computed from the user strokes is shown in the inset. (c) The result with edits applied is shown. This approximately corresponds to what the user sees on the screen just as he presses the shutter. Our appearance-based metering acquired an exposure stack at (0.645 ms, 5.555 ms, 11.101 ms) by optimizing the quality of the final result based on the user's global and local edits. (d) The regions indicated in (c), magnified. (e) We captured another stack with a histogram-based HDR metering at (0.579 ms, 9.958 ms, 23.879 ms) and applied the same post-processing pipeline. %, and show the same regions as in (d). As the standard HDR metering considers equally all the pixels in the scene, it uses too much effort to capture the dark areas that were not as important to the user, leading to a longer capture times that makes ghosting more likely (top), and higher noise in mid-tones (bottom).


On Youtube:

On Vimeo:

Watch it in HD at http://vimeo.com/71116200. (Follow the link and click on "HD".)

Abstract

Digital cameras with electronic viewfinders provide a relatively faithful depiction of the final image, providing a WYSIWYG experience. If, however, the image is created from a burst of differently captured images, or non-linear interactive edits significantly alter the final outcome, then the photographer cannot directly see the results, but instead must imagine the post-processing effects. This paper explores the notion of viewfinder editing, which makes the viewfinder more accurately reflect the final image the user intends to create. We allow the user to alter the local or global appearance (tone, color, saturation, or focus) via stroke-based input, and propagate the edits spatiotemporally. The system then delivers a real-time visualization of these modifications to the user, and drives the camera control routines to select better capture parameters.

Paper (PDF): Preprint (11MB)

Paper Video (MP4): Paper Video (205MB; 720p; all rights reserved)

Slides: To appear after the conference.

Bibtex:
@article{Baek:2013,
  author = {Baek, Jongmin and Paj\k{a}k, Dawid and Kim, Kihwan and Pulli, Kari and Levoy, Marc},
  title = {WYSIWYG Computational Photography via Viewfinder Editing},
  journal = {ACM Trans. Graph.},
  volume = {32},
  number = {6},
  year = {2013},
  pages = {198:1--198:10},
  doi = {10.1145/2508363.2508421}
}