(also available as AVI)
(also available as AVI)
|
|
|
Click on a thumbnail to watch the corresponding image sequence (MOV file).
(also available as AVI) |
Note that both sequences were computed from a single snapshot by the camera.
(also available as AVI) |
| Our first prototype light field microscope (LFM). The eyepiece has been removed and replaced with a microlens array (circled in red). Above the array is a camera and 1:1 relay lens. Click here to see a more portable version. | A light field as recorded by the optical arrangement at left. The specimen is a 200-micron-tall "tower" of fluorescent crayon wax. The objective is 20x/0.5 (dry), used without a cover slip. | Perspective pan. Since microscopes are telecentric, they are normally only capable of producing orthographic views. Computed in real time. | Focal stack. Using deconvolution, this 3D stack can be converted into a 3D (cross-sectional) volume. See below for more examples. |
Light field photography is an image-based technique for recording scene appearance. Unlike conventional photography, light fields permit manipulation of viewpoint and focus after the imagery has been recorded. Devices for capturing light fields range in scale from room-size arrays of cameras to a handheld camera in which a microlens array has been inserted between the main lens and sensor plane.
In this project, we are exploring light field photography at the microscopic scale. Specifically, by inserting a microlens array into the optical train of a conventional optical microscope, we can capture light fields of biological specimens in a single snapshot. Although diffraction places a limit on the product of spatial and angular resolution in these light fields, we can nevertheless produce useful perspective flyarounds and 3D focal stacks from them. Applying standard deconvolution algorithms to these focal stacks, we can reconstruct 3D volumes. Since microscope optics produce orthographic views, perspective flyarounds represent a new way to look at microscopic specimens. Focal stacks are not new, but manual techniques for capturing them are time-consuming and hence not applicable to moving or light-sensitive specimens.
Like any new scientific imaging instrument, we expect the light field microscope to have many applications in science, medicine, and industry. For example, since the light field microscope separates image acquisition from the selection of viewpoint and focus. it can be used as a "digital viewfinder" for a conventional microscope. Applying computer vision algorithms to these selection procedures may lead to greater automation of microscope operation when analyzing large numbers of specimens, such as in clinical pathology.
We are currently looking for additional students to work on this project.
|
2005 - our first light field micrographs, including insect legs, silkworm mandibles, and mouse lungs. Also included are volumetric reconstructions using 3D deconvolution, and some experimental all-focus images. |
|
2006 - light field micrographs captured at the Marine Biological Laboratory, including onion skin (under DIC illumination), squid skin (under grazing illumination), a light field scatterometry experiment, and mouse embryos. |
|
2007 - light field micrographs captured at the Hopkins Marine Station, including fern spores and cone snails. Also included is a description of our portable light field microscope and real-time software viewer. |
February 14, 2008
Microscope design: Marc Levoy
Specimen: Shinya Inoue (Marine Biological Laboratory)
|
|
| Compact light field microscope | Prototype high-resolution light field microscope |
At left above is our most compact light field microscope (LFM). As introduced on this web page, it consists of an ordinary research microscope (Nikon Eclipse 80i) and cooled scientific camera (Retiga 4000R) with a microlens array inserted between the two (red circle). The image formed by the microlens array is conveyed to the sensor in the camera by a 1:1 relay lens system consisting of two nose-to-nose Nikon 50mm/1.4 photographic lenses. With this arrangement, instead of capturing a 2048 x 2048 pixel photograph, the camera captures a light field capable of producing 17 x 17 different oblique views, each 120 x 120 pixels in size, from a single snapshot. We have fabricated several copies of this arrangement, which have been used by microscopists in several departments at Stanford University, at the Hopkins Marine Institute (Monterey, CA), the Marine Biological Laboratory (Woods Hole, MA), and ETH (Zurich).
At right above is a prototype high-resolution light field microscope. A 45-degree mirror (A) sends the microscope image through a 2:1 telecentric relay lens (B), our microlens array (C), a 1:1 telecentric relay lens (D), and a Canon 5D full-frame digital SLR (E). Since increasing the lateral magnification using a 2:1 relay lens decreases the angular aperture, this microlens array is f/30, rather than the f/20 array we more commonly use. The camera was manually microstepped to place a green pixel beneath each pixel site, thereby avoiding color demosaicing and the consequent loss in spatial resolution. The light fields captured using this arrangement (see below) contain 15 x 15 oblique views, each approximately 300 x 200 pixels in size. The field of view is approximately 1.8mm horizontally for the 20x light field and 0.9mm for the 40x. For the objectives we used, these light fields are also diffraction-limited, meaning that no further spatial or angular resolution can be squeezed from the wavefront propagating through the microscope, at least not using conventional image capture protocols.
Light field (click on image for full size) 20x/0.75NA (dry) 
Flyaround (click on image to watch movie) (or reduced to CIF size in this .mp4 file) |
Light field (click on image for full size) 40x/1.3NA (oil) 
Flyaround (click on image to watch movie) (or reduced to CIF size in this .mp4 file) |
| Click here to view these light fields interactively in your browser, using a Flash-based light field viewer. |
The two columns above represent two light fields captured using this high-resolution microscope. The specimen is a Golgi-stained slice of rat brain (courtesy of Shinya Inoue, Marine Biological Laboratory). The light field at left was photographed using a 20x/0.75NA objective, and the light field at right using a 40x/1.3NA oil immersion objective. Note the spatial detail in both movies; you can easily see the fine dendrites. Note also the large amount of parallax visible in the 40x movie, especially on the looping capillaries in the lower-left part of the field of view. This much parallax is available because a 1.3NA objective can capture rays leaving the specimen at angles up to 59 degrees on either side of the optical axis. For photographers, this would correspond to a lens having a relative aperture of f/0.38 - far faster than any commercially available photographic lens.