Notes

Outline

Slide 1

The Digital Michelangelo Project

Slide 3

Slide 4

Eight hard problems in 3D scanning optically uncooperative materials scanning in the presence of occlusions insuring safety for delicate objects scanning large objects at high resolution accurate scanning in the field filling holes in dense polygon models handling large datasets creating digital archives of 3D content Some games with range data

Scanners used in the Digital Michelangelo Project

Our triangulation-based scanner

Scanning St. Matthew

single scan of St. Matthew

Hard problem #1: optically uncooperative materials fuzzy scattering transparent moving etc.

How optically cooperative is marble?  [Godin et al., Optical Measurement 2001] systematic bias of 40 microns noise of 150 – 250 microns worse at oblique angles of incidence worse for polished statues

Laser scattering noise on David’s toe

Some marbles scatter more than others

Hard problem #2: scanning in the presence of occlusions occlusions (& self-occlusion) force grazing scans, which lead to holes a scanner with a fixed triangulation angle cannot circumvent all occlusions a hammer & chisel can reach places a triangulation scanner cannot

Some statues may be “unscannable” (using optical methods)

Hard problem #3: insuring safety for delicate objects energy deposition not a problem for marble statues avoiding collisions

Why are collisions hard to avoid? to insure safety, scan head should stay outside the convex hull of the object in the worst case, required standoff may equal diameter of convex hull a complicated object cannot be scanned entirely from outside its convex hull

Hard problem #3: insuring safety for delicate objects energy deposition not a problem for marble statues avoiding collisions manual motion controls automatic cutoff switches one person serves as spotter avoid time pressure get enough sleep surviving collisions pad the scan head

Hard problem #4: scanning large objects at high resolution David is 5 meters tall chisel marks need 1/4mm dynamic range of  20,000:1 20,0002  =  1 billion polygons 14cm wide working stripe David was ~30 stripes around

Digital Michelangelo Project reconfigurable robotic gantry calibrated motions pitch  (yellow) pan  (blue) horizontal translation  (orange) uncalibrated motions vertical translation remounting the scan head moving  the entire gantry

Scanning St. Matthew 104 scans 800 million polygons 4,000 color images 15 gigabytes 1 week of scanning

Scanning the David 480 individually aimed scans 2 billion polygons 7,000 color images 32 gigabytes 30 nights of scanning 22 people

Hard problem #5: accurate scanning in the field rotating scan motion is hard to make accurate field reconfigurations are not repeatable need a way to recalibrate in the field

Digital Michelangelo Project range processing pipeline steps 1.  manual initial alignment - should have tracked gantry 2.  ICP to one existing scan [Besl92] 3.  automatic ICP of all overlapping pairs [Rusinkiewicz01] 4.  global relaxation to spread out error [Pulli99] 5.  merging using volumetric method [Curless96]

Digital Michelangelo Project range processing pipeline steps 1.  manual initial alignment - should have tracked gantry 2.  ICP to one existing scan [Besl92] 3.  automatic ICP of all overlapping pairs [Rusinkiewicz01] 4.  global relaxation to spread out error [Pulli99] 5.  merging using volumetric method [Curless96]

Digital Michelangelo Project range processing pipeline steps 1.  manual initial alignment 2.  ICP to one existing scan 3.  automatic ICP of all overlapping pairs 4.  global relaxation to spread out error 5.  merging using volumetric method

What really happens? steps 1.  manual initial alignment 2.  ICP to one existing scan 3.  automatic ICP of all overlapping pairs 4.  global relaxation to spread out error 5.  merging using volumetric method

What really happens? steps 1.  manual initial alignment 2.  ICP to one existing scan 3.  automatic ICP of all overlapping pairs 4.  global relaxation to spread out error 5.  merging using volumetric method

Digital Michelangelo Project color processing pipeline steps 1.  compensate for ambient illumination 2.  discard shadowed or specular pixels 3.  map onto vertices – one color per vertex 4.  correct for irradiance ® diffuse reflectance

What really happens? steps 1.  compensate for ambient illumination 2.  discard shadowed or specular pixels 3.  map onto vertices – one color per vertex 4.  correct for irradiance ® diffuse reflectance

The importance of subsurface scattering

artificial surface reflectance

estimated diffuse reflectance

Slide 34

Hard problem #6: filling holes in dense polygon models

Hard problem #7: handling large datasets

Some solutions we’ve used range images instead of polygon meshes z(u,v) yields 18:1 lossless compression multiresolution using (range) image pyramid multiresolution viewer for polygon meshes 2 billion polygons immediate launching real-time frame rate when moving progressive refinement when idle compact representation fast pre-processing

The Qsplat viewer [Rusinkiewicz and Levoy, Siggraph 2000] hierarchy of bounding spheres with position, radius, normal vector, normal cone, color traversed recursively subject to time limit spheres displayed as splats

Streaming Qsplat  [Rusinkiewicz and Levoy, I3D 2001]

Hard problem #8: creating digital libraries of 3D content metadata – data about data versioning – cvs for data archives secure viewers for 3D models robust 3D digital watermarking viewing, measuring, extracting data indexing and searching 3D content insuring longevity for the archive

How to think about range data range data is measured data both the geometry and the topology are noisy “geometric signal processing” range data is not a cloud of unorganized points use connectivity between range samples use lines of sight to scanner and camera range datasets are large representations that are exact only in the limit algorithms that are fast and approximate implementations that don’t need entire model in memory

Games with range data volumetric scan conversion range image synthesis multi-modality scanned models

Game #1: volumetric scan conversion convert point cloud to volume densities facilitates volumetric / statistical analyses new texture synthesis algorithms volume texture synthesis

Game #2: range image synthesis texture synthesis of range data in range images in merged surface meshes application to hole filling

Game #3: multi-modality scanned models time-of-flight scans for overall shape triangulation scans for range texture photography for reflectance

Three final questions Can range scanning be fully automated? Can we build a 3D fax machine? Will IBR  replace 3D scanning?

The Forma Urbis Romae: a marble map of ancient Rome

Slide 48

Can we replace this?

Can we replace this?

Can we replace this?

Can we replace this?

Automatic 3D scanning? requires 6 DOF robot automatic view planning  [Connolly85, Maver93, Pito96, Reed97,…] hard to guarantee safety use CT scanning instead?

Can we build a 3D fax machine? quality and cost of rapid prototyping is improving need automatic scanning need color printing how to match BRDF? looking for a killer app

Will IBR replace 3D scanning? Yes simple, brute force universal meta-primitive geometry is implicit independent of scene complexity multiresolution is trivial hardware acceleration No discards structure of model rasterized geometry big and slow