Introduction.

Traditional 3D modeling pipelines. Computer vision as inverse graphics. Neural 3D representations and neural rendering. Democratization of 3D content creation. Synthetic 3D data for ML training pipelines.

Lecture Slides: Intro

Reading: SyntheticDataforML

Classical 3D Geometry Representations.

Low level: voxel grids, point clouds, triangular and quad meshes. High level: parametric and implicit boundary representations. Review of some classical geometric concepts (normals, curvature).

Lecture Slides: ClassicalGeometry

Reading: Old survey, normal estimation

Neural Architectures for Regular Data.

Brief review of deep nets and convolutional architectures for images. Sparse convolutions. Transformers. Voxel-based 3D methods for shapes and hierarchical variants.

Lecture Slides: NNs,_Voxels

Reading: 3D-R2N2, 3D-Vox-Trans, 3D-Sem-Seg, OctGenNet, VoxNet2

Irregular Geometries: Point Clouds.

PointNet and PointNet++. KPConv and other related methods. Sampling issues. Applications to object detection, classification, and segmentation.

Lecture Slides: PointClouds

Reading: PointNet, PointNet++, DGCNN, VoteNet, FlowNet3D

Martin Luther King, Jr., Day (holiday, no classes).

Introduction to Generative Models (VAEs, deepSDF).

Autoencoders and autodecoders. Variational autoencoders. Deep signed distance functions.

Lecture Slides: VAE_deepSDF

Reading: VarEncoders1, VarEncoders2, VQVAE, DeepMetafunctionals, IMNet, DeepSDF

Staff Student Presentation Sample: Slides,

KPConv, PointConv, 3DETR

Homework 1 out.

Parametric Models. Generative Adversarial Networks (GANs) for 3D. Disentanglement.

AtlasNet. InfoGAN, 3D-GAN, StyleGAN.

Lecture Slides: Param_GANs

Reading: GAN1, GAN2, AtlasNet, InfoGAN, WassersteinGAN, StyleGAN1

Student Presentation: Slides1, Slides2,

PC Implicit Autoencoder, StructuredImplicits, LocalImplicits

3D Shape Public Data Sets. Flow and Auto-Regressive Models.

ShapeNet, PartNet, ... PointFlow, PolyGen.

Lecture Slides: DataSets_PolyGen_PointFlow

Reading: ShapeNet, PartNet, ABC, PolyGen, PointFlow, CaSPR, MeshODE

Student Presentation: ParSeNetSlides, StyleGAN2Slides, StyleGAN3Slides,

ParSeNet, StyleGAN2, StyleGAN3

Homework 1 due. Homework 2 out.

Hierarchical Generation of Structure and Geometry.

GRASS, StructureNet, ComplementMe.

Lecture Slides: Stuct

Reading: GRASS, StructureNet, StrucEdit, StructSynth, Abstractions

Student Presentation: scene_completion_slides, gradientfields_slides,

NeuralODEs, NormalizingFlows, GradientFields, PCGen

Vector Graphics, Deep Architectures for Meshes.

Vector graphics generation, convolutions on meshes. BReps.

Lecture Slides: Vector

Reading: Im2Vec, MeshCNN, BRepNet, Sketch2CAD

Student Presentation:

Latent_Fact_Slides, GlobalLocalSlides,

GlobalLocal, SDM-NET, CompModel, SCORES

Pose Equivariance and Invariance in 3D Data.

Vector neurons, TFN networks.

Lecture Slides: EquivInv

Reading: Vector Neurons, TFNs

Student Presentation:

DeepSketchSlides, PCEdgesSlides

DeepSketchModeling, DeepCAD, PointCloudEdges

Conditional Generation: 3D Shape Completion.

Lecture Slides: CondGen

Reading: struct_complete, complement_me, dual_space

Student Presentation:

SurfaceCNNSlides, MultiViewSlides, ConDorSlides

ConDor, SurfaceCNNs, EquivariantMultiView

Homework 2 due. Homework 3 out.

Conditional Generation: From Image to Shape.

Lecture Slides: Img2Shp

Reading: deepmetafunctionals, deeppointset, NOCS, Pix2Surf, DISN, PiFu, no3Dsup, unseenclasses

Student Presentation:

MultimodaSlidesl, GANinversionSlides

PU-GAN, multimodal, GAN_invesrion

Learning Discrete and Continuous Shape Edits/Deformations. Shaping Latent Spaces.

Latent shape differences. Neural shape deformations/edits.

Lecture Slides: Deform

Reading: MeshODE, ShapeFlow, DeformSyncNet, StructEdit, NeuralCages, DeepMetaHandles, DefAwareRetrieval

Student Presentation:

D2IMSlides, PlatonicGANSlides

PlatonicGAN, D2IM-Net, CoReNet

Project proposals due.

Presidents' Day (holiday, no classes).

Neural Functions: 3D from 2D Supervision.

Neural rendering. Neural radiance fields (NeRFs).

Lecture Slides: NeRFs

Reading: DiffVolRend, GeomApp, NeRF, NeurText, NeurVol, StableView, StereoMag

Student Presentation:

3DNSlides, CAD-DeformSlides

3D Deformation Network, CAD-Deform, LOGAN

Homework 3 due.

Neural Fields and Surfaces.

NeRF variations and extensions.

Lecture Slides: NeRVar

Reading: Nerfies, HyperNerF, NeuS, VolSDF, ObjectNeRF

Student Presentation:

GRAFSlides, GIRAFFESlides, PixelNeRFSlides

GRAF, GIRAFFE, pixelNeRF

Scene Generation and Object Placement.

GRAINS, MetaSim, MetaSim2.

Lecture Slides: Scenes

Reading: GRAINS, MetaSim, MetaSim2

Student Presentation:

NSG_Slides, ING_Slides

NSG. ObjectNerF, InstantNeuralGraphicsPrimitives

Object/Scene Generation and Language.

ShapeGlot, PartGlot.

Lecture Slides: Language

Reading: ShapeGlot, PartGlot, VLGrammar

Student Presentation: BlockGAN, RELATE, PlanIt, DeepGenSynth

Student Project Presentations.

Lecture Slides:

Reading:

Project due.