Paradigm Shifts in 3D Shape Auto-Encoders

Complete Historical Review - 2015-2026

Key Themes
3D形状表示 编码器-解码器 排列不变性 谱偏置 Eikonal方程 元学习 生成建模 多尺度 神经场 有符号距离函数 潜在空间
2015-2018

Discrete Representations Era

Foundational period of deep learning for 3D shapes using explicit discrete structures (voxels, point clouds, meshes).

3D ShapeNets
arXiv:1406.5670 (2015)
- One of the earliest deep volumetric auto-encoders
3D体素概率分布 卷积深度信念网络 对比散度 Wake-Sleep算法 下一最佳视角
VoxNet
IROS (2015)
- Pioneered 3D CNNs on voxel grids
3D CNN 体素网格
OctNet
arXiv:1611.05009 (2016)
- Octree-based sparse 3D convolutions for high-resolution volumetric representations
混合网格-八叉树 位字符串编码 浅层八叉树 截断卷积核
PointNet
arXiv:1612.00593 (2017)
- First deep network on raw point clouds
排列不变性 对称函数 联合对齐网络 最大池化聚合
PointNet++
arXiv:1706.02413 (2017)
- Hierarchical feature learning on point sets
层次化特征学习 集合抽象层 多尺度分组 最远点采样 球查询
2019-2020

Implicit Neural Representations Revolution

Major shift to continuous implicit neural fields (SDFs and occupancy functions).

Occupancy Networks
arXiv:1812.03828 (2019)
- Introduced occupancy probability fields
隐式占用函数 多分辨率等值面提取 条件批归一化 变分目标
DeepSDF
arXiv:1901.05103 (2019)
- Landmark continuous SDF auto-encoder
有符号距离函数 自解码器 潜在向量 最大后验估计 零等值面
SIREN
arXiv:2006.09661 (2020)
- Periodic activation functions for implicit representations - Major improvement in representing high-frequency details
周期激活函数 正弦表示 隐式神经表示 高频细节
IM-Net
arXiv:1812.02822 (2019)
- Early implicit surface auto-encoder
隐式场 隐式场解码器 内部/外部场 等值面提取
Convolutional Occupancy Networks
arXiv:2003.04618 (2020)
- Extended implicit representations to large scenes
平移等变性 特征平面 体积特征网格 二元交叉熵损失
IF-NET
arXiv:2003.01456 (2020)
- Multi-scale feature grids in implicit space
多尺度3D特征张量 特征空间分类 三线性插值 体素超分辨率
2020-2021

Self-Supervised & Efficient Implicit Era

Techniques to train implicit representations from raw unlabeled data with major efficiency improvements and early meta-learning approaches.

SAL: Sign Agnostic Learning
arXiv:1911.10414 (2019)
- Sign-agnostic losses for raw point clouds
符号无关学习 无符号距离函数 符号无关损失函数 几何网络初始化 零水平集
Implicit Geometric Regularization (IGR)
arXiv:2002.10099 (2020)
- Eikonal equation as geometric prior
隐式几何正则化 Eikonal项 有符号距离函数 梯度约束 隐式正则化
Curriculum DeepSDF
arXiv:2003.08593 (2020)
- Curriculum learning strategy for improving DeepSDF training
课程学习 形状课程 样本难度 硬样本挖掘
StEik
arXiv:2305.18414 (2023)
- Stabilizing optimization of neural signed distance functions for finer shape representation
Eikonal损失不稳定性 Laplacian法向正则化 方向性散度 二次层 后向扩散
MetaSDF
arXiv:2006.09662 (2020)
- Meta-learning for signed distance functions - Early application of meta-learning to implicit representations
元学习 基于梯度的元学习 自动解码器 内循环适应 每参数学习率
NeuralPull
arXiv:2011.13495 (2020)
- Explicit surface-pulling loss for detail preservation
空间拉动 可微分拉动操作 符号距离函数 几何网络初始化
Fourier Features
arXiv:2006.10739 (2020)
- Fourier feature mapping for learning high-frequency functions in low-dimensional domains
傅里叶特征映射 神经正切核 谱偏置 随机傅里叶特征 位置编码
NGLOD
arXiv:2101.10994 (2021)
- Octree-based feature volume for real-time neural SDF rendering
神经几何细节层次 稀疏体素八叉树 特征体积 实时渲染
HYVE
arXiv:2310.06644 (2023)
- Hybrid graph + voxel architecture for single-pass encoding
混合顶点编码器 图卷积 粒子网格特征投影 Eikonal方程 无符号距离场
2021-2022

Semi-Supervised + Meta-Learning Era

Meta-learning and semi-supervised techniques for generalization to unseen object categories.

GenSDF
arXiv:2206.02780 (2022)
- Two-stage semi-supervised meta-learning - Zero-shot to 100+ unseen classes
神经符号距离函数 两阶段半监督元学习 片段式训练方案 自监督符号预测损失 零样本推理
Semi-supervised Implicit Scene Completion from Sparse LiDAR
arXiv:2111.14798 (2021)
- Eikonal-constrained semi-supervised SDFs on LiDAR
半监督隐式场景补全 空间变化稀疏性 形状嵌入体 可微三线性采样 位置编码
SSP3D
arXiv:2209.15383 (2022)
- Prototype-based semi-supervised single-view reconstruction
原型注意力模块 形状自然性模块 教师-学生互学习 指数移动平均
SSR: Semi-supervised Soft Rasterizer
arXiv:2108.09593 (2021)
- Semi-supervised differentiable rendering for 3D reconstruction
可微分软光栅化器 孪生视角预测器 度量学习 渐进式伪标签策略 剪影损失
2023-2026

Generative & Foundation Model Era

Large-scale generative models and foundation models for 3D.

3DShape2VecSet
arXiv:2301.11445 (2023)
- Set-based representation for diffusion models
神经场 径向基函数 交叉注意力 KL正则化 生成扩散模型
Point-E
arXiv:2212.08751 (2022)
- Text-to-3D diffusion model
两阶段生成过程 文本到图像扩散模型 图像到点云扩散 分层生成策略
LION
arXiv:2210.06978 (2022)
- Latent point diffusion for 3D generation
潜在点扩散模型 层次化潜在空间 去噪扩散模型 变分自编码器 可微分泊松表面重建
3D Gaussian Splatting
arXiv:2308.04079 (2023)
- Hybrid explicit-implicit real-time rendering
3D高斯场景表示 可微分光栅化渲染 自适应密度控制 各向异性协方差 球谐系数
MeshXL
arXiv:2405.20853 (2024)
- Neural coordinate field foundation model
神经坐标场 生成预训练自回归模型 预定义排序策略 条件生成 网格VQVAE