Computer Graphics

Neural Representations

Learned scene representations, and their reconstruction lineage.

Colour is the family; a dashed line is the second member of it.

Drag to pan · scroll to zoom · click a node to open it
G gaussian-splatting 3D Gaussian Splatting nerf Neural Radiance Fields gaussian-splatting->nerf rasterizes explicit primitives instead of querying a field rasterization Rasterization gaussian-splatting->rasterization the speed comes from sorted alpha-blended splatting stochastic-gradient-descent Stochastic Gradient Descent gaussian-splatting->stochastic-gradient-descent the primitives are fitted to photographs by gradient descent instant-ngp Instant NGP multiresolution-hash-encoding Multiresolution Hash Encoding instant-ngp->multiresolution-hash-encoding the speedup is the encoding, not the network mip-nerf Mip-NeRF mip-nerf->nerf point samples alias when rendered at another scale multiresolution-hash-encoding->nerf the deep MLP, not the idea, is what made it slow radiative-transfer-equation Radiative Transfer Equation nerf->radiative-transfer-equation an MLP supplies density and colour along the ray nerf->stochastic-gradient-descent the field is fitted to photographs by gradient descent neural-radiance-cache Neural Radiance Cache irradiance-caching Irradiance Caching neural-radiance-cache->irradiance-caching a learned cache instead of an interpolated one path-tracing Path Tracing neural-radiance-cache->path-tracing terminates paths into a cache learned while rendering neural-radiance-cache->stochastic-gradient-descent the cache is trained online, while rendering plenoxels Plenoxels plenoxels->nerf a sparse grid works too; the network was not the point

7 nodes

Neural Radiance Fields

Fits an MLP mapping position and direction to density and colour, then renders it by volumetric ray marching. Optimizing a radiance field directly ag… · 2020

Mip-NeRF

Encodes conical frusta rather than points, so the representation is aware of the footprint each sample covers and stops aliasing when rendered at a d… · 2021

Neural Radiance Cache

Trains a small network online, during rendering, to predict outgoing radiance, so paths can terminate into a learned estimate after a couple of bounc… · 2021

Instant NGP

A fused CUDA implementation built on hash encoding that trains a usable radiance field in seconds rather than hours, which is what turned NeRF from a… · 2022

Multiresolution Hash Encoding

Replaces most of the network's capacity with trainable feature vectors in hash tables at several resolutions, so a tiny MLP suffices and collisions a… · 2022

Plenoxels

Optimizes a sparse voxel grid of spherical harmonic coefficients with no network at all, matching NeRF quality and demonstrating that the neural part… · 2022

3D Gaussian Splatting

Represents the scene as anisotropic Gaussians with spherical harmonic colour and rasterizes them sorted by depth. Explicit primitives and a rasterize… · 2023