What Is Gaussian Splatting? A Plain Guide for Creators

What is Gaussian splatting?

Gaussian splatting is a way to turn ordinary photos or a phone video into a 3D scene you can move around in. If you have seen a captured room, statue or street corner that you can orbit smoothly in a browser, and that looks almost like the original footage, there is a good chance it was a Gaussian splat.

The technique comes from the 2023 paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering" by Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler and George Drettakis (Inria, Université Côte d'Azur and the Max Planck Institute for Informatics), published at SIGGRAPH 2023. It is a method for novel-view synthesis: showing a captured scene from viewpoints the camera never stood at, fast enough to look around in real time.

A classic 3D model is a surface made of triangles. A Gaussian splat is different. It describes the scene as a very large number of small, soft, semi-transparent shapes, each with its own position, size, orientation, color and opacity. In the scenes tested in the paper, the finished result contained roughly 1 to 5 million of them. Seen together from any angle, they blend into an image that can look very close to a photograph.

How does Gaussian splatting work?

The process has three broad steps.

• Find the cameras. A structure-from-motion tool such as COLMAP compares your photos, matches the same visual features across them, and works out where each picture was taken from. As a by-product it produces a sparse point cloud, a rough set of 3D points that outlines the scene.
• Place Gaussians. Each of those points becomes the starting position of a 3D Gaussian, a small ellipsoid-shaped blob. Every Gaussian stores a position, a shape and orientation (its covariance), an opacity, and a color described with spherical harmonics, which lets its color shift slightly depending on the viewing angle.
• Optimize. The software renders the Gaussians from the same viewpoints as your photos, compares each render with the real photo, and nudges every parameter to reduce the difference. Along the way it adds detail where the scene is under-represented, by cloning small Gaussians or splitting large ones, and removes Gaussians that become almost transparent. The paper's full-quality setting runs 30,000 of these iterations.

Viewing is fast because the result is explicit data. The renderer sorts the Gaussians by depth for the current camera and blends them onto the screen with the GPU's normal rasterization hardware, rather than querying a neural network for every pixel. The authors report real-time rendering at 1080p resolution.

Gaussian splatting vs NeRF

Gaussian splatting is often compared with NeRF (Neural Radiance Fields), introduced by Ben Mildenhall and colleagues in 2020. A NeRF stores a scene inside a neural network. To draw a pixel, the renderer samples many points along a ray from the camera, asks the network for the density and color at each point, and combines the answers with volume rendering. Quality can be excellent, but every frame needs a very large number of network queries.

The 3DGS paper measured the gap on its test scenes. On the Mip-NeRF 360 dataset, the high-quality Mip-NeRF 360 method took about 48 hours to train and rendered at about 0.06 frames per second. Gaussian splatting trained in about 41 minutes and rendered at around 134 frames per second, with comparable image quality. Faster NeRF variants such as Instant-NGP already existed, but in the same test they rendered at roughly 9 to 12 frames per second, with lower quality on these large scenes.

The practical difference for a creator: a NeRF is an implicit representation hidden in network weights, while a Gaussian splat is an explicit list of blobs that you can view, crop and clean up directly. Both are radiance field methods, and both start from photos with known camera positions.

Is Gaussian splatting AI?

It depends on what you mean by AI. Gaussian splatting is not generative AI. It does not write, imagine or invent content the way a chatbot or an image generator does. It fits a model to your own photos, so what it shows comes from what your camera recorded. Areas the camera never saw usually end up blurry, sparse or empty rather than filled in with made-up detail.

It does borrow tools from machine learning. The fitting step uses gradient-based optimization, the same family of methods used to train neural networks, and it normally runs on a GPU. The original method does not contain a neural network, though, so the fairest description is an optimization technique closely related to machine learning rather than an AI model.

How to capture photos for a good splat

Capture matters more than any setting. The software can only reconstruct what your camera saw, ideally from several angles.

• Overlap. Each photo should share most of its view with the next one. Apple's Object Capture guidance asks for 70% overlap or more between neighboring shots and warns that results may fail below 50%. The same rule of thumb works well for splats.
• Walk around the subject. Move your feet rather than turning on the spot, and do loops at more than one height so the top and lower sides are covered.
• Enough photos. A typical photo set is 30 to 150 overlapping photos, depending on the size and detail of the subject.
• Soft, even light. Hard shadows from direct sun or a flash can confuse reconstruction. Overcast daylight or bounced indoor light is easier.
• Consistent exposure. On iPhone you can long-press the camera view until AE/AF Lock appears, so brightness and focus do not jump between shots.
• Video works too. A slow, steady video lets the software extract frames for you. Move smoothly and keep the light bright enough to avoid motion blur.

Limitations to know about

Splats are excellent at making a captured place look real from new angles, but they do not suit every 3D task.

• Not a measuring tool. A splat is meant for viewing. It is not built for CAD work or precise measurement.
• No solid surface. Because the scene is made of soft blobs, there is no clean mesh to 3D print, use for collisions in a game engine, or model in Blender. Tools that convert splats to meshes exist, but the mesh usually loses much of the look that made the splat appealing.
• Difficult subjects. Mirrors, glass, polished metal and water change appearance as you move, and moving things such as people, pets or leaves in the wind break the assumption that the scene is still. Expect floating artifacts, blur or ghosting.
• Gaps in coverage. Anything the camera did not see clearly will be missing or smeared.
• Viewer support. Splats need a splat-aware viewer or editor, while a mesh opens almost anywhere.

Making Gaussian splats on your own computer with SplatLoom

You do not need a server or a command line to try this. SplatLoom is a desktop app for Mac (Apple Silicon, macOS 15 or later) and Windows 10/11 x64 that turns overlapping JPG or PNG photos, or an MP4 or MOV video, into a 3D Gaussian splatting scene entirely on your own computer. There is no upload, no account and no cloud processing.

The default flow is short: pick photos or a video, click Generate, then view the result and clean it up with box or sphere crop, rectangle select, one-click spike cleanup and undo/redo. You can export a Gaussian PLY file, a PNG still, a 12-second 1080p MP4 orbit video, or a self-contained offline HTML viewer. It is built on open-source engines bundled in the installer: COLMAP for camera positions, Brush for training the splat on your local GPU, and PlayCanvas for the viewer. You do not need to install Python.

In our test, a 13 to 20 second smartphone video was processed end to end in about 9 to 14 minutes on an Apple M5 Pro MacBook. That was a single test, not a promise; your time will depend on your computer and your footage.

SplatLoom is a US$19.90 one-time purchase that includes both the Mac and Windows installers, with no subscription and no credits, and a 7-day refund if it does not run on your computer. It does not export meshes, it is not a measurement tool, and its Windows GPU support is newer than its Mac support.