What Is COLMAP? Structure From Motion in Plain Words

What is COLMAP?

COLMAP is a free, open-source program that works out where each photo in a set was taken from, and builds a 3D point model of the scene while doing so. Its own documentation describes it as a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline.

Structure from motion is the first half. Given overlapping photos, COLMAP finds the same physical spots in several pictures and uses them to calculate each camera's position, direction and lens settings, plus a sparse cloud of 3D points. Multi-view stereo is the optional second half: once the cameras are known, it estimates depth for every pixel and builds a dense point cloud or a mesh.

If you have tried NeRF or Gaussian splatting, you have probably used COLMAP without seeing it. Many trainers either run it for you or expect its output as their input. This guide explains what COLMAP does, where it tends to go wrong, and when you can skip touching it at all.

Who makes it, the licence and the platforms

COLMAP was originally written by Johannes Schönberger, with support from his PhD advisors Jan-Michael Frahm and Marc Pollefeys. The two papers it asks you to cite are Structure-from-Motion Revisited (CVPR 2016) and Pixelwise View Selection for Unstructured Multi-View Stereo (ECCV 2016). The GitHub repository lists the current core maintainers as Johannes Schönberger, Paul-Edouard Sarlin, Shaohui Liu and Linfei Pan.

The COLMAP library is released under the new BSD licence, a permissive open-source licence. The project notes that bundled third-party components carry their own licences.

Ways to get it, according to the install page:

• Windows: pre-built binaries that contain both the graphical and the command-line program.
• Mac: Homebrew, with the command brew install colmap.
• Linux: distribution packages or a build from source. The docs point out that the distribution packages do not come with CUDA support.
• Docker: a pre-built image with CUDA support.
• Python: bindings called PyCOLMAP, published on PyPI.

A short COLMAP tutorial: the pipeline in plain language

Whatever interface you use, a reconstruction passes through the same stages.

• Feature extraction. COLMAP looks at every photo and picks out distinctive points, called keypoints, and stores a numerical description of each. SIFT is the default feature type.
• Feature matching. It compares those descriptions between photos to find the same spot seen from different angles. Exhaustive matching tries every pair of images, and the documentation warns that this grows quadratically with the number of images. Sequential matching is meant for ordered frames such as video. Vocabulary tree matching is the recommended mode for collections of several thousand images. Spatial matching uses GPS positions when the photos have them.
• Sparse reconstruction, also called the mapper. Starting from one pair of images, COLMAP adds the remaining photos one at a time and triangulates 3D points. The result is a set of camera poses and a sparse point cloud.
• Undistortion. Real lenses bend straight lines. This step produces corrected images that fit a simple camera model, which the dense stage and many outside tools expect.
• Dense reconstruction, optional. COLMAP computes a depth map and a normal map for each image, fuses them into a dense point cloud, and can then build a mesh with Poisson or Delaunay meshing.

Everything COLMAP extracts is kept in a single SQLite database file, and your input images are never modified.

COLMAP GUI vs the command line

COLMAP ships with both, and they do the same work.

The COLMAP GUI is started with the command colmap gui on Mac and Linux, or by double-clicking COLMAP.bat on Windows. The quickest path is Reconstruction > Automatic Reconstruction: you point it at a folder of images and it writes a sparse and a dense reconstruction into a workspace folder. The GUI also lets you watch the cameras and points appear as the model grows, which is useful when you are learning what a healthy reconstruction looks like. The install page mentions that Windows Defender may show a warning because the binaries are not officially signed.

The command line exposes each stage as its own command: feature_extractor, exhaustive_matcher, mapper, image_undistorter, patch_match_stereo and stereo_fusion. A single automatic_reconstructor command runs the whole chain. This is the interface that scripts, training tools and desktop apps call behind the scenes.

For a one-off project the GUI is easier. For repeated jobs the command line is more practical, because every run uses exactly the same settings.

Which parts need an NVIDIA GPU

This confuses many newcomers, so it is worth being precise. CUDA is NVIDIA's GPU computing system.

• Feature extraction and matching can use a GPU to go faster, but the COLMAP FAQ states that without a usable GPU you can run all steps on the CPU by setting the appropriate options. It is slower.
• The sparse reconstruction that produces camera positions does not depend on CUDA.
• Dense reconstruction is different. The FAQ says that with a GPU that does not support CUDA you can use all COLMAP functionality except the dense reconstruction part, and suggests external dense reconstruction software as an alternative.

The practical consequence: if all you want is camera positions for a NeRF or Gaussian splatting trainer, you need only the sparse half, and an NVIDIA card is not required. If you want COLMAP's own dense point cloud and mesh, you need a CUDA-capable NVIDIA GPU with enough memory. The tutorial warns that the dense step may crash if GPU memory runs out.

Why COLMAP fails, and how to fix it

A failed run usually shows up in one of two ways: only some of your photos are placed, or COLMAP produces several separate sub-models instead of one. The cause is nearly always the photos, not the settings. The capture advice in COLMAP's tutorial lines up with the common problems:

• Too little overlap. The tutorial asks that every object be seen in at least three images. Take more photos with smaller steps between them.
• Blank surfaces. A white wall or an empty desk gives the feature step nothing to hold on to. Keep textured objects or backgrounds in the frame.
• Pure rotation. Standing in one place and turning the camera gives no depth information. The tutorial says plainly not to take images from the same location by only rotating the camera. Move your feet between shots.
• Reflections and changing light. The tutorial recommends avoiding shiny, specular surfaces and high-contrast scenes such as shooting towards the sun, and keeping lighting similar across photos.
• Blur. Motion-blurred frames match poorly. The nerfstudio documentation sums it up: capture overlapping, non-blurry images.
• Too many near-identical frames. More images are not automatically better and make everything slower. For video, the tutorial suggests lowering the frame rate.

If you end up with several sub-models that share some images, COLMAP has a model_merger command for joining them. For scenes with little texture or few overlapping views, the FAQ notes that the ALIKED feature type can give more repeatable features than the default. For most people, though, reshooting with more overlap is the faster fix.

How COLMAP feeds NeRF and Gaussian splatting

A sparse COLMAP model consists of three things: the cameras (lens parameters), the images (a position and rotation for every photo) and the 3D points (each with a position and a colour). That is precisely what radiance field trainers need and cannot work out on their own.

• The original 3D Gaussian Splatting paper describes starting from the sparse points produced during camera calibration and turning them into 3D Gaussians. Its reference code includes a converter script that calls COLMAP and outputs undistorted images, because its renderer expects a simple pinhole camera model.
• Nerfstudio states that its data processing script uses COLMAP and FFmpeg, and adds a frank warning that COLMAP can be finicky.
• Brush, an open-source Gaussian splatting trainer, accepts COLMAP output as training data.

The trainer then spends its time improving the look of the scene, trusting the camera poses it was given. This is why a weak COLMAP result leads to a blurry or broken scene no matter how long you train. Capture quality decides the outcome earlier than most people expect.

Skipping the command line: apps that bundle COLMAP

You do not have to install or learn COLMAP to benefit from it. Some desktop apps include it and run it for you.

SplatLoom is one of them. It is a desktop app for Mac (Apple Silicon, macOS 15 or later) and Windows 10/11 that turns overlapping photos or a short video into a 3D Gaussian splatting scene on your own computer, with no upload and no account. It bundles COLMAP for one job only, finding the camera positions, then trains the scene with Brush on your local GPU. You pick your files and press Generate; there is no Python or command line.

Two honest notes. SplatLoom does not use COLMAP's dense stage and does not export meshes, so if you need a mesh for 3D printing, COLMAP's own dense pipeline on an NVIDIA GPU or a dedicated photogrammetry program is the right tool. And the capture rules above still apply: with poor overlap or shiny, moving subjects, the bundled COLMAP will struggle exactly as it would on its own.