Camera Calibration¶
Every 3D feature in LUC3D — triangulation, reprojection error, the 3D viewport, reprojection-aided labeling — depends on knowing exactly where each camera is and how it maps the world onto its sensor. That information is the camera calibration, and LUC3D reads it from a calibration.toml (or JSON) file. LUC3D never computes a calibration itself; it consumes one.
calibrat3 is the companion tool that produces it. Like LUC3D, it runs entirely in the browser with no installation, and it exports a sleap-anipose-compatible calibration.toml that LUC3D loads directly.
| App | talmolab.github.io/calibrat3 |
| Source | github.com/talmolab/calibrat3 |
| From LUC3D | Click Calibrate in the menu bar to open it in a new tab |
→ For the click-by-click walkthrough, see Calibrating Cameras.
What a calibration contains¶
For each camera, two groups of numbers:
| Group | Parameters | What it describes |
|---|---|---|
| Intrinsics | matrix (fx, fy, cx, cy), distortions (k1, k2, p1, p2, k3) |
The lens and sensor of one camera — how a 3D point in that camera's frame lands on its image |
| Extrinsics | rotation (Rodrigues), translation |
Where that camera sits and points, relative to the shared world frame |
Intrinsics are a property of the camera and lens alone; extrinsics only exist relative to the rig. The world frame is the reference camera — the camera you designate during calibration gets rotation = [0, 0, 0], translation = [0, 0, 0], and every other camera's pose is expressed relative to it. All 3D coordinates LUC3D reports are in this frame, in the units of your calibration board's square size (millimeters, if you measured the board in millimeters).
See Multi-Camera Setup for the exact file format.
The calibrat3 pipeline¶
calibrat3 takes synchronized videos of a ChArUco board — a chessboard with an ArUco marker in every white square — waved through the shared working volume, and turns them into the numbers above. A ChArUco board is used instead of a plain chessboard because the markers are individually identifiable: a partly occluded or partly out-of-frame board still yields usable, correctly-labeled corners, which matters when 18 cameras see the board from very different angles.
1. Detect¶
Every sampled frame of every view is decoded and searched for the board. A detection is a set of corner IDs plus their sub-pixel image coordinates. Because corner IDs are global to the board, a corner seen in camera A and camera B at the same frame is known to be the same physical point — this is what ties the cameras together later.
Detection is the expensive stage (roughly 100 ms per megapixel frame), so calibrat3 runs it as one streaming decode pass per view feeding a pool of web workers. Thousands of frames are practical.
2. Intrinsics¶
For each camera independently, calibrat3 fits the intrinsic matrix and distortion coefficients from that camera's own detections (OpenCV calibrateCameraExtended), then evaluates the per-frame reprojection error on every valid frame so you can spot and exclude bad ones.
Fewer distortion terms usually generalize better
The default model fits k1 only — the same choice Anipose makes. Richer models drive the per-camera RMS down but often make the cross-view error worse, because the extra terms absorb per-camera effects (motion blur, rolling shutter, board flex) that don't transfer between views. On an 18-camera rig: full 5-parameter model → 12.7 px cross-view median, k1+k2 → 12.1 px, k1 only → 10.3 px.
3. Extrinsics¶
With intrinsics known, each camera's pose per frame comes from solvePnP on the board. calibrat3 builds a covisibility graph — an edge between two cameras for every frame in which both see enough of the same corners — and picks a chain of pairs from the reference camera outward, averaging the relative pose over all covisible frames with outlier rejection. Chaining those relative poses gives every camera an absolute pose.
This is why board coverage matters more than board count: two cameras that never see the board at the same time can only be related through a third camera that saw both.
4. Bundle adjustment¶
The chained poses are a good initialization, not an optimum. Bundle adjustment triangulates all the corner observations into 3D points and then jointly refines the camera parameters (and optionally the points) to minimize the total reprojection error, in rounds that progressively reject outlier points.
By default it optimizes the same compact camera model Anipose does — one focal length and k1 per camera, principal point pinned at the image centre — plus a term expressing that the corners seen in a frame belong to one rigid board. Richer models are available, but freeing every parameter tends to let distortion terms drift into large cancelling values when the board never reaches the image corners.
calibrat3 always re-triangulates all points after each round and reports the error over all observations, so the number you see is not flattered by the subset that was fitted. It also compares the refined result against the initial one and keeps whichever is better, so refinement cannot make your calibration worse.
5. Export¶
The result is written as calibration.toml in sleap-anipose format — the file LUC3D loads. calibrat3 can additionally export a full calibration_data.json (all observations, triangulated points, per-camera errors), the board.toml, and a resumable session file.
Judging a calibration¶
The number that matters is the cross-view reprojection error: triangulate each board corner from all views that saw it, project it back into every view, and measure the pixel distance to the detected corner. calibrat3 reports it as a median, p95, per-camera table, and per-frame plots.
| Reading | Interpretation |
|---|---|
| Median error | Typical accuracy. Compare against a reference calibration on the same detections rather than against an absolute target — achievable values depend on resolution, rig size, and board size |
| p95 error | How bad the tail is. A low median with a large p95 usually means a few cameras or frames dominate |
| Per-camera error | Isolates a single bad camera — often one with poor board coverage or a soft focus |
| Camera positions in the 3D view | The sanity check that catches gross errors: cameras should be where they physically are |
A calibration with a good median but implausible camera geometry is not usable — check the 3D viewport in LUC3D after loading it.
Using the calibration in LUC3D¶
Load it via File ▸ Load Calibration…, or place it in the session folder as calibration.toml (any filename containing calib) so File ▸ Load Single Session Folder picks it up automatically. LUC3D matches each camera's name field to your video filenames or camera subdirectories.
Once loaded, the calibration drives:
- Triangulation — DLT and per-point bundle adjustment both treat the cameras as fixed and known
- Reprojection error overlays — the primary signal for annotation quality
- Reprojection-aided labeling — label two views, get the rest for free
- The 3D viewport — camera frustums are drawn from the extrinsics
Recalibrate when the rig changes
A calibration is only valid for the rig geometry it was recorded with. If a camera is bumped, remounted, refocused, or its zoom changed, the extrinsics (and possibly intrinsics) are stale. Record calibration videos in the same session as your experimental recordings whenever possible.