Tutorial: Calibrating Cameras with calibrat3¶

LUC3D needs a camera calibration before it can triangulate anything. This tutorial walks through producing one with calibrat3 and loading it into LUC3D.


App	talmolab.github.io/calibrat3 — runs in the browser, nothing to install
Source	github.com/talmolab/calibrat3
Shortcut	From LUC3D, click Calibrate in the menu bar to open it in a new tab

For the concepts behind each stage, see Camera Calibration.

Try it without your own data

calibrat3 ships with a 4-camera sample session. Open the app, click Load sample session, and run the whole pipeline in about a minute to see what each stage produces.

Before you start: recording the videos¶

The calibration is only as good as the footage. Record synchronized videos of a ChArUco board with every camera at once, and while recording:

Move the board slowly through the whole working volume — the region where your animals will actually be — not just the middle.
Tilt and rotate it. A board that stays parallel to one camera gives that camera almost no information about focal length.
Reach the image corners of each view. Distortion is estimated from the periphery; a board that only ever appears in the center makes distortion coefficients unreliable.
Make sure every pair of neighboring cameras sees the board at the same time at some point. Cameras are related to each other through shared views of the board.
Keep it in focus and unblurred. Slow, deliberate motion beats a long recording of a fast-moving board.
Don't change zoom or focus afterward — that invalidates the intrinsics.

A minute or two of video per camera is usually plenty. Note the board's square size and marker size in millimeters (measure the printed board, don't trust the design file) — this sets the physical scale of every 3D coordinate LUC3D reports.

Organize the videos in one of the two layouts calibrat3 accepts:

# flat                          # anipose-style
session/                        session/
├── back.mp4                    ├── back/
├── side.mp4                    │   └── calibration_images/
├── top.mp4                     │       └── back.mp4
└── board.toml   (optional)     ├── side/
                                │   └── calibration_images/
                                │       └── side.mp4
                                └── board.toml   (optional)

A board.toml next to the videos pre-fills the board form in step 2.

Step 1 — Load videos¶

Open calibrat3 in Chrome or Edge, then in 1 Load videos choose:

Open session folder… — pick the folder holding your videos, or
Load sample session — the bundled 4-camera example, or
Load saved session… — resume a previous .calibrat3.json

The views appear in a grid. Scrub through and confirm all cameras are in sync and the board is visible.

Use a Chromium browser

calibrat3 relies on WebCodecs and the File System Access API. Chrome and Edge are fully supported; Firefox works only for the sample session and the folder-input fallback.

Step 2 — Detect ChArUco corners¶

Fill in the Board panel to match your printed board exactly:

Field	Meaning
Squares X / Squares Y	Number of chessboard squares across and down
Square (mm)	Side length of one chessboard square — sets the units of your 3D coordinates
Marker (mm)	Side length of the ArUco marker inside a square
Dictionary	The ArUco dictionary the board was generated from (e.g. `4x4 (50)`)

Verify before committing to a long run:

Navigate to a frame where the board is clearly visible.
Click Detect current frame (D). Detected corners appear as an overlay with a corner count.
If nothing is found, the board parameters or dictionary are wrong — fix them before continuing.

Then run the batch:

Set Target samples (default 100) — how many frames to sample across the videos. A few hundred well-spread frames beat every frame of a short clip; the displayed Stride shows the resulting spacing. Tick every frame only for short recordings.
Click Run batch detection and watch the progress bar. The frame strip and the table below fill in as results arrive.
Review the Detections table and strip: you want most frames detected in most views, and enough frames where pairs of cameras both see the board. Set Min common to check how many frames have shared corners.

If coverage is thin, increase Target samples and re-run rather than pushing on.

Step 3 — Compute intrinsics¶

In 3 Intrinsics:

Leave Distortion model at k1 only (anipose-style) unless you have a strongly distorting lens and board coverage that reaches the image corners.
Max frames / camera (default 50) caps how many frames the fit uses — the solver's cost grows steeply with frame count, and frames are chosen for image coverage. Every valid frame is still evaluated afterward.
Click Compute intrinsics.

Then inspect the results:

The swarm plot shows per-frame reprojection error for each camera. Look for outliers far above the cloud.
The worst-frames gallery shows the offenders as thumbnails. A blurred, badly-lit, or misdetected board here is worth removing.
Press X on a frame (or click ✕ on a gallery card) to exclude it, then Compute intrinsics again.

Excluding a handful of genuinely bad frames helps. Excluding frames just because their error is high does not — you'd be fitting to the frames that happen to agree.

Step 4 — Extrinsics and bundle adjustment¶

In 4 Extrinsics & bundle adjustment:

Pick a Reference camera. It becomes the world origin; every 3D coordinate in LUC3D will be expressed relative to it. Choose a central, stable camera with a good view.
Click Compute extrinsics. calibrat3 builds the covisibility graph, solves relative poses, and chains them out from the reference camera. It then triangulates all shared corners and reports the cross-view reprojection error — median, p95, per-camera table, and per-frame plots.
Click Refine (bundle adjustment). The defaults follow Anipose — focal + k1 intrinsics model, principal point at the image centre, aggressive outlier rejection over 6 rounds, no robust loss — and are a good starting point. Progress, a live cost chart, and a before/after comparison appear as it runs.

The refined section shows initial vs refined error histograms and per-camera stats. If the refinement didn't help, Revert to initial restores the pre-BA calibration.

What error should I expect?

There is no universal target — it depends on resolution, rig size, and board size. Judge a change by comparing the median error against a reference calibration computed on the same detections, and sanity-check the camera geometry in LUC3D's 3D viewport.

Step 5 — Export¶

In 5 Export, click Download calibration.toml. That's the file LUC3D reads.

Also worth saving:

File	Use
`calibration.toml`	Required — load this into LUC3D
`board.toml`	Reuse the board configuration next time
Session (`.calibrat3.json`)	Resume in calibrat3 later without re-detecting
`calibration_data.json`	Full observations, triangulated points, per-camera errors — for analysis

Step 6 — Load the calibration into LUC3D¶

Two paths:

As part of a session folder (recommended). Put the file in the session root — any filename containing calib works — next to per-camera subdirectories:

session/
├── calibration.toml
├── back/
│   └── video.mp4
├── side/
│   └── video.mp4
└── top/
    └── video.mp4

Then File ▸ Load Single Session Folder picks up videos and calibration together. See Loading a Session.

Manually. Load videos with File ▸ Load Videos…, then File ▸ Load Calibration… and select the calibration.toml. See Loading Videos & Linking Calibrations.

Camera names must match

LUC3D matches the name field of each camera in the TOML to your video filenames or camera subdirectory names (case-insensitive, cam prefixes stripped). calibrat3 takes the names from your video filenames, so this normally lines up — but if you renamed videos in between, you'll need to assign cameras manually in the Session tab.

Step 7 — Verify in LUC3D¶

Before annotating, confirm the calibration is sane:

Open the Cameras tab — resolutions should match your videos, focal lengths and principal points should look reasonable.
Press \ for the 3D viewport. Camera frustums should sit roughly where the cameras physically are, all pointing at a common volume. Cameras stacked at the origin mean extrinsics failed to load.
Label the same keypoint in two or three views, group the instances, and triangulate. The reprojection error should be small and the reprojected point should land on the subject in the other views.

If reprojections are consistently offset in one view, that camera's calibration is the problem — go back to calibrat3 and check its per-camera error and board coverage.

Troubleshooting¶

Symptom	Likely cause
No corners detected at all	Wrong dictionary or wrong Squares X/Y — verify with Detect current frame (D)
One camera has much higher error	Poor board coverage or focus in that view — record more footage covering it
Extrinsics fail or a camera is missing	That camera never shared enough board views with the rest; lower Min covisible corners or re-record
3D coordinates are the wrong scale	Square (mm) doesn't match the printed board
Cameras all at the origin in LUC3D's 3D view	Calibration loaded without extrinsics — check the TOML has `rotation` and `translation`
Reprojections drift after some time	The rig moved; recalibrate