Skip to content

Multi-Camera Setup

LUC3D works with multi-view recordings where the same scene is captured from multiple synchronized cameras. This page covers how to prepare your camera calibration data and organize your video files.

Camera Calibration Format

LUC3D accepts camera calibration in TOML or JSON format. The TOML format is compatible with Anipose and sleap-anipose calibration output.

Don't have a calibration yet?

Use calibrat3 (source), the companion browser app that computes a calibration.toml from ChArUco board videos — no installation. Choose Calibrate → Calibrate in New Tab in the LUC3D menu bar to open it, and follow Calibrating Cameras.

TOML Format

[cam_0]
name = "back"
size = [1280, 1024]
matrix = [[1000.0, 0.0, 640.0], [0.0, 1000.0, 512.0], [0.0, 0.0, 1.0]]
distortions = [0.0, 0.0, 0.0, 0.0, 0.0]
rotation = [0.1, -0.2, 0.05]
translation = [100.0, 50.0, 500.0]

[cam_1]
name = "side"
size = [1280, 1024]
matrix = [[1000.0, 0.0, 640.0], [0.0, 1000.0, 512.0], [0.0, 0.0, 1.0]]
distortions = [0.0, 0.0, 0.0, 0.0, 0.0]
rotation = [0.3, 0.1, -0.1]
translation = [-200.0, 50.0, 500.0]

Only sections named cam_* are read; a [metadata] section (as written by Anipose) is ignored.

Calibration Parameters

Parameter Shape Description
name string Camera name — used to match with video filenames or camera folders
size [width, height] Image resolution in pixels
matrix 3x3 Intrinsic camera matrix (focal length + principal point)
distortions [k1, k2, p1, p2, k3] Lens distortion coefficients (OpenCV model)
rotation [rx, ry, rz] or 3x3 Rotation as Rodrigues vector or rotation matrix
translation [tx, ty, tz] Translation vector

JSON Format

{
  "cameras": [
    {
      "name": "back",
      "size": [1280, 1024],
      "matrix": [[1000, 0, 640], [0, 1000, 512], [0, 0, 1]],
      "distortions": [0, 0, 0, 0, 0],
      "rotation": [0.1, -0.2, 0.05],
      "translation": [100, 50, 500]
    }
  ]
}

A bare array of cameras is also accepted, and the keys dist, rvec, and tvec are synonyms for distortions, rotation, and translation.

Folder Structure

File → Session Folder Wizard... lets you pick which layout File → Load Single Session Folder (Ctrl+O) expects. The choice is remembered in your browser; the default is Per-Camera SLP.

Each camera has its own folder containing a video and optionally an annotation file (.slp or .h5).

session/
├── calibration.toml          # Filename must contain "calib"
├── skeleton.json              # Optional: skeleton definition
├── back/                      # Camera folder (name matches calibration)
│   ├── video.mp4
│   └── back.slp               # Optional: SLEAP annotations
├── side/
│   ├── video.mp4
│   └── side.slp
└── top/
    └── video.mp4

One .slp at the session root covers all cameras, with videos in a videos/ subfolder.

session/
├── calibration.toml
├── skeleton.json              # Optional
├── labels.slp                 # Required: one .slp (or .h5) at the root
└── videos/
    ├── back.mp4
    ├── side.mp4
    └── top.mp4

Load with File → Load Multi-Session Folder (Chrome/Edge only). Each session subfolder follows the per-camera layout with its own calibration.

project/
├── skeleton.json              # Optional: applied to every session
├── session_01/
│   ├── calibration.toml
│   ├── back/
│   │   ├── video.mp4
│   │   └── back.slp
│   └── side/
│       └── video.mp4
└── session_02/
    ├── calibration.toml
    ├── back/
    │   └── video.mp4
    └── side/
        └── video.mp4

A subfolder counts as a session if it contains a file with calib in its name or any subdirectory.

File Discovery Rules (per-camera layout)

File Type Location Naming Rule
Calibration Session root .toml or .json whose filename contains calib
Skeleton Session root .json whose filename contains skeleton (optional; overrides a skeleton found in an .slp)
Video Camera subdirectory Any .mp4, .avi, .webm, .mov, or .mkv file (first file used)
Annotations Camera subdirectory Any .slp or .h5 file (highest version loaded if multiple)
Environment Camera subdirectory .slp/.h5 files with .externals. in the name (used by Set Env in the 3D viewport)

Files nested deeper than one level are ignored, as are calibration clips (calibration_images/ folders or files ending in _calibration).

Annotation versioning

If a camera folder contains multiple SLP files with version suffixes (cam1_v1.slp, cam1_v2.slp, or cam1_3D_v2.slp), LUC3D loads the highest version number; ties go to the most recently modified file. A warning is shown if a different file is newer on disk than the one chosen, or if versions differ between cameras. Exports bump the version automatically (see Exporting Data).

Camera Directory Matching

LUC3D matches camera subdirectory names to the name field in the calibration file. Matching is case-insensitive and tries, in order:

  1. Exact — the folder name equals the camera name (back/ → back)
  2. cam prefix — the folder is cam + the camera name (camback/ → back)
  3. Strip cam — removing the first cam from the folder name leaves the camera name (backcam/ → back)
Calibration name Matches Does not match
back back, Back, camback, CamBack, backcam Cam-back, cam_back (the separator is kept)

If a calibration camera has no matching folder, a Missing Camera Directories popup lets you import each camera's files manually or continue without them. Without a calibration file, every subfolder with a video loads as a plain view and 3D features are disabled.

Load all cameras as one unit

A session folder's cameras are loaded together: if any camera's prediction file is large enough to need the lazy loader (below), every camera is loaded lazily, so no view is silently left empty. Cameras that load with no labels are reported in the status bar.

Loading large files

LUC3D loads .slp and analysis .h5 files through sleap-io.js. For large files it automatically switches to a lazy, streaming loader so the browser tab doesn't run out of memory:

File Threshold Behavior above threshold
Prediction .slp > 150 MB Lazy-loaded — frames are materialized on demand, only a resident window is kept in memory
Analysis .h5 > 20 MB Lazy-loaded the same way
Saved project .slp > 200 MB Lazy-reopened; you are prompted to re-attach the videos

Smaller hand-labeled files use the normal eager path. This is automatic — there is no setting to toggle. Downstream operations on a large lazy session (Track All, Triangulate All, save, and per-camera export) are also memory-bounded and streamed, so you can process sessions with hundreds of thousands of instances.

Producing a Calibration

Tool Notes
calibrat3 Browser-based ChArUco calibration, no install. Exports sleap-anipose calibration.toml directly. See Calibrating Cameras
Anipose Its calibration.toml loads directly; both Rodrigues vector and rotation matrix forms are supported
OpenCV / MATLAB Usable, but you must convert the output to the TOML or JSON layout above

Tips

  • Name camera folders exactly like the name field in your calibration (or with a bare cam prefix)
  • Each camera folder needs exactly one video file
  • All videos in a session should have the same number of frames for synchronized playback
  • Ensure calibration was performed at the same resolution as your recording videos
  • Sessions without a calibration file can still load videos, but 3D features will be disabled