Multi-Camera Setup¶

LUC3D works with multi-view recordings where the same scene is captured from multiple synchronized cameras. This page covers how to prepare your camera calibration data and organize your video files.

Camera Calibration Format¶

LUC3D accepts camera calibration in TOML or JSON format. The TOML format is compatible with Anipose and sleap-anipose calibration output.

Don't have a calibration yet?

Use calibrat3 (source), the companion browser app that computes a calibration.toml from ChArUco board videos — no installation. Click Calibrate in the LUC3D menu bar to open it, and follow Calibrating Cameras.

TOML Format¶

[cam_0]
name = "back"
size = [1280, 1024]
matrix = [[1000.0, 0.0, 640.0], [0.0, 1000.0, 512.0], [0.0, 0.0, 1.0]]
distortions = [0.0, 0.0, 0.0, 0.0, 0.0]
rotation = [0.1, -0.2, 0.05]
translation = [100.0, 50.0, 500.0]

[cam_1]
name = "side"
size = [1280, 1024]
matrix = [[1000.0, 0.0, 640.0], [0.0, 1000.0, 512.0], [0.0, 0.0, 1.0]]
distortions = [0.0, 0.0, 0.0, 0.0, 0.0]
rotation = [0.3, 0.1, -0.1]
translation = [-200.0, 50.0, 500.0]

Calibration Parameters¶

Parameter	Shape	Description
`name`	string	Camera name — used to match with video filenames
`size`	[width, height]	Image resolution in pixels
`matrix`	3x3	Intrinsic camera matrix (focal length + principal point)
`distortions`	[k1, k2, p1, p2, k3]	Lens distortion coefficients (OpenCV model)
`rotation`	[rx, ry, rz] or 3x3	Rotation as Rodrigues vector or rotation matrix
`translation`	[tx, ty, tz]	Translation vector (camera position in world coordinates)

JSON Format¶

{
  "cameras": [
    {
      "name": "back",
      "size": [1280, 1024],
      "matrix": [[1000, 0, 640], [0, 1000, 512], [0, 0, 1]],
      "distortions": [0, 0, 0, 0, 0],
      "rotation": [0.1, -0.2, 0.05],
      "translation": [100, 50, 500]
    }
  ]
}

Folder Structure¶

LUC3D expects a per-camera subdirectory structure. Each camera has its own folder containing a video file and optionally an annotation file (.slp or .h5).

Single Session¶

session/
├── calibration.toml          # Camera calibration (filename must contain "calib")
├── skeleton.json              # Optional: skeleton definition
├── back/                      # Camera subdirectory (name matches calibration)
│   ├── video.mp4              # Video file for this camera
│   └── back.slp               # Optional: SLEAP annotations
├── side/
│   ├── video.mp4
│   └── side.slp
├── top/
│   └── video.mp4
└── front/
    └── video.mp4

Multi-Session¶

project/
├── session_01/
│   ├── calibration.toml       # Each session has its own calibration
│   ├── back/
│   │   ├── video.mp4
│   │   └── back.slp
│   ├── side/
│   │   └── video.mp4
│   └── top/
│       └── video.mp4
└── session_02/
    ├── calibration.toml
    ├── back/
    │   └── video.mp4
    ├── side/
    │   └── video.mp4
    └── top/
        └── video.mp4

File Discovery Rules¶

File Type	Location	Naming Rule
Calibration	Session root	Filename must contain `calib` (e.g., `calibration.toml`, `my_calib.json`)
Skeleton	Session root	Filename must contain `skeleton` and end in `.json` (optional)
Video	Camera subdirectory	Any `.mp4`, `.avi`, `.webm`, `.mov`, or `.mkv` file (first file used)
Annotations	Camera subdirectory	Any `.slp` or `.h5` file (highest version loaded if multiple)
Environment	Camera subdirectory	`.slp`/`.h5` files with `.externals.` in the name

Annotation versioning

If a camera folder contains multiple SLP files with version suffixes (e.g., cam1_v1.slp, cam1_v2.slp, cam1_v3.slp), LUC3D automatically loads the highest version number.

Camera Directory Matching¶

LUC3D matches camera subdirectory names to the name field in the calibration file using these rules (case-insensitive):

Calibration `name`	Matching Directory Names
`back`	`back`, `Cam-back`, `camback`, `cam_back`
`side`	`side`, `Cam-side`, `camside`, `cam_side`
`top`	`top`, `Cam-top`, `camtop`, `cam_top`

The matching logic strips cam prefixes and is case-insensitive. If auto-matching fails, a popup lets you import missing files manually.

Loading large files¶

LUC3D loads .slp and analysis .h5 files through sleap-io.js. For large prediction files it automatically switches to a lazy, streaming loader so the browser tab doesn't run out of memory:

File	Threshold	Behavior above threshold
Prediction `.slp`	> 150 MB	Lazy-loaded — frames are materialized on demand, only a resident window is kept in memory
Analysis `.h5`	> 20 MB	Lazy-loaded the same way

Smaller hand-labeled files use the normal eager path (which keeps the full grouped-instance display). This is automatic — there is no setting to toggle. Downstream operations on a large lazy session (triangulate-all, save, and per-camera export) are also memory-bounded and streamed, so you can process sessions with hundreds of thousands of instances without exhausting memory.

Producing a Calibration¶

Tool	Notes
calibrat3	Browser-based ChArUco calibration, no install. Exports sleap-anipose `calibration.toml` directly. See Calibrating Cameras
Anipose	Its `calibration.toml` loads directly; both Rodrigues vector and rotation matrix forms are supported
OpenCV / MATLAB	Usable, but you must convert the output to the TOML or JSON layout above

Tips¶

Camera subdirectory names should match the name field in your calibration file
Each camera folder needs exactly one video file
All videos in a session should have the same number of frames for synchronized playback
Ensure calibration was performed with the same resolution as your recording videos
Sessions without a calibration file can still load videos, but 3D features will be disabled