LUC3D — Label Unification and Correspondence in 3D

LUC3D (pronounced lu·​cid) is a browser-based multi-view pose annotation GUI for 3D animal and human pose estimation. It runs entirely in the browser with no build system — pure vanilla JS served as static files.

Key Features

• Camera calibration with calibrat3 — Produce the calibration.toml LUC3D needs from ChArUco board videos, in the browser, with the companion app calibrat3
• Multi-view synchronized video playback — Load videos from multiple camera angles and scrub through them in sync, with frame-accurate WebCodecs decoding
• 2D pose annotation — Create and edit skeleton instances with keypoints directly on video frames, with a full inline skeleton editor
• Cross-view tracker — Automatically group the same individual across views and assign consistent identities (a port of the sleap-3d CrossViewTracker)
• DLT & Bundle Adjustment triangulation — Reconstruct 3D poses from 2D annotations using camera calibration, with optional robust outlier rejection
• Tracking Wizard — Tune the tracker with per-node weights, per-view inclusion, and cost thresholds
• Reprojection-aided labeling — Label 2 views, get free annotations in all other views via 3D reprojection
• Reprojection error visualization — Validate annotation quality with per-keypoint error overlays
• Interactive 3D viewport — Visualize triangulated skeletons in a Three.js scene with camera frustums; import 3D points and navigate without videos; export the 3D view to MP4
• Track and identity management — Per-session tracks and identities with Tracks↔IDs propagation
• SLEAP-compatible I/O — Import predictions from SLEAP and export annotations in native .slp, JSON, and HDF5 formats; large prediction files load lazily to stay within memory
• Multi-session support — Work with multiple recording sessions in a single project
• Editable keyboard shortcuts — Rebind any shortcut in Settings
• Timeline widget — SLEAP-style timeline for navigating frames, tracks, and labeled regions

How It Works

Calibrate Cameras → Load Videos + Calibration → Annotate 2D Poses → Group / Track Across Views → Triangulate 3D → Export

1. Calibrate your cameras with calibrat3 to get a calibration.toml (skip if you already have one)
2. Load your multi-camera videos and camera calibration (TOML or JSON)
3. Annotate 2D keypoints on each camera view
4. Group corresponding instances across views by hand, or run the cross-view tracker to group and assign identities automatically
5. Triangulate to reconstruct 3D points and verify with reprojection error
6. Export as SLEAP .slp, JSON labels, HDF5 3D points, or an MP4 of the 3D view

Camera Calibration

Every 3D feature depends on knowing where your cameras are. calibrat3 (source) is the companion browser app that turns ChArUco board videos into the calibration.toml LUC3D loads — click Calibrate in the LUC3D menu bar to open it.

• How camera calibration works — what the file contains and how calibrat3 computes it
• Calibrating Cameras — step-by-step, from recording the board to verifying the result in LUC3D

Tracking & Triangulation

Curious how LUC3D turns 2D labels into 3D? Learn how subjects are linked across views and how DLT and Bundle Adjustment reconstruct each keypoint.

Quick Links

• Installation — Get LUC3D running locally
• Calibrating Cameras — Produce a calibration.toml with calibrat3
• Quickstart — End-to-end walkthrough in 5 minutes
• Loading a Session — Load your first recording session
• Annotating Poses — Create and edit pose annotations
• Camera Calibration — What a calibration is and how it is computed
• Tracking & Triangulation — How DLT and Bundle Adjustment reconstruct 3D
• Keyboard Shortcuts — Full shortcut reference

Requirements

• A modern browser — a recent Chromium-based browser (Chrome/Edge) is recommended for the most reliable frame-accurate video seeking (via WebCodecs), with an automatic HTML5 fallback otherwise
• Python 3 for local serving (or any static file server)
• No installation or build step required