LUC3D — Label Unification and Correspondence in 3D¶
LUC3D (pronounced lu·cid) is a browser-based multi-view pose annotation GUI for 3D animal and human pose estimation. It runs entirely in the browser with no build system — pure vanilla JS served as static files.
Video Walkthrough¶
A captioned, 10-minute tour of a full project in LUC3D v1.0.5: defining the arena floor, building a skeleton, labeling and triangulating a mouse, tracking three mice across five cameras with Track All and reading its summary and automatic ID-switch check, and proofreading in the 3D viewport.
Key Features¶
- Camera calibration with calibrat3 — Produce the
calibration.tomlLUC3D needs from ChArUco board videos, in the browser, with the companion app calibrat3 - Arena plane & origin — Annotate the arena floor as a plane and re-base the 3D view on it, with your choice of origin and +Z direction
- Multi-view synchronized video playback — Load videos from multiple camera angles and scrub through them in sync, with frame-accurate WebCodecs decoding
- 2D pose annotation — Create and edit skeleton instances with keypoints directly on video frames, with a full inline skeleton editor
- Cross-view tracker — Automatically group the same individual across views and assign consistent identities (adapted from the
sleap-3dCrossViewTracker), for one frame, a frame range, or the whole recording - DLT & Refined triangulation — Reconstruct 3D poses from 2D annotations using camera calibration; the refined solver helps when one view's 2D point is clearly wrong
- Temporal smoothing — Remove one-frame spikes and jitter from the 3D over time, with a median and a Gaussian window
- Tracking Wizard — Tune the tracker with per-node weights, per-view inclusion, association thresholds, and stale-detection eviction
- Reprojection-aided labeling — Label 2 views, get free annotations in all other views via 3D reprojection
- Reprojection error visualization — Validate annotation quality with error vectors on the video and per-instance node × camera error tables
- Interactive 3D viewport — Visualize triangulated skeletons in a Three.js scene with camera frustums; import 3D points and navigate without videos
- Video export — Render camera views with poses burned in (Export Video Overlays) or the 3D view to MP4
- Track and identity management — Per-session tracks and identities with Tracks↔IDs propagation and forward-propagating identity fixes
- SLEAP-compatible I/O — Import predictions and skeletons from SLEAP and export annotations in native
.slp, JSON, and HDF5 formats; large prediction files load lazily to stay within memory - Multi-session support — Work with multiple recording sessions in a single project
- Editable keyboard shortcuts — Rebind any shortcut in Settings
- Timeline widget — SLEAP-style timeline for navigating frames, tracks, and labeled regions
How It Works¶
Calibrate Cameras → Load Videos + Calibration → Annotate 2D Poses → Group / Track Across Views → Triangulate 3D → Export
- Calibrate your cameras with calibrat3 to get a
calibration.toml(skip if you already have one) - Load your multi-camera videos and camera calibration (TOML or JSON)
- Annotate 2D keypoints on each camera view
- Group corresponding instances across views by hand, or run the cross-view tracker to group and assign identities automatically
- Triangulate to reconstruct 3D points and verify with reprojection error
- Export as SLEAP
.slp, JSON labels, HDF5 3D points, or MP4 videos with poses burned in
Camera Calibration¶
Every 3D feature depends on knowing where your cameras are. calibrat3 (source) is the companion browser app that turns ChArUco board videos into the calibration.toml LUC3D loads — choose Calibrate → Calibrate in New Tab in the LUC3D menu bar to open it.
- How camera calibration works — what the file contains and how calibrat3 computes it
- Calibrating Cameras — step-by-step, from recording the board to verifying the result in LUC3D
Cameras & Calibration Boards¶
Which ChArUco board to buy (calib.io and other vendors), and how many cameras you need and where, measured on an 18-camera rig: see Choosing Cameras & Calibration Boards.
Arena & Origin¶
Put the 3D origin where you want it: annotate the arena floor as a plane in Define Planes mode, then pick its centre as (0, 0, 0) and which way +Z points. See Defining the Arena and Setting the Origin.
Tracking & Triangulation¶
Curious how LUC3D turns 2D labels into 3D? Learn how subjects are linked across views and how DLT and the Refined solver reconstruct each keypoint.
Temporal Smoothing¶
See what the median and Gaussian windows do to the 3D over time.
Building a Rig¶
Putting together cameras for 3D? What to Buy lists the parts, with what the lab's rigs use and what they cost, and Tested Rigs & Datasets describes the rigs LUC3D was developed and tested on: 1 to 18 cameras, 30 to 150 fps, 1 to 5 mice.
Quick Links¶
- Video Walkthrough — A 10-minute captioned tour of a full project
- Installation — Get LUC3D running locally
- Calibrating Cameras — Produce a
calibration.tomlwith calibrat3 - Cameras & Calibration Boards — Which board to buy, and which camera positions matter most
- Quickstart — End-to-end walkthrough in 5 minutes
- Defining the Arena & Origin — Annotate the floor as a plane and set the 3D origin and +Z
- Loading a Session — Load your first recording session
- Annotating Poses — Create and edit pose annotations
- Camera Calibration — What a calibration is and how it is computed
- Tracking & Triangulation — How DLT and the Refined solver reconstruct 3D
- Temporal Smoothing — What the median and Gaussian windows do
- Keyboard Shortcuts — Full shortcut reference
- What to Buy — Cameras, lenses, sync, cables, computers and lights for a multi-camera rig
- Tested Rigs & Datasets — The rigs LUC3D was developed and tested on
Requirements¶
- Chrome or Edge recommended — required for Load Multi-Session Folder and for streaming large saves to disk (File System Access API), and the most reliable for frame-accurate seeking via WebCodecs. Other browsers work for single-session annotation with an automatic HTML5 video fallback
- Nothing to install — use luc3d.sleap.ai, or serve the repo with Python 3 (or any static file server)