Skip to content
LUC3D Logo

LUC3D — Label Unification and Correspondence in 3D

LUC3D (pronounced lu·​cid) is a browser-based multi-view pose annotation GUI for 3D animal and human pose estimation. It runs entirely in the browser with no build system — pure vanilla JS served as static files.

Video Walkthrough

A captioned, 10-minute tour of a full project in LUC3D v1.0.5: defining the arena floor, building a skeleton, labeling and triangulating a mouse, tracking three mice across five cameras with Track All and reading its summary and automatic ID-switch check, and proofreading in the 3D viewport.

Key Features

  • Camera calibration with calibrat3 — Produce the calibration.toml LUC3D needs from ChArUco board videos, in the browser, with the companion app calibrat3
  • Arena plane & origin — Annotate the arena floor as a plane and re-base the 3D view on it, with your choice of origin and +Z direction
  • Multi-view synchronized video playback — Load videos from multiple camera angles and scrub through them in sync, with frame-accurate WebCodecs decoding
  • 2D pose annotation — Create and edit skeleton instances with keypoints directly on video frames, with a full inline skeleton editor
  • Cross-view tracker — Automatically group the same individual across views and assign consistent identities (adapted from the sleap-3d CrossViewTracker), for one frame, a frame range, or the whole recording
  • DLT & Refined triangulation — Reconstruct 3D poses from 2D annotations using camera calibration; the refined solver helps when one view's 2D point is clearly wrong
  • Temporal smoothing — Remove one-frame spikes and jitter from the 3D over time, with a median and a Gaussian window
  • Tracking Wizard — Tune the tracker with per-node weights, per-view inclusion, association thresholds, and stale-detection eviction
  • Reprojection-aided labeling — Label 2 views, get free annotations in all other views via 3D reprojection
  • Reprojection error visualization — Validate annotation quality with error vectors on the video and per-instance node × camera error tables
  • Interactive 3D viewport — Visualize triangulated skeletons in a Three.js scene with camera frustums; import 3D points and navigate without videos
  • Video export — Render camera views with poses burned in (Export Video Overlays) or the 3D view to MP4
  • Track and identity management — Per-session tracks and identities with Tracks↔IDs propagation and forward-propagating identity fixes
  • SLEAP-compatible I/O — Import predictions and skeletons from SLEAP and export annotations in native .slp, JSON, and HDF5 formats; large prediction files load lazily to stay within memory
  • Multi-session support — Work with multiple recording sessions in a single project
  • Editable keyboard shortcuts — Rebind any shortcut in Settings
  • Timeline widget — SLEAP-style timeline for navigating frames, tracks, and labeled regions

How It Works

Calibrate Cameras → Load Videos + Calibration → Annotate 2D Poses → Group / Track Across Views → Triangulate 3D → Export
  1. Calibrate your cameras with calibrat3 to get a calibration.toml (skip if you already have one)
  2. Load your multi-camera videos and camera calibration (TOML or JSON)
  3. Annotate 2D keypoints on each camera view
  4. Group corresponding instances across views by hand, or run the cross-view tracker to group and assign identities automatically
  5. Triangulate to reconstruct 3D points and verify with reprojection error
  6. Export as SLEAP .slp, JSON labels, HDF5 3D points, or MP4 videos with poses burned in

Camera Calibration

Every 3D feature depends on knowing where your cameras are. calibrat3 (source) is the companion browser app that turns ChArUco board videos into the calibration.toml LUC3D loads — choose Calibrate → Calibrate in New Tab in the LUC3D menu bar to open it.

Cameras & Calibration Boards

Which ChArUco board to buy (calib.io and other vendors), and how many cameras you need and where, measured on an 18-camera rig: see Choosing Cameras & Calibration Boards.

Arena & Origin

Put the 3D origin where you want it: annotate the arena floor as a plane in Define Planes mode, then pick its centre as (0, 0, 0) and which way +Z points. See Defining the Arena and Setting the Origin.

Tracking & Triangulation

Curious how LUC3D turns 2D labels into 3D? Learn how subjects are linked across views and how DLT and the Refined solver reconstruct each keypoint.

Temporal Smoothing

See what the median and Gaussian windows do to the 3D over time.

Building a Rig

Putting together cameras for 3D? What to Buy lists the parts, with what the lab's rigs use and what they cost, and Tested Rigs & Datasets describes the rigs LUC3D was developed and tested on: 1 to 18 cameras, 30 to 150 fps, 1 to 5 mice.

Requirements

  • Chrome or Edge recommended — required for Load Multi-Session Folder and for streaming large saves to disk (File System Access API), and the most reliable for frame-accurate seeking via WebCodecs. Other browsers work for single-session annotation with an automatic HTML5 video fallback
  • Nothing to install — use luc3d.sleap.ai, or serve the repo with Python 3 (or any static file server)