Supplementary Video 1. SLAP-2M. Flythrough of a home-cage recording, showing the enclosure and its cameras rendered in 3D, four mice tracked with per-animal identity colors, and the same skeletons projected onto one camera’s footage. 13 s, silent.
Annotate and proofread 3D pose in the browser, with no installation
Animal behavior unfolds in three dimensions over time, yet accessible tools for analyzing it in that native form remain scarce. Advances in 2D pose estimation have made it far easier to track behavior over time, but a single view leaves genuinely ambiguous estimates — depth is unobservable, and animals occlude each other. Multiple calibrated cameras resolve that ambiguity, at the cost of a harder problem: the same animal must be given the same identity in every view before anything can be triangulated.
LUC3D — the Label Unification and Correspondence in 3D Annotation GUI — is browser-based, reduces the manual labor of labeling, proofreading and multi-animal tracking, and requires no installation. Alongside it we release two comprehensive datasets of proofread multi-camera 3D pose during multi-animal mouse social behavior, a protocol for generating 3D-consistent labels, and a cross-view re-identification algorithm for proofreading multi-camera multi-animal pose.
Try it now
The GUI runs entirely in your browser — nothing to install, and your data never leaves
your machine. Launch LUC3D, or read the
documentation first. It reads SLEAP .slp
files and exports SLP 2.8 or HDF5.
Multi-camera work starts with a camera calibration.
calibrat3
(source) is the companion browser tool that
computes one from ChArUco board videos and exports the calibration.toml LUC3D reads —
also reachable from Calibrate in the LUC3D menu bar.
Recording new data? Panopticon is the acquisition app at the front of the pipeline: it hardware-triggers N Basler cameras off one clock, encodes to H.264 on the GPU during capture, and writes each session out frame-aligned in the folder structure LUC3D reads, with live calibration coverage and an on-board optogenetic stimulation editor.
Overview
Figure 1. LUC3D enables annotation and proofreading multi-camera 3D pose in a
browser with no installation. A, Cross-view association and triangulation on
one frame of an 8-camera, 3-mouse recording with a 15-node skeleton. The two columns
are the same two views, cam 0 mid and cam 7 sideR. Top, the per-camera tracks
supplied to the app, each labeled with the track name its own camera assigned. Track
colors are arbitrary, so the same animal is a different color and a different track
name in the two views. Middle, the same two views after association, each labeled
with the resolved identity. Identity colors are shared by every view, so each animal
keeps one color and one number in both views. Bottom left, a render of the
triangulated 3D poses. Bottom right, the 3D poses reprojected into cam 0 mid,
magnified. The lines follow one animal, ID 1. Dashed arrows point from its two
tracks to its identity, solid lines join its identity in each view to its 3D pose,
and the dotted arrow points to its reprojection. B, The two datasets’ arenas
with their camera rigs and poses. Top, the SLAP-2M rig with 8 calibrated cameras and
the cage with the tracked 3D poses of four animals. Bottom, the MouseDyad-10M rig
with the 5 calibrated cameras and enclosure with tracked 3D poses of the two
animals. C, Pipeline of a standard recording and labeling session. Videos from N
synchronized cameras are acquired with Panopticon. A 2D pose model (SLEAP or any
other predictor, as .slp) predicts keypoints in every view, and the cameras are
calibrated with Calibrat3, a browser-based tool. The LUC3D app then associates
detections across views into one identity per animal, triangulates them from two or
more views, and proofreads the 3D poses against per-view reprojections. The output
is a 3D pose for each animal, saved as .slp or HDF5. Green outlines mark the stages
contributed by this work.
Full resolution (PDF)
Cross-view tracking: one identity per animal, in every view
The published approach enumerates every way detections could be grouped into identities and keeps the lowest reprojection error — (A!)^C hypotheses per frame, which becomes intractable exactly where multi-animal work gets interesting. LUC3D commits one Hungarian assignment per camera instead, at O(C·A³). With a 3D correspondence term it resolves identity across all cameras at up to 14,694 frames per second. On MouseDyad-10M it keeps identity switches below one per 100,000 camera frames and reaches a mean cross-view IDF1 of 0.971, and over the multi-animal SLAP-2M sessions its mean cross-view IDF1 is 0.932.
Figure 2. Cross-view association is fast and accurate. A, The tracker’s 2D and 3D correspondence terms on one frame. Top, the 2D term (black dashed line) is the image distance between the current detection and that camera’s retained 2D anchor (dashed translucent blue pose). Bottom, the 3D term (dotted black line) is the distance between the 3D anchor node and the ray from the camera through the current node. B, The possible groupings for three animals seen by two cameras. Correct groupings are in color and incorrect ones in gray. C, LUC3D’s association in one frame. The 3D anchors are frozen at the start of the frame, and detections older than N frames are dropped. Each camera’s detections are then assigned to the anchors with the Hungarian algorithm, which grows as C·A³ for A animals and C cameras. Gray lines are scored pairings and colored lines kept pairings. Each anchor is re-triangulated once at the end of the frame. D, Groupings that exhaustive enumeration (Maree et al., 2024) must test per frame, which grow as (A!)^C. The dotted line marks 10⁶, above which it was not run. E, An identity switch. The two identities are exchanged in the top camera and correct in the side camera. A line that changes color marks one animal given two identities. The 3D poses are LUC3D’s reconstruction of the HardFight recording, and the exchange is drawn for illustration. F, Percentage of frames grouped incorrectly in each session, compared with proofread ground truth. Both medians are below 0.01%. A few outlying sessions are not drawn (up to 4.1% for exhaustive enumeration and 3.1% for LUC3D). G, Time per frame on the same detections for each combination of animals and cameras. At four animals in six cameras exhaustive enumeration was not run, and the value shown is a lower bound. H, Identity switches per 100,000 camera frames as the weight r of the 3D term relative to the 2D term is varied, with exhaustive enumeration as a reference. I, Cross-view IDF1 over the same range. F and G use the ~9.9 million frames of 97 sessions (55 MouseDyad-10M and 42 SLAP-2M) in which every camera holds one detection per animal. H and I use the 55 MouseDyad-10M sessions. A and B are frames of SLAP-2M sessions. Box plots show the median and interquartile range, with whiskers at 1.5 times the interquartile range. Frame sampling and timing are described in the Methods. Full resolution (PDF)
Labeling and triangulation: let the geometry do the work
Labeling every view of every frame is what makes multi-camera annotation expensive. Label two views, solve the 3D from those two alone, and the reprojection into the remaining views is already close enough to accept — so the annotator’s job becomes checking rather than clicking. The same two-view solve is also where a full reconstruction’s accuracy comes from: once every view is labeled, what sets the 3D error is how many views contribute and whether a badly fitting one gets dropped, not which solver does the triangulating. LUC3D’s linear and refined solvers match Anipose’s accuracy and are 32 and 4.9 times faster per keypoint than the matching Anipose solvers (11 times for the linear path including lens undistortion). The linear solver takes 0.89 µs per keypoint, and LUC3D triangulates up to 16,030 frames per second.
Figure 3. Reprojection from two labeled views reduces manual labeling, and LUC3D triangulates as accurately as existing tools and faster. A, Reprojection-aided labeling on one frame of an eight-camera recording. 1) the user labels the keypoints in two views. 2) LUC3D triangulates each keypoint from these two views. 3) reprojects it into the other views. 4) The user accepts each reprojection or moves it onto the animal. B, Keypoints placed by hand per animal as cameras are added to the rig, when every keypoint is labeled by hand and when reprojections are used. Reprojections are accepted within 10 pixels (solid) or corrected to within 5 pixels (dashed) of the keypoint’s true position. The lines are a model, and the filled marker is the measured five-camera rig. C, Distance of the 3D keypoints from the proofread reconstruction as cameras are added to the solve. Each dot is the median of one session. The five-camera value measures the size of the proofreading corrections. D, Two view 3D error against the angle between the two cameras, measured at the animal. Each marker is one camera pair. The dashed curve falls as 1/sin θ, with a band of ±25%, and the dotted line is the error when all five cameras are used. E, Top, reprojection error in a camera left out of the solve, as cameras are added, for the linear (DLT) and refined LUC3D solvers. Bottom, reprojection error in the cameras that were used, with all cameras and with each solver’s worst camera dropped. Error bars, 95% confidence intervals of the median (top) and the mean (bottom). F, Median reprojection error of each session, in the cameras used for the solve, for two LUC3D and two Anipose solvers. The LUC3D refined solver has the lowest error in all 55 sessions. Box plots show the median and interquartile range, with whiskers at 1.5 times the interquartile range. G, Time to triangulate one keypoint with each solver, measured on the same 200,010 keypoints for every solver. Bars are medians over timed passes and whiskers the interquartile range. A shows one frame of the HardFight recording. B to G use MouseDyad-10M. Full resolution (PDF)
Datasets
Two proofread multi-camera corpora ship with this work, 129 sessions and over 65 million video frames between them. Both are on Hugging Face: SLAP-2M (74 sessions, 8 cameras, 1 to 4 animals) and MouseDyad-10M (55 sessions, 5 cameras, 2 animals, 150 fps).
Figure 4. Tracking difficulty in SLAP-2M follows the level of occlusion. A, SLAP-2M sessions arranged by the number of animals and the amount of environmental enrichment, which together set each session’s difficulty rating. Each cage is one session, drawn from its own cage corners and tracked 3D poses, with the front wall shown as an outline so that the interior is visible. The inset shows one instant of the four-animal session in its six proofread cameras (scale bars, 50 mm). B, Percentage of keypoints that cannot be triangulated, because proofreaders could label them in no camera (purple) or in only one camera (orange), by difficulty rating. C, Reprojection error of the proofread 2D keypoints by difficulty rating. The solid line is the mean, the dotted line is the 80th percentile, and the dashed lines are the 95th and 98th percentiles. D, Cross-view IDF1 of the LUC3D tracker by difficulty rating, one point per multi-animal session. Rating 1 contains only single-animal sessions. Means in B and C are shown ± s.e.m. across sessions. Box plots show the median and interquartile range, with whiskers at 1.5 times the interquartile range. B and C use the proofread keypoints of all 74 sessions, tail keypoints excluded. B counts ~30.9 million 3D keypoints, and C scores their ~161.9 million 2D labels across the cameras. D uses the 42 multi-animal sessions, tracked from SLEAP predictions as in Figure 2 (Methods). Full resolution (PDF)
Supplementary Table 1. Dataset datasheets. Composition, metadata and per-session file inventory of the two datasets: SLAP-2M and MouseDyad-10M. Full resolution (PDF)
Supplementary Video 3. SLAP-2M difficulty. Each recorded combination of animal count (one to four) and environmental enrichment rating, one SLAP-2M session each, rendered in 3D from its own cage geometry and tracked poses. Played at 2× speed. 63 s, silent.
What 3D buys you: a behavior only the reconstruction can see
Every camera in the MouseDyad-10M rig sits 58 to 76 degrees above the animals, so the height that defines a rearing display exists only after triangulation. Proofread 3D turns that into a measurable social event.
Figure 5. Male-female pairs rear together face to face, and the female usually leads. A, One social rearing display, shown as the 3D reconstruction projected into each of the five cameras. All cameras look down on the arena from 58 to 76 degrees, so the animals’ height can only be measured in 3D. B, Nose height of the male (blue) and the female (red), and the distance between their noses, around the start of a display. Lines are the median across sessions and bands the interquartile range. C, Delay between the female’s and the male’s rear onset in the 398 displays led by the female. The striped bar collects the 9% of delays longer than 2 s. D, Speed of each animal in the 0.5 s before a display, in body lengths per second. The male is faster (paired Wilcoxon test, P = 6.3 × 10⁻⁴). E, Pursuit score before a display, which is negative when an animal faces its partner and moves toward it and positive when it moves away. The male’s score is lower in all 36 sessions (paired Wilcoxon test, P = 2.9 × 10⁻¹¹). F, Share of displays started by each sex in the 22 sessions with at least six displays (paired Wilcoxon test, P = 4.8 × 10⁻⁷). The dashed line marks equal shares. G, Probability that one animal is rearing around the other’s rear onset, relative to its own chance rate. Left, the male around female onsets. Right, the female around male onsets. The main lines are onsets with the animals within two body lengths, yellow lines onsets with the animals further apart, and gray lines a shuffled control. The box plots in D to F summarize one value per session (36 sessions in D and E). They show the median and interquartile range, with whiskers at 1.5 times the interquartile range. Data are from MouseDyad-10M, nine mice (four males and five females) in 18 pairings, recorded with five cameras at 150 fps. A display is counted when both animals rear, with the neck above 0.75 of the body length, and their tail bases stay within two body lengths for at least 0.25 s. There are 497 displays in 36 of the 55 sessions. G uses 9,073 rear onsets from all 55 sessions, 2,688 of them with the animals within two body lengths. Its curves are medians over the sessions with at least 20 onsets, 31 sessions for the male and 12 for the female. Full resolution (PDF)
Supplementary Video 2. MouseDyad-10M. The 3D poses of a male (blue) and a female (red) from one MouseDyad-10M session. 14 s, silent.
Benchmarks: identities that survive being pooled across cameras
A per-camera tracker can score well within a view and still have no idea that camera 0’s animal 1 is camera 3’s animal 1. Scoring the same sessions within view and then across views measures exactly that: LUC3D’s IDF1 on MouseDyad-10M is 0.971 within view and 0.971 across, while the per-camera trackers keep about 23% of theirs. The same figure also shows the tracker’s identity mechanism, the staleness sweep on SLAP-2M, and the tracker on a second species, the social-DANNCE rats.
Evicting stale evidence from the 3D anchor reduces identity switches. Evicting retained detections older than 20 frames cut within-view identity switches on SLAP-2M from 3,753 to 3,333, 11% fewer, and raised mean cross-view IDF1 over the 42 multi-animal sessions from 0.911 to 0.932. Identity tracking also holds up with fewer cameras: on MouseDyad-10M, mean within-view IDF1 is 0.92 with two cameras, 0.96 with three and 0.97 with four or all five. In the social-DANNCE rats the tracker recovers identity from 3D geometry alone, with 88% of sessions free of identity switches; running the same detections through a deliberately wrong calibration drops the IDF1 to 0.519.
Supplementary Figure 4. Within-view identity, cross-view identity, and 3D parameter search performance comparisons. A, Mean within-view and cross-view IDF1 for LUC3D, SLEAP and ByteTrack on MouseDyad-10M, with 95% confidence intervals. B, The share of SLAP-2M sessions whose within-view IDF1 is at or above each threshold, with one step per session. Each session’s score is the mean over its six cameras. C, The within-view IDF1 difference between LUC3D and SLEAP in each SLAP-2M session, grouped by the number of animals. Values above zero mean LUC3D scored higher. Boxes show the median and interquartile range, whiskers reach 1.5 times the interquartile range, and each dot is one session. D, False positives and identity switches on SLAP-2M, each as a percentage of camera frames. E, LUC3D’s performance on s-DANNCE on cross-view IDF1, one curve per dataset, with all 41 sessions as the dashed line. The counts under “No Switches” are the sessions in each dataset with no identity switch. F, Sweep of the staleness horizon (the anchor node eviction window) on SLAP-2M. Left, cross-view IDF1 as a survival curve, one curve for no eviction (stale 0) and for eviction windows of 1, 10, 20 and 30 frames. Right, the reduction in identity switches relative to no eviction (stale 0). A uses 55 MouseDyad-10M sessions. B to D use all 74 SLAP-2M sessions, ~11.7 million camera frames, tracked from SLEAP predictions. E uses 41 social-DANNCE rat sessions with six cameras and no added noise. F uses all 74 SLAP-2M sessions, ~11.7 million camera frames, with sessions paired across settings. Full resolution (PDF)
Calibration: comparing calibrat3 and Anipose on three rigs
Every 3D stage above rests on a camera calibration. calibrat3 solves one in the browser from ChArUco board videos. To validate it, we compared its calibrations with Anipose’s on three rigs of 5, 8 and 18 cameras, where its reprojection error is lower at the median and in the tail. Every panel is scored on Anipose’s own detected corners, so the scoring does not favour calibrat3.
Supplementary Figure 2. Calibration accuracy of calibrat3 and Anipose on three camera rigs. A, Cumulative reprojection error for calibrat3 and Anipose on the SLAP-2M rig, pooled over four calibration recordings, with each recording also drawn as a thin line. The dotted line marks the median, and the numbers give each tool’s median and 95th percentile error in pixels. B, Reprojection error for each of the SLAP-2M rig’s eight cameras. Boxes show the median and interquartile range, and whiskers reach the 5th and 95th percentiles, here and in D and F. C, Cumulative reprojection error on the MouseDyad-10M rig, pooled over five recordings. This rig was held out when choosing calibrat3’s default settings. D, Reprojection error for each of the MouseDyad-10M rig’s five cameras. E, Cumulative reprojection error on the 18-camera rig, from one recording. F, Reprojection error for each camera of the 18-camera rig. G, Median reprojection error on the 18-camera rig when calibrat3’s solver is re-run on the same detections with one setting changed at a time, such as fixing the intrinsics or using Anipose’s simpler intrinsic model. The dashed line is Anipose. H, Time to calibrate one session on each rig, split into board detection (pale) and solving (solid), with each tool running alone on the same machine. I, Percent error in focal length, camera center and rig scale on a synthetic rig where the true cameras are known. Camera center error is relative to the spacing between cameras. (A to G are scored on the corners Anipose itself detected, so the comparison does not favor calibrat3. The SLAP-2M rig is 9,029 frames over four sessions at 1280 × 1024, the MouseDyad-10M rig 12,621 frames over five sessions at the same resolution, and the 18-camera dataset 1,800 frames at 1680 × 1200, all showing one 8 × 11 ChArUco board with 24.0 mm squares. Anipose is aniposelib 0.8.0.) Full resolution (PDF)
Acquisition: hardware-synchronized video with Panopticon
Every camera in a rig has to expose on the same trigger for the views to describe one instant. Panopticon triggers the cameras off one clock, encodes during capture, and checks calibration coverage live, before any animal enters the arena.
Supplementary Figure 1. Panopticon acquires hardware synchronized multi-camera video. All panels come from one nine-camera rig of Basler a2A1920-165g5m cameras at 1920 × 1200, hardware-triggered by an Arduino Mega 2560. The synchronization recordings in C to E ran at 100 fps, and A and F at 30 fps. A, The acquisition window during a live nine-camera session. Left, one live preview pane per camera. Right, the sidebar with session metadata, calibrate and record toggles with the solver entry point, a snapshot button, the stimulation editor, the live calibration-coverage graph with its per-camera nodes and readiness state, and display controls above a status line. 1, camera preview grid; 2, camera pane; 3, real time frame rate; 4, profile; 5, output directory; 6, session metadata; 7, calibrate toggle; 8, record toggle; 9, calibration solve; 10, snapshot; 11, stimulation editor; 12, live calibration-coverage graph; 13, preview brightness; 14, preview contrast; 15, encode progress; 16, session state; 17, status bar. B, One camera view of the arena with infrared LED tied to stimulator, circled. The inset shows the lit LED. The LED is driven from the trigger board’s stimulation pin by a 2 Hz, 50% duty paradigm. C, LED intensity against trigger ordinal for all nine cameras over time. All nine cameras register the LED, 542 to 2,118 times above the per-pixel background at that frequency. D, Histogram of each camera’s inter-frame interval over a 19-minute recording. The dashed line marks the median. E, Each camera’s clock rate as parts per million of the nominal 100 Hz, showing the crystal spread across the array. The dotted line marks the array mean. F, Cumulative reprojection error for a 30 fps calibration recorded through Panopticon, solved by Panopticon’s solver vs calibrat3 on 758,665 ChArUco corner observations seen by two or more cameras. G, Focal length recovered by each solver, one point per camera. The dotted line marks equality. The two agree to a median of 2.1%. Full resolution (PDF)
Getting started
The GUI needs no installation — open it, load a session, and its calibration, cross-view identities and 3D all round-trip through SLP 2.8. To run it locally, or to read the module map:
git clone https://github.com/talmolab/luc3d.gitcd luc3dpython3 -m http.server 8080 # then open http://localhost:8080/Documentation is at talmolab.github.io/luc3d-docs.
Supplementary Video 4. Walkthrough. The full workflow in LUC3D v1.0.5: load the synchronized camera videos and the calibration, define the arena floor as a plane (four corners and the centre, triangulated, corrected and fitted) and make its centre the 3D origin with +Z toward the cameras, so the 3D grid lies on the floor, build a skeleton, label a frame with reprojection error flagging a misplaced keypoint, load a session folder with its videos, calibration and SLEAP predictions, run Track All, whose summary reports each identity’s coverage and the automatic ID-switch check, run Triangulate All from that summary, assign an identity in the Instances panel, and inspect the 3D poses from each camera’s viewpoint and in a free orbit. 10 min 9 s, silent.
BibTeX
@article{leonardis2026luc3d, title = {LUC3D: Multi-animal 3D pose tracking in the browser}, author = {Leonardis, Eric J. and Oline, Stefan N. and Tang, Isaac and Park, Joshua and Mimica, Bartul and Maree, Liezl and Afshar, Shayan and Timmermans, Olivia and Jiang, Jinrun and Whitmire, Liza and Zhu, Aaron and Wines, Talisa and Tye, Kay M. and Murthy, Mala and Falkner, Annegret L. and Pereira, Talmo D.}, journal = {bioRxiv}, publisher = {Cold Spring Harbor Laboratory}, year = {2026}, doi = {10.64898/2026.11.17.767188}, url = {https://www.biorxiv.org/content/10.64898/2026.11.17.767188v1}}