3D Triangulation¶
LUC3D reconstructs 3D poses from 2D annotations across multiple camera views using two methods: DLT (Direct Linear Transform — fast) and Refined (Ref — iterative and slower; helps when one view's 2D point is clearly wrong, otherwise about the same as DLT). Set the default in Help ▸ Settings ▸ Triangulation Wizard or with the toolbar Triangulate ▾ menu's Method choice. For the math behind each, see How triangulation works.
Prerequisites¶
Before triangulating, you need:
- Camera calibration loaded (TOML or JSON with intrinsic and extrinsic parameters)
- 2D annotations in at least 2 camera views
- Instance groups linking the same individual across views — or per-frame identities from the tracker, which Triangulate turns into groups automatically
Triangulating a Frame¶
Single Frame¶
Press T, go to Edit → Triangulate Current Frame, or click Triangulate in the toolbar (hover the ▾ to choose DLT or Ref explicitly).
LUC3D will, for every group in the current frame:
- Gather 2D keypoints from all linked camera views
- Compute the 3D position of each keypoint with the chosen method
- Store the 3D points on the instance group
- Generate reprojected instances (the 3D points projected back to each camera's 2D view)
Multiple Frames¶
Edit → Triangulate Multiple Frames opens the Choose Frames for Triangulation dialog: drag the range slider or type Start / End, untick any cameras to exclude, and click Continue. The status bar reports frames processed and the average error.
All Frames¶
- Triangulate ▾ (toolbar) — Click the button, or pick All frames in its menu, to triangulate every frame that has groups. Choosing DLT on a session with identities first groups by identity; Ref runs the refined solver over all existing groups.
- Edit → Group by Track & Triangulate All — Opens a dialog to pick which tracks and cameras to include, groups instances sharing a track across cameras, then triangulates every frame
- Edit → Group by Identity & Triangulate All — Groups by identity, then triangulates every frame
Reprojection Error¶
After triangulation, LUC3D computes the reprojection error — the mean distance (in pixels) between where a keypoint was annotated and where the triangulated 3D point projects back onto that camera view.
Reading the Error¶
In the Instances tab of the info panel, the Reprojection Error section shows:
- The method used (DLT or Refined) and the frame's mean error, color-coded; for Refined, the undistorted-space equivalent ("undist") as well
- Per-camera means to show which views fit worst
- A collapsible Per-instance node breakdown — one node × camera table per instance, with that instance's mean and an Avg column, every cell color-coded
The Grouped Instances table also gains an Error column, and the status bar shows Error: X px for the current frame even when the panel is hidden.
Error Color Coding¶
| Error | Color | Meaning |
|---|---|---|
| < 2 px | Green | Excellent — annotation is consistent across views |
| 2–5 px | Amber | Acceptable — minor inconsistencies |
| ≥ 5 px | Red | Poor — check for annotation errors or calibration issues |
Toggling Overlays¶
- Press R (toolbar Reproj) to show reprojected instances — dashed edges with white nodes by default
- Press E (toolbar Errors) to draw error vectors: a line from each labeled node to its reprojection
- Turn on Display Legend in the Visibility tab to label detected, reprojected, and error-vector marks in each view
- Use the Visibility tab's Reprojections section to change node color (white / black / track), edge style, brightness, and labels
Robust triangulation¶
An optional reprojection error threshold (set in the toolbar's Triangulate ▾ menu or the Tracking Wizard) automatically drops per-view keypoint observations whose reprojection error exceeds the threshold and re-triangulates from the remaining views. It is off by default. Use it to make triangulation resilient to the occasional badly mislabeled point without hand-nulling each one. See Robust triangulation.
3D Viewport¶
Press \, go to View → Toggle 3D Viewport, or click Hide/Show 3D View in the toolbar. A hidden viewport stops rendering, which helps on large sessions.
Import existing 3D points
You can load precomputed 3D points via File ▸ Load 3D Points (H5)... and view them without any videos loaded (skeleton + points only). With no video, playback is driven by a frame stepper, and the timeline still works. Cameras that have no linked video are drawn in red.
Viewport Controls¶
- Left-click + drag — Orbit the camera
- Scroll wheel — Zoom in/out
- Right-click + drag — Pan
- Click a camera frustum — Select that camera (highlights its video pane too)
Viewport Features¶
- Camera frustums — Wireframe pyramids showing each camera's position and viewing direction; sizes are adjustable in the Visibility ▸ 3D Viewer table
- 3D skeleton — Triangulated keypoints connected by skeleton edges
- Color coding — Follows the toolbar Color: Tracks / ID toggle
- Node trails — Tracks ▸ Node Trails draws the last 10–500 frames of each node's path
Viewport Buttons¶
| Button | Action |
|---|---|
| Show Camera View | Snap the 3D camera to the perspective of the camera you selected by clicking its frustum |
| Show Initial View | Return to the orbiting viewpoint (shown while in a camera view) |
| Set Env | Load the session's environment files (.externals. .slp/.h5 in each camera folder), triangulate them, and show the result as a persistent overlay — e.g. the arena or apparatus |
| Clear Env | Remove the environment overlay |
View → Reset 3D View and View → Fit 3D to Scene reset the orbit camera or frame everything in view.
Troubleshooting¶
High reprojection error
If error is consistently high across all keypoints:
- Check that your calibration file matches the video resolution
- Verify that camera names are correctly mapped to videos
- Check the grouping — instances from different animals may be linked
- Re-examine the calibration quality (try recalibrating)
Some keypoints have high error
If only specific keypoints show high error:
- Check those keypoints in each camera view — they may be mispositioned
- Right-click keypoints you can't see to mark them not visible rather than guessing
- Keypoints visible in only 1 view cannot be triangulated
Note
Triangulation requires a keypoint to be visible in at least 2 camera views. Keypoints marked not visible, or visible in only one view, are excluded from 3D reconstruction.