Skip to content

3D Triangulation

LUC3D reconstructs 3D poses from 2D annotations across multiple camera views using two methods: DLT (Direct Linear Transform — fast) and Refined (Ref — iterative and slower; helps when one view's 2D point is clearly wrong, otherwise about the same as DLT). Set the default in Help ▸ Settings ▸ Triangulation Wizard or with the toolbar Triangulate ▾ menu's Method choice. For the math behind each, see How triangulation works.

Prerequisites

Before triangulating, you need:

  1. Camera calibration loaded (TOML or JSON with intrinsic and extrinsic parameters)
  2. 2D annotations in at least 2 camera views
  3. Instance groups linking the same individual across views — or per-frame identities from the tracker, which Triangulate turns into groups automatically

Triangulating a Frame

Single Frame

Press T, go to Edit → Triangulate Current Frame, or click Triangulate in the toolbar (hover the ▾ to choose DLT or Ref explicitly).

LUC3D will, for every group in the current frame:

  1. Gather 2D keypoints from all linked camera views
  2. Compute the 3D position of each keypoint with the chosen method
  3. Store the 3D points on the instance group
  4. Generate reprojected instances (the 3D points projected back to each camera's 2D view)

Multiple Frames

Edit → Triangulate Multiple Frames opens the Choose Frames for Triangulation dialog: drag the range slider or type Start / End, untick any cameras to exclude, and click Continue. The status bar reports frames processed and the average error.

All Frames

  • Triangulate ▾ (toolbar) — Click the button, or pick All frames in its menu, to triangulate every frame that has groups. Choosing DLT on a session with identities first groups by identity; Ref runs the refined solver over all existing groups.
  • Edit → Group by Track & Triangulate All — Opens a dialog to pick which tracks and cameras to include, groups instances sharing a track across cameras, then triangulates every frame
  • Edit → Group by Identity & Triangulate All — Groups by identity, then triangulates every frame

Reprojection Error

After triangulation, LUC3D computes the reprojection error — the mean distance (in pixels) between where a keypoint was annotated and where the triangulated 3D point projects back onto that camera view.

Reading the Error

In the Instances tab of the info panel, the Reprojection Error section shows:

  • The method used (DLT or Refined) and the frame's mean error, color-coded; for Refined, the undistorted-space equivalent ("undist") as well
  • Per-camera means to show which views fit worst
  • A collapsible Per-instance node breakdown — one node × camera table per instance, with that instance's mean and an Avg column, every cell color-coded

The Grouped Instances table also gains an Error column, and the status bar shows Error: X px for the current frame even when the panel is hidden.

Error Color Coding

Error Color Meaning
< 2 px Green Excellent — annotation is consistent across views
2–5 px Amber Acceptable — minor inconsistencies
≥ 5 px Red Poor — check for annotation errors or calibration issues

Toggling Overlays

  • Press R (toolbar Reproj) to show reprojected instances — dashed edges with white nodes by default
  • Press E (toolbar Errors) to draw error vectors: a line from each labeled node to its reprojection
  • Turn on Display Legend in the Visibility tab to label detected, reprojected, and error-vector marks in each view
  • Use the Visibility tab's Reprojections section to change node color (white / black / track), edge style, brightness, and labels

Robust triangulation

An optional reprojection error threshold (set in the toolbar's Triangulate ▾ menu or the Tracking Wizard) automatically drops per-view keypoint observations whose reprojection error exceeds the threshold and re-triangulates from the remaining views. It is off by default. Use it to make triangulation resilient to the occasional badly mislabeled point without hand-nulling each one. See Robust triangulation.

3D Viewport

Press \, go to View → Toggle 3D Viewport, or click Hide/Show 3D View in the toolbar. A hidden viewport stops rendering, which helps on large sessions.

Import existing 3D points

You can load precomputed 3D points via File ▸ Load 3D Points (H5)... and view them without any videos loaded (skeleton + points only). With no video, playback is driven by a frame stepper, and the timeline still works. Cameras that have no linked video are drawn in red.

Viewport Controls

  • Left-click + drag — Orbit the camera
  • Scroll wheel — Zoom in/out
  • Right-click + drag — Pan
  • Click a camera frustum — Select that camera (highlights its video pane too)

Viewport Features

  • Camera frustums — Wireframe pyramids showing each camera's position and viewing direction; sizes are adjustable in the Visibility ▸ 3D Viewer table
  • 3D skeleton — Triangulated keypoints connected by skeleton edges
  • Color coding — Follows the toolbar Color: Tracks / ID toggle
  • Node trails — Tracks ▸ Node Trails draws the last 10–500 frames of each node's path

Viewport Buttons

Button Action
Show Camera View Snap the 3D camera to the perspective of the camera you selected by clicking its frustum
Show Initial View Return to the orbiting viewpoint (shown while in a camera view)
Set Env Load the session's environment files (.externals. .slp/.h5 in each camera folder), triangulate them, and show the result as a persistent overlay — e.g. the arena or apparatus
Clear Env Remove the environment overlay

View → Reset 3D View and View → Fit 3D to Scene reset the orbit camera or frame everything in view.

Troubleshooting

High reprojection error

If error is consistently high across all keypoints:

  • Check that your calibration file matches the video resolution
  • Verify that camera names are correctly mapped to videos
  • Check the grouping — instances from different animals may be linked
  • Re-examine the calibration quality (try recalibrating)

Some keypoints have high error

If only specific keypoints show high error:

  • Check those keypoints in each camera view — they may be mispositioned
  • Right-click keypoints you can't see to mark them not visible rather than guessing
  • Keypoints visible in only 1 view cannot be triangulated

Note

Triangulation requires a keypoint to be visible in at least 2 camera views. Keypoints marked not visible, or visible in only one view, are excluded from 3D reconstruction.