Phase 3 — Visual-Inertial Odometry

Status: complete (2026-05-26). Epic E3 (#33) closed; all 16 sub-issues merged.

Phase 3 delivered a complete MSCKF visual-inertial odometry pipeline. See the VIO Overview for the architecture; this page is the phase record.

What landed

Each row is one merged PR, through the full gate (ready PR → CodeRabbit review → gcc+clang+CI green → merge):

Issue	Component	Page
#34	`VioBackend` interface	MSCKF Backend
#48	Image container + PGM/PNG I/O	Front End
#49	Gaussian image pyramid	Front End
#39	Camera models (pinhole/fisheye/unified)	Cameras
#37	FAST corner detector	Front End
#38	Pyramidal KLT tracker	Front End
#40	IMU preintegration	IMU
#41	Feature representations + null-space types	Camera Updaters
#43	State + Propagator + StateHelper	MSCKF State
#42	IMU initialization (static + dynamic)	IMU
#44	Camera updaters (null-space MSCKF)	Camera Updaters
#35	MSCKF backend	MSCKF Backend
#45	Top-level `VioEstimator`	VioEstimator
#36	Sliding-window backend skeleton	MSCKF Backend
#46	EuRoC replay harness + ATE/RPE	Accuracy
#47	Per-frame latency budget	Latency

Filter stability — covariance stays PD/PSD and the innovation well-conditioned over 1000 consecutive camera updates; trace monotonically non-increasing.
Initialization — gravity recovered within 0.5° (static); gravity, velocities, and gyro bias recovered exactly on synthetic trajectories (dynamic).
Accuracy — ATE/RPE via the EuRoC harness; V1_01_easy gate < 0.50 m (dataset-gated).
Latency — per-frame feed_image budget, ~38 ms median at -O2 on 752×480 (details).

Jacobi over power iteration for ATE’s Horn alignment — the 4×4 profile matrix’s top two eigenvalues are near-degenerate, so power iteration silently returned wrong rotations.
Dataset / optimized-build gating so CI stays green without the EuRoC sequence and without an optimized build.

Square-root covariance backend — #187, split out rather than shipping a static_assert-only template knob.