MSCKF Backend

MsckfBackend<T> (sdk/include/branes/sdk/msckf_backend.hpp, #35) is the VioBackend implementation that wires the Phase-3 pieces into one estimator. It is the full-covariance MSCKF.

The VioBackend interface

The backend implements a small abstraction (#34) that separates the front end from the estimator:

void initialize(const VioConfig&);
void process_imu(const ImuMeasurement<T>&);          // high rate
void process_camera(double t, span<const FrontendObservation<T>>);  // frame rate
NavState<T> current_state() const;

Swapping MSCKF for a different backend requires no front-end change — see the skeleton below.

What it wires together

• Initialization — buffers an opening window of IMU samples and runs the static initializer; if the platform never settles it falls back to an identity attitude (a moving start is left to dynamic VI init).
• Propagation — between IMU samples, per-sample mean + covariance propagation via the Propagator, driven by dt from the timestamps.
• Clone window — every camera frame augments a clone; the window is bounded by config.max_clones and marginalized oldest-first. Crucially, process_camera first propagates the state to the image timestamp (zero-order hold on the last IMU sample) before cloning, so each clone is anchored at exactly the frame time even on an asynchronous IMU/camera feed.
• Feature management — the standard MSCKF policy: a feature track is fed to the camera updater when it ends (no longer observed) or when the oldest clone it touches is about to be dropped.

Robustness details

• Observations are pixel coordinates per the VioBackend contract; each camera’s model unprojects them to bearings, keeping the estimator core intrinsics-agnostic.
• An out-of-order or duplicate IMU sample (dt ≤ 0) is dropped without advancing the filter time, so a later valid sample can’t integrate across already-processed time.

Square-root covariance form

The backend is generic over its covariance representation: MsckfBackendT<T, Cov> (msckf_backend.hpp), with Cov either msckf::FullCovariance<T> (Joseph form, the default; alias MsckfBackend<T>) or msckf::SqrtCovariance<T> (sdk/include/branes/sdk/msckf/sqrt_covariance.hpp, #187; alias MsckfSqrtBackend<T>). The square-root form carries the Cholesky factor S, with P = S·Sᵀ, instead of P, and replaces the Joseph update with a QR array update. It is clean-room from the textbook array algorithms (Kailath, Sayed & Hassibi, Linear Estimation; Mourikis’s MSCKF).

Every covariance operation reduces to one primitive, a Householder QR that returns the upper-triangular R (msckf/qr.hpp):

• Predict P ← F·P·Fᵀ + Q: retriangularize [F·S | √Q].
• Augment a clone: retriangularize G·S, with G the clone selector. This handles the singular clone block naturally.
• Marginalize: drop the clone’s rows from S and retriangularize.
• Update: the array algorithm on [[√R, H·S], [0, S]] yields the innovation factor Rₑ, the gain block and the updated S′ in one QR; δx = K̄·(Rₑ⁻¹r) by triangular solves.
• Gate: γ = ‖Rₑ⁻¹r‖², where Rₑ factors H·P·Hᵀ + R. No explicit inverse is formed.

Only these five operations differ between the two forms. The covariance-independent code is shared, and it is the bulk: the IMU mean strapdown, triangulation, projection and Jacobians, the null-space projection, and the whole backend feature-management policy (~700 lines).

Select it with vio_pipeline --covariance sqrt (the default is full). Its behaviour in float (#533) is measured under Mixed precision.

A second backend: the sliding-window skeleton

SlidingWindowBackend<T> (#36) is a deliberate placeholder implementing the same VioBackend interface but performing no estimation — every operation raises a NotImplemented marker, and a kImplemented = false constant advertises that. Its purpose is to prove the backend abstraction is real and substitutable before MSCKF accreted implementation-specific assumptions: a VioEstimator can be templated on it and link. The real sliding-window-optimization math (keyframe marginalization, factor-graph solve over the preintegration and reprojection factors) lands post-MVP.

Validated

The backend’s end-to-end test runs a stationary init then frames under acceleration and asserts that it produces finite, forward-moving poses while the covariance stays PSD and the clone window respects max_clones. Trajectory accuracy on real data is the EuRoC harness’s job (#46).