Reading the Metrics

Every stage page reports its health in native units. This page is the shared primer for the metrics they use, so the stage pages can stay focused. There are two distinct questions:

1. Is the estimate accurate? — how close is it to truth. (ATE, RPE)
2. Is the estimate’s claimed uncertainty honest? — does the covariance match the real error. (NEES, NIS, innovation whiteness)

The second is the one most people skip, and it is the one that kills missions.

Accuracy — ATE and RPE

• ATE (Absolute Trajectory Error) — RMS position error between the estimate and ground truth after a rigid (Horn) alignment. One number for “how far off, globally.” Sensitive to drift accumulating over the whole run.
• RPE (Relative Pose Error) — error over short fixed-length sub-trajectories. Measures drift rate (e.g. % of distance), which for long missions matters more than a single global number.

Accuracy tells you nothing about whether the filter knows it’s wrong.

Consistency — the calibration of the uncertainty

A Kalman-type filter produces an estimate and a statement of its own uncertainty, the covariance P. When VIO says “the camera is here,” it means “I’m 68% sure it’s within this ellipsoid.” A filter is consistent when that self-report matches reality: it says ±10 cm and is actually wrong by ~10 cm — not 1 m (over-confident, lying that it’s sure) and not 1 mm (under-confident, wasting information).

The weather-forecaster analogy

A forecaster who says “70% chance of rain” is well-calibrated if, across all the days they said 70%, it actually rained about 70% of the time. NEES and NIS are that calibration check for an estimator — they don’t ask “is the estimate close?”, they ask “is the estimate’s claimed uncertainty honest?”

The one idea: “how many sigmas off?”

For a scalar estimate x̂ with standard deviation σ and truth x, the error e = x̂ − x measured in units of σ is z = e/σ, and z² = e²/σ² has expected value 1 if the filter is honest. Average it over many estimates: ≈ 1 is honest, ≫ 1 means real errors exceed what σ claims (over-confident), ≪ 1 means under-confident.

In many dimensions the state is correlated, so 1/σ² becomes the inverse covariance and e²/σ² becomes the squared Mahalanobis distance:

  d² = eᵀ P⁻¹ e

If the filter is consistent, d² ~ χ²(n) with n the dimension, and the key fact is E[χ²(n)] = n. So divide by the dimension and a consistent filter sits at 1.0. Everything below is “measure d²/n and check it’s near 1.”

NEES vs NIS — same idea, two places

Both are the dof-normalized Mahalanobis distance above; they differ only in what error and what covariance you plug in:

• NEES — Normalized Estimation Error Squared: eᵀ P⁻¹ e on the state estimate’s error vs the state covariance P.
• NIS — Normalized Innovation Squared: the same form on the innovation (measurement − prediction) vs the innovation covariance S.

	NEES	NIS
error	x̂ − x_true (state error)	z − ẑ (innovation: measured − predicted)
covariance	P (state covariance)	S = HPHᵀ + R (innovation covariance)
dof	state dim (15 in the cortex core)	measurement dim
needs truth?	yes → offline (EuRoC)	no → online, always

• NEES tests the whole state’s calibration, including directions measurements don’t directly observe. Per-state-block NEES (slice out attitude, or velocity) localizes which state is over/under-confident.
• NIS tests the filter against its own incoming measurements — no ground truth — so it runs live on the robot and powers the χ² outlier gate.

NIS consistent while NEES is inconsistent is itself a clue: the measurement model is fine, but a hidden state (a bias, a scale, an unmodeled calibration) is drifting.

From one number to a verdict

A single d² is very noisy (χ²(n) has standard deviation √(2n), comparable to its mean). The test averages over a run: the sum of N samples is χ²(D) with D = Σ dofᵢ, which for large D is ≈ Normal(D, 2D). The dof-normalized average S/D lands near 1.0 with a tight confidence band, turning a noisy stream into a verdict: consistent, over-confident (> 1, the dangerous one), or under-confident (< 1). The two-sided $(1-\alpha)$ acceptance band is $1 \pm z\sqrt{2/D}$ with $z = \Phi^{-1}(1-\alpha/2)$ ($z = 1.96$ at 95%): the classic Bar-Shalom NEES/NIS consistency test. The band tightens as the run grows.

The per-update χ² gate is a different use of the same statistic. Before applying a measurement, the filter checks its NIS against a χ² threshold and rejects a wild outlier (a mistracked feature). That is a single-sample gate; the consistency test is a run average. Both compute the same νᵀ S⁻¹ ν, and #264 logs the gate’s value so the consistency test can run on it. The shipped gate accepts γ ≤ χ²₀.₉₅(dof), the 95% quantile for the residual’s dimension, so a consistent inlier passes 95% of the time whatever its dof. It replaced a fixed γ ≤ 5 per dof, which sat far above the quantile at low dof (5 against 3.84 at 1 dof) and admitted most 2σ-biased short tracks (#481).

Innovation whiteness

NIS tests the innovation’s magnitude. Whiteness tests its structure: a healthy innovation sequence is zero-mean and temporally uncorrelated (white). A biased mean points at a systematic error (extrinsics, triangulation, time-sync); temporal correlation points at unmodeled dynamics or a linearization/observability inconsistency — failure modes a magnitude test cannot see.

How cortex computes them

The quadratic form eᵀ P⁻¹ e is normalized_squared(e, P) in eval/consistency.hpp (a Cholesky solve, never an explicit inverse). NIS is accumulated on every camera update (nis_consistency()); innovation whiteness is the directional/temporal discriminator (innovation_whiteness()). The headers are under sdk/include/branes/sdk/.

Piece	Header	Role
normalized_squared / nis / nees	eval/consistency.hpp	the eᵀ P⁻¹ e form; a Cholesky solve that throws on a non-positive-definite covariance
ConsistencyAccumulator / ConsistencyReport	eval/consistency.hpp	add(d², dof) per sample; report() gives the normalized average and the band verdict
per-update NIS	msckf/camera_updater.hpp → MsckfBackend::nis_consistency()	the gate’s γ = νᵀ S⁻¹ ν, recorded on every camera update
nav_error / gauge_align / align_truth / core_covariance	eval/nav_consistency.hpp	the 15-vector state error against ground truth, in the filter’s convention and frame
EuRoC report	tests/sdk/vio_euroc.cpp	warns with the run’s NIS and NEES verdicts

Two VIO-specific subtleties

1. The error lives in the filter’s tangent space and ordering. The state error is [δθ δp δv δbg δba], with the attitude error a rotation on the manifold, δθ = Log(R̂⁻¹ R_true), not a quaternion difference. nav_error builds exactly this, so it matches the layout of P.
2. The gauge is yaw-only, not SE(3). A VIO solution is fixed only up to four unobservable degrees of freedom: global position and yaw about gravity. Roll and pitch are observable, because the accelerometer feels gravity. The estimate and the truth therefore live in different world frames, and only that 4-DoF gauge may be removed before differencing. gauge_align builds a yaw-only rotation plus a translation (Zhang & Scaramuzza’s “posyaw” alignment). A full 6-DoF SE(3) alignment would rotate away the observable roll/pitch error and report a flatteringly small NEES, a bug caught in review.

Status

On EuRoC the cortex filter’s NEES runs persistently above dof — structural over-confidence. The worked diagnostic chain is on Consistency Analysis; the stage-by-stage decomposition and tracking issues are on Engineering Status.

Reading the verdict

Reading	Meaning	First suspects
normalized ≈ 1, in band	consistent: the covariance honestly bounds the error	(healthy)
normalized above the band (over-confident)	real errors exceed the claimed uncertainty	process noise Q too small; a wrong or sign-flipped Jacobian; an unmodeled bias making residuals systematically biased
normalized below the band (under-confident)	the claimed uncertainty exceeds the real errors	measurement noise R too large; over-inflated Q; the filter ignoring usable information
NIS consistent, NEES ≫ 1	innovations fine, state drifting unmodeled	a systematic scale or bias error the covariance doesn’t represent (a prime #212 signature)
per-block NEES spikes in one state	that state is the culprit	e.g. attitude-block NEES climbing first ⇒ gyro bias or the propagation Jacobian

A divergence or a drift now has a direction: not “it’s wrong, edit something and re-measure ATE,” but “the filter is over-confident, in the attitude block first, from the second propagation,” which points at a component and a term (see the diagnostic methodology).

Caveats

• It is a large-sample test. The Gaussian band assumes hundreds to thousands of updates; on a handful of samples it is meaningless. At the nominal α a perfectly consistent filter is still flagged α of the time by chance, so treat a marginal verdict as a prompt to look, not a proof.
• Samples within one run are correlated. The band assumes independent χ² terms; consecutive frames of one run are not, so the sum’s variance can exceed 2D and the band is too narrow. A single-run verdict is approximate; independent Monte Carlo runs give the exact test.
• The covariance and the error must be the same object: the same tangent space, state ordering and units. A NEES against a mismatched P is nonsense, which is why nav_error and core_covariance are tied to msckf::State’s documented layout.
• A non-positive-definite covariance is itself a failure. normalized_squared throws on it, and the EuRoC report counts and surfaces any frames it had to skip.

References

• Y. Bar-Shalom, X.-R. Li, T. Kirubarajan, Estimation with Applications to Tracking and Navigation (2001): the NEES/NIS consistency tests and the χ² acceptance bounds.
• J. Hesch, D. Kottas, S. Bowman, S. Roumeliotis, Consistency Analysis and Improvement of Vision-aided Inertial Navigation (T-RO 2014): VIO observability and filter consistency, the 4-DoF unobservable subspace.
• Z. Zhang, D. Scaramuzza, A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry (IROS 2018): the yaw-only (“posyaw”) gauge alignment.