Stage Benches
A stage bench is the focused debug, test, characterization and research environment for one transformation. It runs the stage on fixtures, in several arithmetic types and implementation variants, and prints the stage’s invariants in native units. A bench is one small executable, so an edit to a stage costs seconds to re-check, not a EuRoC replay.
What a bench gives you
Section titled “What a bench gives you”- Debugging: a failing invariant points at the transformation and names the size of the violation.
- Testing: fixtures are regression tests, and a captured fixture must replay bit for bit.
- Characterization: declarative sweeps over noise, Δt, parallax, conditioning or window size, written to CSV for the figure pipeline.
- Research: alternative implementations of the stage (variants) run on identical fixtures, side by side.
- Mixed precision: the same stage runs in
double,float,posit<32,2>,posit<16,1>,long doubleandstrict_double, each compared with thedoublerun.strict_doubleis double with only explicit conversions out (#449). Any implicit narrowing of T to double is a compile error in it, and its results must equal double’s bit for bit.
Fixtures
Section titled “Fixtures”| Kind | Input | Judged against | Made by |
|---|---|---|---|
known_answer | synthetic | an analytic expected output, computed independently of the stage | hand, in the bench |
ground_truth | built with truth injected upstream (#266) | the true values the output must reproduce | a synthetic-world run with the mean reset to truth |
captured | a recorded stage boundary | the output the stage produced in that run | capture(...); from any run with vio_pipeline --capture (#446) |
The runner adds two checks to every stage’s own invariants:
replay.bit_identical: a captured fixture, run in the type it was captured in, must reproduce its recorded output exactly.known_answer.residual: a known-answer fixture must match within the type’s arithmetic tolerance.
Fixtures are JSON (branes.vio.fixture/1) with an exact codec:
- numbers go through
double, written in shortest round-trip form; - non-finite values are written as strings;
- rotations are restored without renormalizing.
A file reads back bit-identical; the tests prove it for −0, denormals, the largest double, ±∞, NaN and a full captured state.
{"schema": "branes.vio.fixture/1", "stage": "S9_marginalization", "kind": "captured", "arithmetic": "double", "source": "synthetic_world, S2/S3/S9 stage sequence, frame 6", "input": {"state": {"R": [...], "p": [...], "clones": [...], "P": {"rows": 39, ...}}, "clone_index": 0}, "expected": {"state": {...}, "dim_before": 39, "dim_after": 33, "removed_time": ...}}Running a bench
Section titled “Running a bench”cmake --build build --target s9_marginalization_bench./build/tools/s9_marginalization_bench # built-in fixtures; double, float, posit32./build/tools/s9_marginalization_bench --types double,posit16 # choose number systems./build/tools/s9_marginalization_bench --variant all # shipped vs research variants./build/tools/s9_marginalization_bench --fixture F.json # e.g. a fixture captured from EuRoC./build/tools/s9_marginalization_bench --sweep clones=2,4,8 --csv out/./build/tools/s9_marginalization_bench --precision --csv out/ # bits lost per type (#450)--precision is the per-stage precision characterization; it is described under
Mixed precision.
In that mode a failed invariant in another type is reported, not fatal: the exit status is
non-zero only when the double reference fails or nothing ran.
The bench prints the stage’s contract from the registry, then one report per fixture, variant and type. This is real output:
captured_stage_sequence [captured] variant=shipped type=double------------------------------------------------------------------------------ invariant value threshold unit verdict------------------------------------------------------------------------------ marginalization.dimension 33 33 count PASS marginalization.principal_submatrix 0 1.386e-13 dimensionless PASS covariance.symmetric 0 6.541e-13 matrix units PASS covariance.psd 0 2.829e-12 matrix units PASS window.clones_minus_1 4 4 count PASS window.removed_the_requested_clone 0 0 s PASS mean.untouched 0 0 count PASS replay.bit_identical 0 0 count PASS------------------------------------------------------------------------------ max |output - double output| = 0 stage run time (best of 5) = 7.993 usThe same fixture in float reports max |output − double output| ≈ 2e-7, and every invariant
still passes, because each threshold scaled to float’s ε. A normal run exits non-zero on any failure,
so it can gate CI. A run that executes nothing, for example an unknown --types, also fails rather
than reporting a vacuous pass.
A characterization sweep
Section titled “A characterization sweep”The S9 bench sweeps window size × covariance conditioning and reports, per type, the principal- submatrix residual, the smallest eigenvalue after removal, and the run time. A sweep makes a precision limit visible before it bites in a full run:
| clones | log₁₀ κ | λ_min after (double) | λ_min after (posit32) | λ_min after (float) |
|---|---|---|---|---|
| 2 | 0 | 1.0e-2 | 1.0e-2 | 1.0e-2 |
| 2 | 4 | 1.0e-6 | 1.0e-6 | 9.5e-7 |
| 2 | 8 | 1.0e-10 | −4.1e-9 | −5.2e-8 |
At κ ≈ 1e8, neither posit<32,2> nor float can resolve the smallest eigenvalue. The matrix is
still positive-definite in double, but not measurably so in the narrower types, and float is
an order of magnitude further off than posit32. That is exactly the kind of per-stage precision finding the
mixed-precision work builds on.
What the first benches found
Section titled “What the first benches found”The S0 and S1 benches (#454) turned up three things on their first run, none of which an end-to-end metric had shown:
- The synthetic world’s camera wasn’t quite EuRoC cam0. Its tangential coefficient p2 was 1000× the EuRoC value (1.76e-2 vs 1.76e-5). The ground-truth fixture caught it as a 0.1 disagreement in normalized coordinates. The world is self-consistent, so nothing downstream failed. Fixed in #467, which re-baselined the synthetic fixtures; the S0 bench now uses the EuRoC intrinsics directly.
- The S1 seeding stage didn’t compile for posits: an implicit
T→doublenarrowing, now explicit. - The dynamic visual-inertial alignment is precision-fragile. It declines entirely in
floatandposit<32,2>on inputs where it resolves indouble. Its solve forms normal equations (squaring κ), adds a fixed1e-9ridge, and needs a Cholesky factorization, which breaks down once κ² exceeds 1/ε. That is exactly the kind of per-stage precision finding the mixed-precision work is built to act on.
The S2 and S3 benches (#455) added two more:
- The propagator didn’t compile for posits either:
timestamp += dtmixedTinto adouble(now explicit, bit-identical for float and double). - Propagation leaks the yaw direction, as the body-frame filter is known to do. Φ preserves global translation exactly, but its first-order form, linearized at the current estimate, leaks yaw by ≈2.4e-6 per 5 ms step on a tumbling trajectory. The bench now measures this directly, per step and across Δt and dynamics: the propagation half of the observability question behind the over-confidence (#212).
Two later variants (#471) answer that half:
- The R-IEKF’s propagation keeps yaw exactly. The
invariantvariant runsInvariantPropagatorand checks it in its own error coordinates, against an unobservable basis that does not depend on the state (translation ξp = eᵢ, yaw ξθ = ẑ). Its yaw leak is 0 on every fixture and at every sweep point. So where the R-IEKF loses to the body-frame filter end to end, the cause is not its propagation. - A higher-order body-frame Φ leaks more. With Φ = I + AΔt + ½(AΔt)² the yaw leak grows 3.5–9× on tumbling dynamics (2.2e-5 against 2.4e-6 per 5 ms step), and exp(AΔt) is the same to three digits. The leak comes from linearizing against the mean’s Euler step, not from truncating Φ: extra terms in Φ that the mean doesn’t integrate make it worse. Removing it takes a different error state, not a better Φ.
The S4 and S5 benches (#456) found four more:
- Border churn in the front end (opt-in fix). KLT drops features whose window doesn’t fit at
the coarsest pyramid level, a ≈24 px border band at the defaults, but FAST detects up to the edge.
On the bench’s frames over half of the new detections die on their first tracked frame and are
re-detected under new ids; on EuRoC V1_01, 81%.
FrontendParams::detect_trackable_onlyseeds only pointscv::klt_trackableaccepts (the tracker’s own window test on every level): the churn falls to 0 on every fixture (thetrackable_detectvariant, which enforces at most 5%) and to 2.9% on V1_01. It is off by default because it costs about 10% EuRoC ATE on V1_01 and V2_03 with the body-frame filter, a tie on MH_05, measured over severaltarget_featuressettings to rule out noise; the mechanism is open (#474, #574). - The forward-backward gate is off by default, so drifted tracks reach the backend. The bench
measures them, and its
fb_gate_1pxvariant enforces the contract. - The camera updater didn’t compile for posits: qualified
std::acos/std::sqrtcalls, now ADL. - The synthetic world steps from rest to ≈1 m/s at the end of its warm-up, a velocity jump no IMU stream can produce. Pure IMU propagation from rest is off by ≈1 m/s after one frame (#475).
Dynamic initialization from images: S1’s SfM front end
Section titled “Dynamic initialization from images: S1’s SfM front end”The backend’s moving-start init builds its keyframes from images: a two-view bootstrap on the frame with the most parallax, PnP for the rest, then the visual-inertial alignment. The S1 bench now runs that front end on SfM windows (#472): synthetic windows with known truth, and windows captured from EuRoC MH_05 by the S1 tap. Its sweep crosses observation noise, window length and camera speed. It found five things:
- The eigensolver never converged. It ran 100 sweeps per call; a window build went from 0.55 s to 28 ms once it stopped at convergence (#497).
- The metric lever arm is added to up-to-scale positions. Even exact data comes out 0.2–5.5 % off in scale (#499).
- The RANSAC gates are tighter than pixel noise. The two-view gate is ≈0.9 px. At 2 px no window builds, and on EuRoC 82 of 170 attempts fail the two-view while the drone moves. At rest the bootstrap accepts 0.05 px of “parallax” (#500).
- The alignment accepts badly scaled windows. A track only ~1 % off aligns with a 21–34 % scale
error, because
try_dynamicchecks observability, not consistency (#501). - Spacing the keyframes 0.25 s apart helps only in places; the short-baseline SfM track, with no refinement, is the cause.
Since #500 and #501 the backend builds the window with gates from the track noise (track_sigma in
the fixture, absent from older ones, which keep the fixed gates) and bundle-adjusts it. On the
0.1 px window that halves the track error (0.77 % → 0.32 %). Its scale is then 8 % off with the
accelerometer bias held at zero (was 21 %). Estimating the bias, as the backend does, this window
cannot tell a bias along gravity from vertical acceleration. The scale is pulled 66 % low, and the
settling gate declines the window. PnP’s gate does not model the landmarks’ triangulation error:
on synthetic windows it needs 2–4× the true noise to place every frame.
The MSCKF update, one sub-step at a time: S6
Section titled “The MSCKF update, one sub-step at a time: S6”S6 is five transformations, and each has its own bench, input and contract: the Jacobians (S6a) against finite differences, the null-space projection (S6b) against NᵀN = I and Nᵀ H_f = 0, the compression (S6c) against the normal equations, the gate (S6d) against χ², and the update (S6e) against K·r and P − K S Kᵀ. Each sub-step’s fixtures come from running the ones before it, so a failure points at one sub-step. The S6 benches (#457) found:
- The update adds no information along the unobservable directions. At the estimated state as much as at the true one, the projected Jacobian annihilates them (global position, yaw) to ≈1e-16. That rules out this mechanism inside S6; it doesn’t rule out other sources of the #212 over-confidence. The leak happens between updates: each update moves the linearization point the next one is evaluated at. The observability ledger books 95–99% of the spurious yaw information on EuRoC to that, and 1–5% to S2’s Φ (#480; see Capture and composition).
- The χ² gate was loose at low dof. Inliers are χ²-consistent, but with 1–3 dof the 5-per-dof gate admitted 59–70% of measurements biased by 2σ. The shipped gate is now the 95% χ² quantile, which admits 46% and 22.5% of them and 95–97.5% of inliers at every dof (#481).
- A square-root filter must carry its factor. The clone blocks are almost perfectly correlated
with the IMU pose, so their Schur-complement pivots are below float’s (and posit32’s) resolution of
P. Re-factoring the covariance in those types zeroes 36 of 51 pivots, where double zeroes only the 6
exact ones cloning makes, and represents a different covariance; measured that way the square-root
update looked ≈330× less precise than Joseph in posit32. The
sqrt_arrayvariant therefore factors the fixture’s covariance in double and rounds the factor to T, as a filter that carried its factor would hold it. Then it lands within 1.3–2.3× of Joseph in every type (#482). The S2 and S3 benches’ square-root variants still re-factor P in T. - Posits are precise near 1, not near 1e-17. Residuals at the true state and covariance entries
near σ² sit where a posit has few fraction bits, so checks on them use T’s precision at that
magnitude (
safety_at), not its ε at 1.
Calibration as known, as noise, or as state: S10
Section titled “Calibration as known, as noise, or as state: S10”The S10 bench applies the online-calibration stage to a fresh state with a believed extrinsic off from the world’s true one, runs the filter, and then updates a window of tracks three ways: with the extrinsic estimated (shipped), trusted as fixed, or trusted with its prior folded into the measurement noise. Over one 6-clone window (16 tracks), a constant camera↔IMU rotation error raises NIS only to ≈0.002 per dof at 5° and ≈8e-5 at 1°: the null-space projection absorbs almost all of it into the feature positions, so a wrong calibration is invisible to NIS and to the gate. Folding the prior into R lowers NIS further. Estimated, the extrinsic moves toward the truth, but by under 30% per window (1° → 0.955° with a 2° prior; 5° → 3.98° with a 5° prior), while its covariance stays consistent (NEES 0.49 against a χ²₀.₉₉₉(6) bound of 22.5), so recovery needs many windows of excitation. A calibration error is something the filter has to be told about, not something it will notice. The filter has no time-offset state, so the bench has no t_d sweep.
The S9 window policies (#458) leave the marginalization contract unchanged: removing the oldest or the second-newest clone keeps the principal submatrix exactly. What a policy changes is which information the update had a chance to use first, a pipeline question (#447).
Comparing alternative methods: S5 triangulation
Section titled “Comparing alternative methods: S5 triangulation”A bench variant can run a different algorithm, not just different parameters. The S5 bench runs three alternatives to the shipped ray-perpendicular solve with Gauss-Newton refinement: the two-view midpoint, the DLT, and anchored inverse depth with Gauss-Newton. All four pass every fixture in every type, and the sweep shows where they differ (mean depth error in m, feature at 5 m):
| parallax | px noise | shipped | midpoint | DLT | inverse depth |
|---|---|---|---|---|---|
| 0.25° | 0.5 | 2.38 | fails | 1.60 | 2.41 |
| 5° | 1 | 0.140 | 0.168 | 0.147 | 0.140 |
| 15° | 1 | 0.0455 | 0.0553 | 0.0465 | 0.0455 |
- Inverse depth gives the shipped answer. Both minimize the reprojection error, so the parameterization changes the path, not the answer.
- The DLT is biased toward the cameras. It wins where depth is barely observable and costs ≈5% where it is observable.
- The midpoint is the worst. It is also the only one that loses precision with exact observations: its closed form cancels catastrophically at low parallax in float.
- No method recovers a depth the geometry doesn’t hold. Below about 1° the parallax gate is the answer, not the solver.
The S4 inspector, which draws the front end’s tracks on EuRoC frames, now calls the extracted stage too. What it shows is what the estimator does.
Writing a bench
Section titled “Writing a bench”A bench is a struct describing its stage: the I/O types, the codec, run (which calls
stages::<sN_name>::apply, with research variants as extra branches), flatten (every output
number, for the cross-type and replay comparisons), invariants (the stage’s checks), the built-in
fixtures, and optionally a sweep. bench_main<B> turns it into the executable. The S9 bench
(tools/include/branes/tools/bench/s9_marginalization_bench.hpp) is the template; the
developer reference spells out the full contract.
The edit-compile-run loop
Section titled “The edit-compile-run loop”Measured by touching the stage header, then rebuilding and running only the bench target
(i7-12700K, gcc 13.3, -j4):
| Build | Rebuild | Run: double, float | Run: + posit32 |
|---|---|---|---|
default (no -O) | 4.3 s | 0.08 s | 9.1 s |
Release (-O3) | 8.4 s | < 0.01 s | 1.1 s |
Posits are software arithmetic, so posit work is best done in a Release build. Either way, a stage edit is re-checked in seconds.
Developer reference
Section titled “Developer reference”Layout
Section titled “Layout”| Path | What |
|---|---|
tools/include/branes/tools/bench/bench.hpp | the runner, the report, and the CLI (bench_main<B>) |
tools/include/branes/tools/bench/fixture.hpp | the fixture model, the exact JSON codec, and the capture writer |
tools/include/branes/tools/bench/types.hpp | the arithmetic types: double, float, posit32 (posit<32,2>), posit16 (posit<16,1>), long_double as the wider-than-double reference, and strict_double (strict_double.hpp, #449) |
tools/include/branes/tools/bench/variant.hpp | the variant slot: named alternative implementations of a stage |
tools/include/branes/tools/bench/sweep.hpp | declarative characterization sweeps and CSV tables |
tools/include/branes/tools/bench/s<N>_<transformation>_bench.hpp | one stage’s bench (the adapter) |
tools/benches/s<N>_<transformation>_bench.cpp | the bench executable, a one-line main |
tools/benches/fixtures/s<N>_<transformation>/*.json | the stage’s committed fixtures |
tests/tools/stage_bench.cpp | the Catch2 integration: the framework’s guarantees, locked |
The stage transformations the benches call are in sdk/include/branes/sdk/msckf/stages/ (#452), and
the full pipeline is their call sequence, so what a bench establishes about a stage holds where the
backend calls it.
The benches
Section titled “The benches”| Bench | Issue | Variants | Sweep |
|---|---|---|---|
s0_sensor_model_bench | #454 | shipped (Newton undistortion), fixed_point | distortion strength × radius → round trip, Jacobian, px per 1° extrinsic / per 1 ms offset |
s1_initialization_bench | #454, #472 | shipped (static), gravity_align, dynamic (VI alignment); SfM front end: sfm_two_view_pnp, sfm_adjacent_bootstrap, sfm_keyframes_4hz | window × IMU noise × excitation → roll/pitch and scale error; observation noise × window length × camera speed for the SfM variants |
s2_propagation_bench | #455, #471 | shipped (first-order Φ, diagonal Q_d), canonical_qd, sqrt_covariance, higher_order_phi, invariant (R-IEKF) | Δt × dynamics × Q scale → position error (body and invariant), yaw leak (first-order, second-order, exp(AΔt) and invariant Φ), diagonal-vs-canonical position-σ gap, λ_min/‖P‖ |
s3_augmentation_bench | #455 | shipped, sqrt_covariance | clones × conditioning → block residual, λ_min/‖P‖ |
s4_frontend_bench | #456 | shipped (FB gate off), fb_gate_1px, klt_window_7 | image noise × shift → survival, end-point RMS, FB median |
s5_triangulation_bench | #456 | shipped (linear + Gauss-Newton), linear_only, parallax_gate_2deg, midpoint_two_view, dlt_linear, inverse_depth_gn | parallax × pixel noise → depth error per method, κ, gate rejection |
s6a_jacobians_bench | #457 | shipped | observations × depth → H_f, H_x finite-difference error, max |H_f| |
s6b_nullspace_projection_bench | #457 | shipped (Householder), givens | observations × depth → NᵀN − I, Nᵀ H_f, H₀·N per method |
s6c_compression_bench | #457 | shipped (identity), qr | features stacked → rows in/out, normal-equation error |
s6d_gating_bench | #457, #481 | shipped (χ²₀.₉₅ quantile), chi2_99, per_dof (5 per dof, the gate before #481), gate_off | dof × outlier size → inlier NIS/dof, acceptance per gate |
s6e_ekf_update_bench | #457 | shipped (Joseph), sqrt_array | pixel noise × calibration σ → trace ratio, information along N, ‖δx‖, NIS/dof |
s9_marginalization_bench | #453 (worked example), #458 | shipped, direct_gather, policy_oldest, policy_second_newest | clones × conditioning → residual, λ_min |
s10_online_calibration_bench | #458 | shipped (extrinsic estimated), fixed, folded_into_r | extrinsic rotation error × prior → remaining error, NIS/dof per treatment |
- The S6 sub-steps share a scene (
s6_scene.hpp): the synthetic world run through S2 and S3, so the covariance is the one those stages produce, with each sub-step’s input built by running the shipped upstream sub-steps in double. A sub-step’s bench (S6a_…toS6e_…) prints the S6 contract. - Slow stages in software arithmetic set
static constexpr int kTimingReps(S2 and S3 use 1). - A stage that is not generic in T declares
using SupportedTypes = TypeList<...>. S4’s KLT front end runs in double only; #449 decides whether it becomes generic. The S1 SfM variants run only in native types by default (runs_by_default): their RANSAC runs hundreds of eigensolves per window, and an emulated posit is ~10⁴× slower. - Image fixtures are stored losslessly as base64 bytes (
pack(cv::Image),unpack_image). - Square-root variants start from a real filter state with
psd_factor, a Cholesky that tolerates the exactly singular covariance a fresh clone produces.
Fixtures in detail
Section titled “Fixtures in detail”The fixture kinds and the two runner checks are above. The known_answer.residual
tolerance is safety · n · ε_T · scale (#445).
- Variant binding. An
expectedoutput can belong to one implementation. The fixture’svariantfield names it, and the runner applies the known-answer and replay checks only to that variant; it can also name a list ("shipped,direct_gather"). An emptyvariantmeans every variant must reproduce it. Captured fixtures record the variant that ran. S1 needs this: its static and dynamic variants legitimately estimate different things from the same input. - Comparisons against double references. Known answers, ground truth and stored rotations all pass
through
double, so a comparison is never held tighter than double’s ε, even when the stage ran in a wider type (tolerance_vs_double<T>). - Checks on small quantities. A posit is most precise near 1 and loses fraction bits as |x| moves
away; at 1e-17 posit32 keeps about 14 of its 28. A check on a residual at the true state, or a
covariance entry near σ², uses
safety_at<T>(magnitude), which scales the safety factor byeps_at<T>(magnitude) / ε_T: T’s relative spacing at that size. It is 1 for IEEE types. Don’t apply it to a lower bound (an SPD margin): a larger safety factor makes those stricter.
A fixture file is one JSON document, schema branes.vio.fixture/1:
{"schema": "branes.vio.fixture/1", "stage": "S9_marginalization", "kind": "captured", "arithmetic": "double", "source": "…", "seed": 0, "description": "…", "variant": "…", "input": {…}, "expected": {…}, "truth": {…}}The payloads use a small vocabulary: numbers, vectors, row-major matrices {"rows","cols","data"},
rotations as unit quaternions [w,x,y,z], and the full MSCKF state (navigation, calibration blocks,
clone window, covariance P). The codec is bit-exact for every type whose values fit in a double, which
is every type the benches store except long double where it is wider than double (x86 Linux):
a fixture written from a long double run keeps only its double part:
- every number goes through
double, which holds every float,posit<32,2>and double value exactly; - nlohmann/json writes doubles in shortest round-trip form;
- non-finite values are written as the strings
"nan","inf"and"-inf"; - rotations are restored with
SO3::from_unit_quaternion, without renormalizing, because renormalizing is not bit-idempotent.
The bench contract
Section titled “The bench contract”The full contract is the header comment of bench.hpp:
struct S9MarginalizationBench { static constexpr std::string_view kStage = "S9_marginalization"; static constexpr inv::Stage kInvStage = inv::Stage::S9_marginalization; static std::vector<Variant> variants(); // "shipped" first template <class T> struct Input; template <class T> struct Output; template <class T> static Input<T> decode_input(const json&); template <class T> static Output<T> decode_output(const json&); template <class T> static json encode_output(const Output<T>&); template <class T> static Output<T> run(const Input<T>&, std::string_view variant); template <class T> static std::vector<double> flatten(const Output<T>&); template <class T> static std::vector<inv::InvariantResult> invariants(const Input<T>&, const Output<T>&, const Fixture&); static std::vector<NamedFixture> builtin_fixtures(); // optional characterization sweep static Sweep default_sweep(); static std::vector<std::string> sweep_columns(); template <class T> static std::vector<double> sweep_point(const Point&);};runcalls the stage’s transformation,stages::<sN_name>::apply. A research variant is another branch ofrun, selected by name.invariantsreturns the stage’s checks frombranes/sdk/eval/invariants.hpp. The S9 bench checks the principal submatrix; symmetry and positive-semidefiniteness; the dimension and window bookkeeping; that the mean is untouched; and, for a ground-truth fixture, the kept clones against the truth.flattenlists every output number, for the comparison againstdoubleand the replay check.
Report, CSV and exit code
Section titled “Report, CSV and exit code”The report prints the stage contract from the registry (vio_stage_contracts.hpp), then, per
fixture, variant and type: every invariant with its value, threshold, unit and verdict; the largest
deviation from the double run; and the stage’s run time. --capture DIR writes the built-in
fixtures as files.
--csv DIR writes:
<stage>_report.csv: one row per fixture × variant × type × invariant;<stage>_sweep_<type>.csv(with--sweep): one per type, columns = the sweep axes followed by the bench’s metrics, ready for the docs-site figure scripts;<stage>_precision.csv(with--precision).
A normal run exits non-zero if any invariant fails, or if nothing ran, so it can gate CI. Under
--precision only a failure of the double reference, or an empty run, is fatal.
Existing per-stage assets
Section titled “Existing per-stage assets”The benches build on what was already there rather than replacing it at once:
sdk/include/branes/sdk/eval/*_probe.hpp: the stage probes (sensor model, initialization, propagation, clone window, frontend, triangulation, update, calibration, observability). Their scenarios became the benches’ fixtures and sweeps in #454–#458.tools/src/s<N>_*.cppands<N>_inspect.cpp: the contract printers, the CSV probes and the trace inspectors. A bench prints the same contract from the same registry, and its sweep CSVs feed the same figure scripts.tests/sdk/stage tests, e.g.msckf_stages.cppfor the stage transformations: these stay as the fast unit tests. The bench’s Catch2 integration covers its fixtures.