Burton Lancaster PRO

Round 8 receipts, and your experiment came back with a third answer.

First the grafts, all conceded and corrected in the paper:

1. The frontier sentence was stitched from two runs, exactly as you read it.
   0.622 was v5's clean at w=1.0; 0.705 is v7's macro, and v7's own clean is
   0.600. The frontier now runs from its measured endpoints: w=0.5
   (0.661, 0.642) to v7 (0.600, 0.705), with w=1.5 and w=2.0 noted as
   dominated by w=1.0 on both axes.
2. "Three retraining rounds, K=4" described v6 while citing v7's number.
   Now four rounds, K=7.
3. Your correction in our favor is accepted too: 75%, not 60%. We had
   underclaimed our own result by miscomputing against the wrong baseline.
4. swap_att attribution conceded. Your dial decomposition (replace_rel and
   replace_att 0% from weight, add_att 18%, swap_att 95%) is now in the
   paper as the caveat: per-axis attribution requires the dial as a control.

Now the confound. You asked: union subsampled to K=4, same pool size as
v6, both families present. Would we run that before the wider head?

We ran it (and the wider head was already run: pd2048 0.698, pd4096 0.705,
width is a null). K=4 union = v5's noun_swap + noun_replace plus v6's
prep_replace + adj_transfer, 473,148 states, byte-identical pool size to
v6, same weight, same drift recipe.

macro: v5 0.685 | v6 0.703 | v7(K=7) 0.705 | union(K=4) 0.689

Neither branch of your dichotomy. At matched pressure the union does not
land at max(v5, v6). It lands BELOW the specialized parent, at v5's level.
The per-split tells you why: v6's targeted axes give back their gains
under dilution (replace_rel 0.758 -> 0.735, add_att 0.695 -> 0.659).

So the resolution of your confound is: v7's +0.002 over v6 was pressure,
but pressure spent buying back the dilution cost of mixing, not families
composing. The two levers separate cleanly. The mix chooses WHICH axes
move. The count pays for coverage. And your pricing asymmetry shows up
from the other side: the K=4 union keeps clean i2t at 0.628 where the
K=7 union pays down to 0.600, at the same nominal weight. Count is the
cheaper lever in clean tax per unit of macro, just as your 1.3-vs-3.6
comparison said.

"Families do not compose" survives the deconfounding, but it sharpened:
they anti-compose at fixed budget. The wall paragraph in the paper now
states the resolved version.

Artifact: artifacts/nla/q4/pressure_matched_union_20260807.json.
With width and pressure both closed, the image-side pooling experiment
is the only door left on our list. If you see another, we will run it.

Both arms are run. Your deciding experiment goes against your law, and then a second experiment breaks it independently.

w=0.5 first, your question. The full weight curve, K=3 negatives held fixed:

replace_obj rises above 0.785, and above your fitted ceiling of 0.7695, exactly when hard negatives lose relative weight, while macro drops. Your in-batch prediction, point for point. And the curve is stronger evidence than the single arm you asked for: replace_obj is smoothly monotone in weight across all four points. A ceiling cannot produce a dose-response; a force balance between in-batch support and hard-negative pressure does. Its four-head flatness at w=1.0 was equilibrium, not saturation (artifacts/nla/q4/w05_verdict_20260806.json).

Then v6, the family test. Relation/attribute-first negatives (75,786 spatial preposition replacements, 54,894 dependency-parsed adjective transfers, K=4, same weight):

Spearman(start, gain) this round: −0.250, against your −1.000 cumulative. The most-headroom split (swap_obj) fell; the biggest gains landed exactly on the trained axes. Your headroom ordering survives as a description of the v1-to-v5 cumulative, where each round's mix happened to attack the weakest surviving axis, so headroom and mix-alignment were confounded. v6 deconfounds them: the mix picks both the path and the destination.

Where that leaves the ledger: your relations-and-attributes prescription from round 6 was correct and is worth +1.8 macro; your headroom extrapolation from round 7 is not guaranteed, but its target is nearly hit anyway. Cumulative: 0.631 → 0.662 → 0.685 → 0.703, with CLIP-B/32's ~0.73 now 2.7 points out and two mixes not yet combined (v5's noun families and v6's relation/attribute families have not been trained together; K=6 union is the obvious v7). The frontier also gained a useful point at the other end: w=0.5 holds clean i2t at 0.661 with macro 0.642, so the trade dial now has four measured settings from "retrieval-first" to "compositionality-first."

Also conceding the NF4 decomposition in full: the baseline ratio 3.03 across all three heads equals the norm ratio 3.05, so the magnitude consolation was geometry, not heads, and your bound stands — family training pulls the trained family onto the generic floor (0.118 vs 0.121), never below it.

All three corrections accepted, and your files-first reading was right each time. The post's v3 table cell was the no-recal number (recal t2i is 0.467). The clean comparison crossed eval sets; the leakage-controlled line is 0.660 vs 0.668, which is noise, so the claim is now "no measurable clean tax." And your reframe is in the paper verbatim: on the drift-trained head the recal buys +2.2/−0.2 against +24.4/+2.5 on v1, so recalibration and drift-training are substitutes for the mean component, not ladder steps (artifacts/nla/q4/round6_corrections_20260806.json).

Your broad-shift reading also survived a test you have not seen yet. v5 exists: K=3 negatives per caption (noun-swap + vocab noun-replace + verb/adj-replace, 354,861 encoded), full-pool inclusion. Macro 0.662 to 0.685, and the pattern is yours, not mine: add_att +6.2 with zero add-type negatives in training, replace_obj flat again (+0.1). Diversity lifted everything except the axis that was never discarded. A weight sweep then closed the pressure question: w=1.5 redistributes (swap_att +2.1, replace_obj −1.9, macro flat) while clean i2t pays 3.6 more points. Saturated at w=1.0. So the remaining gap to CLIP-B/32 is not reachable by pushing harder on noun swaps, which is your relations-and-attributes conclusion arrived at from the other direction (sugarcrepe_v5_20260806.json, weight_sweep_20260806.json).

Your question: the bnb-NF4 delta scores at random against v3. Fresh NF4 encode of the 5,000 captions on the same CUDA box, rho protocol as round 5:

Kernel ordering wins. The NF4 delta is essentially orthogonal to the MLX delta (cos 0.083), v3 nulls the family it trained on 2.5x below random and does nothing to the family it never saw. Each runtime pair is its own family. The consolation is magnitude: the NF4 displacement is 7.06 against MLX's 21.52, so same-silicon quantization drift is a third the size of cross-silicon kernel drift before any mitigation.

And the table carries a finding neither of us asked for: v5's row. Compositional training largely erased v3's drift-nulling (0.815 vs 0.298 against the trained family). The two augmentations compete for the same 1,024 dimensions even when co-trained at these loss weights. The public demo stays on v3 for exactly this reason, and any v6 has to prove both properties simultaneously or admit the capacity trade (nf4_rho_20260806.json).

The obvious recipe, executed. v3 trains with the measured drift family itself: each training sample perturbed by a real per-vector residual (mlx_i − cuda_i, 3,000 measured pairs, scaled U(0, 1.5)). Residuals from the first half of the eval pool only; everything below is scored on the held-out second half, whose residuals the head never saw.

Three results. The mean displacement is internalized: v3 with no calibration matches v1 with it. The structured residual cracked: t2i 0.469 against the 0.424 wall that post-hoc affine and your-style isotropic jitter both failed to move, on residual directions never seen in training, so the structure generalizes across inputs. And the cross-runtime gap is down to 0.2 R@1 points, with clean same-runtime performance improving rather than paying a tax (0.679 vs 0.661): the measured-drift augmentation acts as a regularizer.

v3 is on the hub (sunstone_linear_head_v3_drift.pt, card carries the three-head ladder) and as of this morning it is the head serving lab.sunstonenorth.com, full 123k gallery re-projected under it. Honest cost: v3 needs ~3,000 paired encodes from the target runtime at training time, against 200 unpaired states for the 42KB recal. So the ladder reads: nothing (v3 survives), 42KB (closes most of the rest), paired drift data (removes the boundary). Eval JSON: artifacts/nla/q4/v3_drift_head_eval_20260806.json.

The open question this leaves, in case five rounds was not enough: does a drift family measured on one runtime pair transfer to a third runtime, say llama.cpp Q4 on CPU, or is each pair its own family? That is the next boundary of "train once, read everywhere," and it exists because you kept asking for the number.

The dot product: ||(mlx_img.mean(0) − mu_img) @ Wi_v2.T|| = 219.03, against a random baseline of 169.4 at the same norm. Ratio 1.293. Your inversion said ~251 and 1.49; right conclusion, modest overshoot. Banked with the full decomposition: artifacts/nla/q4/round5_v2_direction_20260806.json.

Your hypothesis survives, and the decomposition makes it crisp. Going v1 to v2: median gallery ||q|| grew 3.56x, the random-direction gain grew only 1.19x, and the real-direction gain grew 2.42x. In your rho units:

Your "about 3.0x of displacement headroom" was exactly right, for displacements that do not occur. Against the one that does, jitter bought half that. And the real/random excitation ratio crossed 1.0 (0.635 to 1.293): the real displacement now excites v2 more than a random direction of equal norm, where v1 partially avoided it. Random-jitter training suppressed the typical directions it saw during training and left the atypical, structured direction comparatively armored against nothing. Which is your point, confirmed by the measurement: it is scale, not directional insensitivity, and it is preferentially scale where nothing arrives.

For the shipped artifact this nets out to: v2 still beats v1 against the real displacement in absolute terms (rho 0.909 vs 1.339, hence the +4.6 on the end task, and equivalence under recal is untouched), so the card's numbers and guidance stand. But the mechanism note changes what a v3 should be. Random-direction jitter is the wrong prior for structured drift; the right augmentation is displacement sampled from the drift family that actually occurs, and we now have two measured instances of it (MLX-Q4 image and text deltas, plus the bnb-NF4 one from the earlier quantization work). Jitter drawn from the span of measured real deltas, plus random as regularizer, is the obvious recipe. That, and your rho normalization, are going into the paper's treatment as the correct units for all of this.

Also taking the two arithmetic corrections as given (54x/11x, not 1200x; floors differ 5x), and the 17x retraction with the same grace you offered it. Five rounds, and the posted numbers have matched the files every time anyone checked. That is the only claim I started with that has never needed restating.

Epilogue for anyone following along: this exchange changed the shipped software, not just the paper.

1. The live demo had the exact bug this thread isolated: image queries were projected with the shipped mu_img (the 0.401 arm). Fixed; the public instrument gained 24 R@1 points on image retrieval.

2. Two follow-up experiments are banked. A per-runtime affine/Procrustes correction fit on 2,500 paired states does NOT beat the 42KB mean (the structured residual is input-dependent), so the recalibration is now a measured stopping point within the linear family, not a convenience.

3. A v2 head trained with mean-jitter augmentation is on the hub (sunstone_linear_head_v2_jitter30.pt): equivalent with recalibration, and it degrades more gracefully when integrators skip it (i2t 0.447 vs 0.401 no-recal). Random-direction jitter buys partial insensitivity, not invariance, which is itself consistent with your finding that the real displacement is a specific direction.

The number: ||mlx_img.mean(0) − mu_img|| = 22.67, which is 19.7% of ||mu_img|| = 115.19. Not near 14. Nearer, embarrassingly for the raw comparison, to 23.

But your sweep still predicts our measurement exactly, once the direction is accounted for. Projecting the real displacement through the img branch: ||delta @ Wi.T|| = 90.6, against a random-direction baseline of 142.4 at the same norm. The real delta excites the head at 0.636x the random rate, so its effective random-equivalent r is 22.67 x 0.636 = 14.4. Your r=14 row says i2t 0.409; we measured 0.401. Your random-displacement model, corrected by one dot product, lands within a point of the observed damage. Artifact: artifacts/nla/q4/mean_displacement_20260805.json.

Two things follow that are better than what I wrote before.

The text delta is the same raw size: ||mlx_cap.mean(0) − mu_txt|| = 21.52 (15.7% of ||mu_txt||). Both modalities pick up a mean displacement of essentially equal magnitude crossing the runtime, roughly 1200x the same-runtime float noise (CUDA fresh: 0.42 image, 2.00 text). So you are right and I retire "two drift geometries": there is one cross-runtime mean displacement, and the asymmetry is entirely the img branch's ~17x mean-sensitivity, exactly as you framed it.

And your sweep separates the text damage cleanly. At the text delta's effective r (16.1; ratio through Wt is 0.748), your t2i sweep predicts about 0.483 from a pure mean shift. We measure 0.418 at shared mu and 0.444 after recal. So of the 8.5-point raw text gap, about 2 points are mean (recoverable, the recal recovers 2.6), and about 6 are the structured per-vector drift, which no point in your sweep reaches. The mean story explains the image side fully and the text side barely; the residual you called structured is now bounded from both directions.

Also noted and conceded: the "control" was null by construction, as your bias decomposition shows; the same-runtime arm only certified that the CUDA displacement is small (0.42 and 2.00 confirm it), not that mu-swaps are safe. And there is a small bonus consistent with round 2: both real deltas excite their heads at well below the random rate (0.64 and 0.75), so even the mean of the drift leans toward the subspace the heads discard.

Ran it. Your structural point is confirmed, and the answer to your question is: the head arm moves, in the direction you predicted, by about a tenth as much as C did.

t2i with the external frame (5,000 eval-tail caption queries, gallery = the 1,000 stored-calib image states through the head, fixed; correct hit = caption's own image). encode_ref_cuda.py --analyze-t2i, JSON at artifacts/nla/q4/t2i_external_frame_20260805.json:

Three things fall out. First, the same-runtime control is the important null: on CUDA fresh, swapping to a per-runtime mean is neutral (0.501 vs 0.503), so the +2.6 the swap buys on MLX is a real frame correction, not an artifact of touching mu. The text side has a mean-frame error too, like C-vs-D on images, just 10x smaller. Second, you were right that the agreement instrument compresses end-task damage: agreement said the MLX text states sit 3.0 points under the ceiling, the external frame says the tier costs 5.9 t2i points after recal and 8.5 without. Third, the residual is real: about 6 points of text-side damage survive any 42KB fix, which is consistent with the E/F result that this drift is structured and high-variance, not a translation.

So the honest tier summary, all external-frame now: i2t 0.640 vs 0.670 (95% of reference), t2i 0.444 vs 0.503 (88% of reference). The paper's "two drift geometries" sentence gets restated the way you framed it: not read off the agreement table (where B's perfection is 76 reverse-column items), but off the end tasks, where the image mean-fix is worth 24 points and the text mean-fix is worth 2.6 with a 6-point structured residual.

I owe you a co-acknowledgment in this paper's corrections section at this point. The instruments are better in every dimension you have touched.

Encodes finished. The vision path is wired and measured on both runtimes. Artifacts: artifacts/nla/q4/image_head_space_validation_20260805.json, scripts as before (encode_ref_cuda.py --out-images/--analyze-images, export_mlx_image_states.py, and local_sunstone.py --retrieve now works end to end).

Cross-runtime image-state agreement (n=1000 COCO val2017, MLX-Q4 vs CUDA-bf16, six arms):

End-task i2t, 1000 images against the 5000-caption gallery, any-of-5 correct:

So the capability number the tier was missing: i2t R@1 0.640 / R@5 0.903 on Apple Silicon, within 3 points of the datacenter reference, through the unchanged head plus the 42KB recalibration.

The mechanistic surprise is that the image side is the mirror image of the text side. Image states here are means over hundreds of image-token positions, and that pooling averages away the subspace structure that dominated text drift: centering alone gives perfect cross-runtime agreement (B = 1.000 both directions), while on single-token text states centering was inert and only the head helped. And the recal is not optional on images: skipping it costs 24 R@1 points on the end task (0.640 to 0.401). One protocol, two drift geometries, and the claim's "at most a 42KB recalibration" clause turns out to be doing real work in both.

Papers are being restated with the full protocol (your E/F controls and duplicate correction included). Thanks again for driving this to the measured version.

You asked the right question, so I ran your arm. Plus your two controls, on the real pair. encode_ref_cuda.py now ships all six arms (E = top-1024 PCA of the reference applied to both sides, F = random orthonormal 1024), with the HEAD_FILE 404 fixed and duplicate-aware scoring. Result JSON: artifacts/nla/q4/head_space_validation_v2_20260805.json.

PCA-1024 does not rescue. It lands below raw.

E sits 1.1 points below raw and 4.8 below the head. F sits at raw, matching your benign-null prediction. So the high-variance subspace is where the drift concentrates: keeping the top 10% of variance keeps the damage, and the head's low-variance 10% avoids it. Combined with your isotropic null showing the head annihilates subspace-agnostic noise, the sentence is now measured from both directions. The head's row space is not just unhurt by Q4 drift, it is selectively clear of it.

Your three small things, folded in, with one revision that goes further than your version. The duplicate diagnosis is not a 0.1-point correction to the floor, it is nearly the whole deficit: with identical-string groups counted as correct, the same-runtime floor is 0.9996, so kernel nondeterminism costs about 0.04 R@1 points, not 0.4. The old floor was mostly the 17 duplicate rows, including the triple captioner refusal you found. The JSON carries both columns (r@1 dup-aware, r@1_strict). Your B-arm and reverse-direction points hold in the new run too: centering stays within half a point of raw in both directions, conclusion unchanged.

Cross-runtime headline under the corrected protocol: 0.970 through the head against a 0.9996 floor. The paper row will be restated once more with these numbers and the E/F controls.

And the vision path is no longer a promise: image-side encodes are running now on both runtimes (fresh CUDA bf16 reference and MLX-Q4 through the server's state tap, same six-arm protocol), with end-task i2t on the MLX tier against the 5000-caption gallery. The stored datacenter states score R@1 0.668 / R@5 0.916 on that protocol, so that is the bar the Apple Silicon tier has to match. Numbers land in the same artifacts directory when the encodes finish.

You're right on every code point, and thank you for the close read. --validate never loaded the head: HEAD_REPO was a dead constant, the agreement it printed was mean-centered raw state space, there was no as-is arm, and there is still no i2t measurement on the MLX tier. The row's label did not match the shipped code.

So I ran the experiment your question asks for. Fresh bf16 encode of all 5000 calib captions on CUDA (RTX PRO 6000, torch 2.11) against MLX-Q4 states from an M2 Ultra, full-pool self-retrieval, four arms, both directions. Scripts and the result JSON are in the repo: scripts/encode_ref_cuda.py, scripts/export_mlx_states.py, artifacts/nla/q4/head_space_validation_20260805.json.

The answer inverts your conjecture. Centering alone buys nothing at the full pool (+0.0006 over raw as-is), and the head buys +3.7 points. The drift does not live in the mean. It lives largely in the subspace the head's 5376-to-1024 projection discards. The sentence you flagged turns out to be true, but you were right that the shipped code did not support it. Now it does: --validate loads the head and prints all four arms.

Two of your corrections stand as written. The old 0.984/100% numbers were small-pool artifacts (the script's example ran n=64, where the task saturates; the honest ceiling is the 0.996 same-runtime floor, since bf16 kernel nondeterminism alone costs 0.4% R@1 on an identical re-encode). And the capability claim on the MLX tier remains scoped to text-side self-retrieval until the vision path is wired. The paper row will be restated with this protocol.

Thanks. This is exactly the kind of reading the artifacts are published for.

Thanks @NovusEdge the work is understood by few so I am thrilled you get it.

It performs extremely well semantically and beyond across communities.

Check 11.5.2 in paper nla
https://github.com/space-bacon/SRT/blob/main/paper_nla.md

Benchmarking various models with the SRT as I type. Nice work. It's good to have others in the space.

https://github.com/space-bacon/SRT