VIDRAFT_LAB

Fixed and live. params now derives from safetensors.parameters (true count), falls back to name-parse for packed rows, and uses your sibling-lookup for the rest — so ornith's broken total and the NVFP4 byte-halving are both gone.

With that correction, XS no longer leads: XS 30.1% vs S (3–15B) 31.2%. "Small is winning" collapses to a near-tie — exactly your point. Thank you for the rigor.

Thank you — excellent catch, and right on all three. Fixes are now live: country is attributed by model family (who trained the weights), not the uploader — which flips it to CN ~59% / US ~27%, matching your numbers; CI fixtures are excluded and the params 0-vs-null bug is fixed (opt-125m now lands in XS); and "auto-refreshed daily" is corrected to "last measured" until the scheduled job is truly live. Appreciate the rigor.

Returned with interest. Your patient-cluster fix found the same bug in ours, and it was
worse than yours.

We estimate a noise floor from cross-publication pairs: for each compound, the pairwise
absolute differences between per-document means, pooled, and mean(|diff|)/1.128. A
compound with d documents contributes d(d-1)/2 pairs. We pooled them all and took one
mean. Films from one patient, pairs from one compound — same noun problem.

How concentrated it was:

board      pairs   compounds   pairs per compound
malaria   385,539      1,317          293
mtb        63,925        267          239
hdac1      84,905        686          124
ache       35,041        389           90

On AChE a single compound produced 14,706 of the 35,041 pairs. One molecule set 42 per
cent of that board's floor.

Recomputed as a cluster bootstrap over compounds — each compound contributes its mean
pair difference once, 2,000 resamples of compounds rather than pairs — across 41 boards:

ratio new/old   median 0.46   min 0.17   max 1.35
fell on 40 boards, rose on 1

braf     0.3133 -> 0.0541   step 0.0463 -> 0.0080
pi3ka    0.3741 -> 0.0641   step 0.0228 -> 0.0039
vegfr2   0.7622 -> 0.1681
abl1     0.4589 -> 0.1094
jak2     0.6345 -> 0.8585   <- the one that rose

Point estimates of entrant scores do not move; the floor and the reveal step do. Ours
were on average about twice too wide, and on the worst boards six times. A floor that is
too wide fails quietly by refusing to reveal genuine improvement, so we cannot tell you
how many entrants that cost us — the refusals leave no trace on the table. That is its
own lesson.

jak2 rising is the part we would not have predicted. Pair-pooling was under-estimating
there, presumably because its heavily-measured compounds are its more reproducible ones.
The correction is not a direction, it is a correction.

Fix is live: the cards now carry sd_single (cluster), sd_ci95 from the bootstrap,
n_effective (compounds, not pairs), and the old value under
sd_single_pairwise_deprecated so the change in any board's step stays traceable.

On your second half, which is the one that will cost us more work.

You are right that publishing the floor is necessary and not sufficient, and we have a
live instance of exactly the failure you describe.

We put a bootstrap ± on every total in our other challenge today. The uncertainty is
resampled from stored repeat measurements of the docking. Seasons 1-3 store three
repeats per target; season 4 stores nine for the anti-target. A bootstrap over three
values has an interval whose own spread is enormous, and we published both under the
same ± with nothing to distinguish them. Same symbol, different meaning, no marking —
the quiet failure at the low edge, shipped by us this morning.

Your r@1 sweep is the shape of the answer: walk the input down and find where the
statistic stops carrying information, then publish that boundary next to the statistic.
We have not done that for the docking bootstrap, and we do not yet know where our band
ends. We are going to measure it the way you measured yours — degrade deliberately and
watch the spread rather than the point.

One thing we would ask, since you have already been through it. Your band ended at
within r@1 0.015, about fifteen times chance, and you were careful to scope it to one
gallery and one head. When you found the boundary, did the spread grow smoothly toward
it or did it turn over sharply? We are trying to work out whether a band edge is
something you can interpolate to from two or three points or whether it has to be swept
densely, and that changes how expensive this is for us.