evidence boundary
unknownfrontiers / frontier
AI-for-science benchmark state
- id
- vfr_efc649fd772a1ff1
- license
- CC-BY-4.0
- findings
- 12
- accepted core
- 12
- contested
- 0
- links
- 0
- sources
- 1
- evidence
- 12
- avg conf
- 0.30
e24/24 · finding.noted · reviewer:will-blair · 2026-06-10 · 6c12→d02f
Evidence atom
back to sourcesLEAKAGE HAZARD (ProteinGym). PLMs trained on UniProt may have seen sequences related to the assay proteins; 'zero-shot' is zero-shot on the LABELS, not necessarily on the SEQUENCES. VERIFICATION STATE: training-sequence overlap with assay proteins is under-audited. Open obligation: a sequence-similarity leakage audit between each model's training set and the assay proteins before banking any SOTA claim.
- id
- vea_874a2c26d4e672f2
- frontier
- AI-for-science benchmark state
- source
- vs_066123dd29a9c5b4
- finding
- vf_8212daf3d7034a93
finding binding
boundcomputational
LEAKAGE HAZARD (ProteinGym). PLMs trained on UniProt may have seen sequences related to the assay proteins; 'zero-shot' is zero-shot on the LABELS, not necessarily on the SEQUENCES. VERIFICATION STATE: training-sequence overlap with assay proteins is under-audited. Open obligation: a sequence-similarity leakage audit between each model's training set and the assay proteins before banking any SOTA claim.
source binding
source-boundmanual finding
vs_066123dd29a9c5b4
review context
unverified2 events
2 reviewable changes and 0 evaluation records target this atom or its bound objects.
statement
LEAKAGE HAZARD (ProteinGym). PLMs trained on UniProt may have seen sequences related to the assay proteins; 'zero-shot' is zero-shot on the LABELS, not necessarily on the SEQUENCES. VERIFICATION STATE: training-sequence overlap with assay proteins is under-audited. Open obligation: a sequence-similarity leakage audit between each model's training set and the assay proteins before banking any SOTA claim.
extraction method
manual_curation
support relation
unknown
condition refs
vcnd_f640e1e86c8fd161
caveats
- missing evidence locator
Review, event, and evaluation records
4events
vev_07e40e45981061adfinding.assertedManual finding added to frontier state
reviewer:will-blair · 2026-06-10
vev_c4da9db8be63634efinding.notedHARDENING (benchmark-state): label_provenance=attested (records-not-reruns; ground truth is an answer key, not a frozen-verifier rederivation), valid_as_of=2026-06-10, model_cutoff=unknown. Under the trust ladder, attested label provenance caps this record below 'verified' until a deterministic rederivation exists.
reviewer:will-blair · 2026-06-10
reviewable changes
vpr_c555bef607043399finding.addManual finding added to frontier state
applied · reviewer:will-blair · 2026-06-10
vpr_fd307d5d15c2cef7finding.noteHARDENING (benchmark-state): label_provenance=attested (records-not-reruns; ground truth is an answer key, not a frozen-verifier rederivation), valid_as_of=2026-06-10, model_cutoff=unknown. Under the trust ladder, attested label provenance caps this record below 'verified' until a deterministic rederivation exists.
applied · agent:hardening-2026-06-10 · 2026-06-10
evaluations
No evaluation rows are attached.