evidence boundary
unknownfrontiers / frontier
AI-for-science benchmark state
- id
- vfr_efc649fd772a1ff1
- license
- CC-BY-4.0
- findings
- 12
- accepted core
- 12
- contested
- 0
- links
- 0
- sources
- 1
- evidence
- 12
- avg conf
- 0.30
e24/24 · finding.noted · reviewer:will-blair · 2026-06-10 · 6c12→d02f
Evidence atom
back to sourcesBENCHMARK CLAIM (ProteinGym) — ESM-1v REPORTS strong zero-shot substitution performance from masked-marginal scoring of a protein language model. VERIFICATION STATE: author-reported; weights public; depends on the scoring convention (masked-marginal vs wt-marginal) and the ProteinGym version. NOT re-run here. Open obligation: re-score the released model on the pinned v1.1 zero-shot substitution set.
- id
- vea_a9c4e7b494465d60
- frontier
- AI-for-science benchmark state
- source
- vs_066123dd29a9c5b4
- finding
- vf_03776d6cd3e0801b
finding binding
boundcomputational
BENCHMARK CLAIM (ProteinGym) — ESM-1v REPORTS strong zero-shot substitution performance from masked-marginal scoring of a protein language model. VERIFICATION STATE: author-reported; weights public; depends on the scoring convention (masked-marginal vs wt-marginal) and the ProteinGym version. NOT re-run here. Open obligation: re-score the released model on the pinned v1.1 zero-shot substitution set.
source binding
source-boundmanual finding
vs_066123dd29a9c5b4
review context
unverified2 events
2 reviewable changes and 0 evaluation records target this atom or its bound objects.
statement
BENCHMARK CLAIM (ProteinGym) — ESM-1v REPORTS strong zero-shot substitution performance from masked-marginal scoring of a protein language model. VERIFICATION STATE: author-reported; weights public; depends on the scoring convention (masked-marginal vs wt-marginal) and the ProteinGym version. NOT re-run here. Open obligation: re-score the released model on the pinned v1.1 zero-shot substitution set.
extraction method
manual_curation
support relation
unknown
condition refs
vcnd_bb72edd2de97ec48
caveats
- missing evidence locator
Review, event, and evaluation records
4events
vev_bb98a228e1f4a5a7finding.assertedManual finding added to frontier state
reviewer:will-blair · 2026-06-10
vev_eb4221b9d34b54cdfinding.notedHARDENING (benchmark-state): label_provenance=attested (records-not-reruns; ground truth is an answer key, not a frozen-verifier rederivation), valid_as_of=2026-06-10, model_cutoff=unknown. Under the trust ladder, attested label provenance caps this record below 'verified' until a deterministic rederivation exists.
reviewer:will-blair · 2026-06-10
reviewable changes
vpr_7d99f50cb897566dfinding.addManual finding added to frontier state
applied · reviewer:will-blair · 2026-06-10
vpr_e4bebe83c4ec21ebfinding.noteHARDENING (benchmark-state): label_provenance=attested (records-not-reruns; ground truth is an answer key, not a frozen-verifier rederivation), valid_as_of=2026-06-10, model_cutoff=unknown. Under the trust ladder, attested label provenance caps this record below 'verified' until a deterministic rederivation exists.
applied · agent:hardening-2026-06-10 · 2026-06-10
evaluations
No evaluation rows are attached.