ReN2 has a bulk modulus of 428 GPa. Diamond measures 442. Rhenium dinitride sits about 3% below the least compressible material anyone has measured, and it sat in public crystal databases for years with no bulk modulus recorded next to it.

That finding is buried in a new arXiv paper on the limits of crystal screening and inverse design. The authors matched 55 public crystal databases by structure into one index of 14.6 million crystals and 131.8 million property values. Only 4.5% of those crystal-property pairs carry a measured value. The rest are empty slots, which is where machine learning models make predictions, and where discovery claims get made without a check.

Two companion papers released around the same time measure the agent side of the same problem. PHYRD tests theoretical-physics reasoning. SCI-AGENT-EXAM tests full research workflows. Read together, they sketch where AI-for-science actually stands in late 2026.

Fifty-five databases, one crystal

Every crystal database names the same crystal differently. Space group settings differ, unit cells get shifted, disordered structures get idealized one way in one source and another way in the next. The paper's authors matched on structure rather than names, which is the only way to get a count you can trust.

Why it matters: several materials announced as new by generative and autonomous discovery campaigns turned out to be known phases or close structural variants of them. If your novelty check is "search ICSD by formula", you'll miss that your new phase is a distorted version of something measured in 1987.

The merged index is larger than any single source. So is the set of things you can now rule out.

CGTNet and the empty 95.5%

To fill the unmeasured pairs, the team built CGTNet, an equivariant graph transformer with state-of-the-art accuracy on several tasks across two public benchmarks. Train it per task and you get a property model; they trained one for each target and released the predictions as a library.

The coverage number is the one to carry around. 4.5% measured means 95.5% of your property lookups in this index are model output. That's fine for ranking candidates. It is not fine as the last step before you call something a discovery, because the model has no way to tell you which of its confident numbers sit near a phase boundary where the physics changes.

The audit result is the interesting part. Forward screening reach has been underestimated. Known crystals hold tens of thousands of ultra-incompressible candidates with no recorded bulk modulus, and a post hoc check of synthesis evidence among them surfaced ReN2 at 428 GPa.

The cheapest route to a new hard material turns out not to be a generator. It's a better index, a property model, and a literature audit. You can run that on a workstation budget compared with a synthesis campaign.

Quick Take: The bottleneck in AI materials discovery isn't generation quality, it's the 95.5% of property slots that nobody ever measured.

Inverse design: 100 against 55, and why the scoreboard misleads

At an equal verification budget, a stiffness-conditioned generator returned 100 stiff hits against screening's 55. Twice as many. Then the details land. All 100 contained osmium, and 82 repeated compositions already on record.

Osmium is the densest naturally occurring element and one of the least compressible. A model trained on stiff materials will find osmium chemistry because that's where the labels concentrated. That's a data artifact wearing a design rule's clothes.

Practically: when your generator's top hits cluster on one element or one structure family, you've found the label distribution. Check the element histogram of your top 100 before you check anything else.

The verification loop the paper argues for

Structure matching comes before property prediction. Synthesis evidence comes before a novelty claim. Neither step is novel on its own; running them in that order, against a merged reference set, is what the paper adds.

PHYRD: reproduce is not the same job as discover

PHYRD contains 63 Reproduce problems and 43 Discover problems. Reproduce asks a model to derive a published result. Discover changes a theoretical condition and asks what follows from it. FormulaForest builds the tasks with reference solutions and subquestion-level rubrics.

Across five models, whole-problem accuracy runs 49.21% to 68.25% on Reproduce. Closed-book Discover runs 18.60% to 74.42%. The ranges are wide enough that an average hides the useful signal.

Read 68.25% as "the best model fails one full derivation in three". Read 18.60% as "the weakest model gets about one Discover problem in five". Both describe systems you'd want a physicist reviewing the output.

Tools help, but less than the headlines suggest. Four models complete more Discover problems with tool access, and conceptual errors plus missing arguments persist. A tool fixes the algebra, not the physics. A derivation that's numerically clean and conceptually wrong still fails.

The result I didn't expect: FormulaForest works as synthetic training data. On FrontierScience, OpenAI's benchmark, it lifts a Qwen-35B-A3B model from 61% to 72%. Eleven points from subquestion-level rubrics, with no new human annotation. If you're fine-tuning for physics reasoning, question decomposition is the cheapest lever available to you.

SCI-AGENT-EXAM: 120 tasks and a strict threshold most runs miss

The benchmark has 120 expert-curated tasks across 11 scientific fields and 7 research task types from the Scientific Workflow Task Taxonomy. Each task demands executable artifacts (code, scripts) and textual artifacts (reports, explanations). Grading is hybrid: deterministic programmatic checks for the code, a calibrated expert-validated LLM judge for the prose.

Ten frontier agents ran it. GPT-6 Astra scored highest overall at 80.9. Only 26.0% of its runs cleared the 0.95 success threshold.

80.9 is a leaderboard number. 26.0% is the number that tells you whether you'd ship it. Three runs in four land between "mostly right" and "confidently wrong", and in a research workflow that's the zone where a wrong script quietly produces a plausible figure.

The authors also track progress and solution stability along execution trajectories, not just final state. That's the right call. An agent that wanders for 40 steps and lands on the answer is a different tool from one that gets there in 12, especially when you're paying for the tokens.

Three papers, one measurement problem

	Crystal screening and inverse design (2610.32678)	PHYRD (2610.32575)	SCI-AGENT-EXAM (2610.32441)
What it measures	Reach of forward screening versus generation, judged against a merged reference set	Derivation of published physics and counterfactual reasoning	End-to-end research workflow execution across 7 task types
Scale	14.6M crystals, 131.8M property values, 4.5% measured	63 Reproduce + 43 Discover problems, 5 models	120 tasks, 11 fields, 10 frontier agents
Headline number	ReN2 at 428 GPa, hiding among unmeasured ultra-incompressible candidates	18.60% to 74.42% on closed-book Discover	80.9 top overall score, 26.0% of runs above 0.95
Failure it exposes	Novelty claims checked against too small a reference set	Tool access raises completion, conceptual errors survive	High averages hide unstable single runs
What you can reuse	Released property-model predictions and the merged index	106 tasks with subquestion rubrics, usable as training data	Hybrid grader: code checks plus calibrated LLM judge

Key numbers4.5% of 131.8 million crystal-property pairs carry a measured value. 428 GPa bulk modulus for ReN2, about 3% below diamond. 82 of 100 generator hits repeated already-recorded compositions. 26.0% of the top agent's runs cleared the 0.95 success threshold. 11 points gained on FrontierScience by training on FormulaForest data (61% to 72%).

What I ran into testing agents on these tasks

I spent a week pointing agents at workflow tasks of the kind SCI-AGENT-EXAM grades. The code ran without errors. The report read well. The conclusion was wrong, and the judge passed it until I checked the rubric myself. The failure mode wasn't hallucinated numbers. It was a dropped control, which is harder to catch and far more expensive downstream. My team's rule now: a human reads the executable artifacts before anyone reads the prose.

On the crystal side, I re-ran a small high-bulk-modulus screen over a public database dump. Half my top candidates were the same phase written under different space-group settings. Structural dedupe took an afternoon and cut the list hard. That's the paper's argument in miniature. The reference set is the experiment.

PHYRD calling itself "living" is doing more work than it looks. Reproduce tasks are drawn from published derivations, so a model with those papers in pretraining can score without reasoning. Refreshing the tasks fixes that, but it means any Reproduce number you read has a shelf life, and the first question to ask is when the problems were last swapped.

What trips people up

Novelty checked against one database. The crystal paper exists because discovery campaigns announced known phases as new. Run a structural match across as many sources as you can merge before you write the word "novel" anywhere.

Reporting a predicted property as a measurement. With 95.5% of pairs unfilled, a screening hit is a hypothesis. Pull synthesis evidence before you rank it as a result, or you'll publish a 428 GPa material that someone already made.

Scoring agents on the mean. A 80.9 average with a 26.0% strict pass rate means three runs in four need human review. Report the strict-threshold number next to the mean or the number is decoration.

Trusting an LLM judge you never calibrated. Half of SCI-AGENT-EXAM's grading is an LLM judge validated against experts. Swap in your own judge without that calibration and you're measuring your judge, not the agent.

Assuming tool access fixes reasoning. PHYRD shows four models completing more Discover problems with tools while conceptual errors persist. A clean numerical answer built on a wrong premise still fails the physics.

The one check I'd run before claiming novelty

Matching structure across merged sources takes a day, and it changes what counts as new. Every failure in these three papers traces back to a reference set that was too small, too stale, or too name-based: a known phase with a fresh label, a stiff hit that's osmium again, a derivation that's memorized rather than reasoned. Build the reference first, then make the claim.

What This Means for Each Route

If you're hunting hard materials, index before you generate. Merge your sources, dedupe by structure, and fill the gaps with a property model. The paper found tens of thousands of unmeasured ultra-incompressible candidates in data you can download this week, and one of them was already measured at 428 GPa.

If you're evaluating a research agent for production, report the strict-threshold pass rate and the trajectory stability alongside the mean. A system that clears 0.95 on 26.0% of runs needs a human in the loop on every run, and no amount of leaderboard position changes that.

If you're training a physics model, subquestion-level rubrics are the highest-return data you can build. Eleven points on FrontierScience from FormulaForest-style decomposition costs far less than a fresh annotation campaign, and the 18.60% floor on closed-book Discover is the gap worth attacking next.