Both of these papers start from the same uncomfortable fact: the simulation you want to replace is also the only practical source of training data for its replacement. That circularity is why scientific ML for physical systems produces beautiful results on well-studied systems and stalls everywhere else.

The circular problem with surrogate models

One DFT relaxation gives you one data point. A neural operator wants thousands. On a material family that people have studied for thirty years, you can scrape those thousands out of the literature and public databases, and the surrogate works. On a family nobody has touched, you have fifty points and no obvious way to get more without spending a year of compute.

PYTHIA's authors say this out loud in the paper: the reliance on a high number of samples "limits its applicability to relatively explored material families." Their target is the unexplored end of that spectrum, where adsorption energies on defective surfaces have to be predicted for chemistries with almost no prior calculations.

RheOPT hits the same wall from a different direction. A non-Newtonian simulation has to advance velocity and stress together through rate- and history-dependent constitutive dynamics. The fluid remembers what happened to it, so cost grows with integration length on top of the usual mesh and timestep constraints. Training a surrogate to predict those fields requires exactly the simulations you were trying to avoid running.

Different physics, same move. Find a cheap source of supervision that shares structure with your target, learn that structure once, then spend your small budget on the difference.

PYTHIA: freeze the bulk, train the last mile

The pipeline is a foundation model, a frozen backbone, and two small trainable layers.

Two design choices carry the weight here.

The first is the choice of pretraining target. PYTHIA's foundation model was trained on a large set of chemically similar molecules, on a property that sits physically close to adsorption energy. That adjacency is the whole trick. If you bolt a frozen molecular encoder onto a target that lives in different physics, the frozen features describe the wrong thing and the alignment layer has to learn too much from too few labels.

The second is the split between the transformer alignment layer and the Gaussian process. The transformer handles the geometric and chemical remapping between the pretrained embedding space and the new task. The GP sits on top and gives you a variance estimate per prediction, which is what Bayesian optimization actually needs. A point-estimate surrogate plugged into an acquisition function will confidently explore regions where it is simply wrong. With calibrated uncertainty, the loop spends its expensive evaluations where the surrogate is guessing instead of where it is already accurate.

Freezing also changes the arithmetic of the budget. If the only trainable parameters are an alignment layer and a GP, the number of DFT points you need drops by an order of magnitude or more compared with fine-tuning the whole encoder. The paper tests the resulting framework on adsorption and defect optimization for two distinct material systems, and reports finding optimal adsorption sites efficiently in both, including cases with complex site geometry.

Quick Take: The cheapest supervision in physics ML is usually not more simulation, it is a nearby problem you already know how to solve.

RheOPT: transfer the plumbing, not the fluid

RheOPT's insight is subtler, and I think it is the more useful of the two for anyone working on flow.

The obvious thing to transfer from Newtonian to non-Newtonian simulation is the constitutive response, the mapping from strain rate to stress. RheOPT explicitly does not do that, because that mapping is precisely what changes when you add viscoelasticity or a yield stress. What survives the change of fluid is geometry-conditioned transport structure: how the flow negotiates a contraction, where recirculation cells form, how the wake behind an obstacle organizes.

The evaluation covers two geometry-matched settings. The first is a 4:1 contraction channel, where the fluid accelerates fourfold into the narrow section. That ratio is not cosmetic: strong extensional stress builds up right at the corner, and that is exactly where Giesekus, Oldroyd-B, and Bingham fluids disagree with each other most. The second setting is the oscillatory wake behind a triangular obstacle in Oldroyd-B flow, a case where the interesting physics is unsteady.

Key numbers

• 4 target non-Newtonian samples were enough for the fine-tuned contraction model to match or exceed a from-scratch model trained on 16.
• 3 constitutive models tested in the contraction: Giesekus, Oldroyd-B, Bingham.
• 2 evaluation geometries: 4:1 contraction and triangular-obstacle wake.
• 1 Newtonian pretraining corpus, reused across all target fluids.

The numbers, and what they cost you

The headline result is a data-efficiency ratio. Four target samples matching a sixteen-sample baseline means the pretraining step bought roughly a 4x reduction in expensive simulations. In practical terms: if a single non-Newtonian run in your setup costs a day of cluster time, that is the difference between a two-week campaign and a two-day one.

One caveat about that ratio. It comes from a single geometry and a specific family of constitutive models. Transfer ratios are the kind of number that looks portable and isn't. Treat 4x as evidence that the mechanism works, not as a constant you can plug into a project plan.

The wake experiment carries a second lesson that has nothing to do with sample counts. Relative L2 error looked acceptable while the model had flattened out the vortex shedding entirely. A global field error metric cannot distinguish a correct oscillatory wake from a time-averaged mush that happens to sit near the mean. Phase-insensitive diagnostics, specifically shedding frequency and wake fluctuation amplitude, caught what the L2 number missed.

PYTHIA and RheOPT side by side

	PYTHIA	RheOPT
Pretraining source	Foundation model on chemically similar molecules	Cheap Newtonian simulations
What is frozen	Foundation model weights	Pretrained operator backbone
What is trained	Transformer alignment layer plus GP	Fine-tune on target fluid
Uncertainty	Gaussian process per prediction	Not the focus of the paper
Downstream use	Bayesian optimization over sites and defects	Direct field prediction
Data budget	Limited target dataset, stretched by the BO loop	4 target samples matched a 16-sample baseline
Validation	Two material systems	4:1 contraction, triangular wake
Main failure mode	Pretraining target too far from the prediction target	Stress dynamic range far from the pretraining corpus

The structural difference matters when you pick one. PYTHIA is a decision-making framework. It exists to answer "which site do I calculate next," and the uncertainty layer earns its place because the acquisition function consumes it. RheOPT is a field predictor. Its output is a solution, and the fine-tuning step is what makes that solution trustworthy for a new fluid.

The normalization detail that decides the outcome

This is the part of RheOPT I would pin to a wall.

The transfer benefit held reliably under reversible instance normalization (RevIN), where inputs and outputs are normalized per sample and the transform is undone at inference. Under global normalization, using dataset-wide statistics, the benefit became dataset dependent. The authors trace that to the dynamic range of each fluid's velocity and stress fields.

Think about what that means in practice. Oldroyd-B stress can span orders of magnitude across a domain. Bingham has a yield surface with near-discontinuous behavior. If you normalize with statistics pooled over all your fluids, you compress the high-stress regions of one fluid and inflate the low-stress regions of another, and the pretrained features no longer line up with the target. Per-instance normalization keeps each sample on a comparable scale, so the frozen structure stays meaningful.

My take: RevIN is not a hyperparameter you tune after the fact. If your target fluids have wildly different stress ranges, per-instance normalization is a precondition for transfer to work at all.

What the community is saying

When I ran a frozen-backbone experiment on a porous-media flow problem, the thing that broke first was normalization, not capacity. I had a pretrained encoder that transferred fine on one permeability range and produced garbage two orders of magnitude away, and it took me a week to realize the pooled statistics were the culprit rather than the frozen weights. Hearing that RheOPT hit the same wall and traced it to dynamic range made me feel considerably less stupid.

The other complaint I keep hearing from people doing this work is about validation. Everyone has been burned by a surrogate with a good average error and the wrong physics. My team lost a month to a model that predicted mean pressure drops accurately and got the oscillation entirely wrong, which is exactly the failure RheOPT's phase-insensitive diagnostics are designed to catch. If you build surrogates for anything unsteady, add a spectral check to your eval script before you add another layer to the network.

Common pitfalls

1. Validating with a single global error metric. Relative L2 can look fine while the model has averaged away the unsteady behavior you care about. Check spectra, phase-averaged statistics, and fluctuation amplitudes before you trust a field predictor.

2. Pretraining on a physically distant target. PYTHIA's foundation model was trained on chemically similar molecules for a property close to adsorption energy. Swap in an encoder pretrained on different chemistry and the frozen features stop being informative, which pushes all the learning onto a small alignment layer with too few labels.

3. Using global normalization with fluids that have very different stress dynamic ranges. RheOPT's gain was reliable under RevIN and inconsistent under global normalization. Check the range of your fields before assuming the transfer holds.

4. Fine-tuning the whole backbone on a handful of samples. Freezing is the mechanism, not an optimization convenience. With four target samples, updating the full network destroys the pretrained structure and you are back to from-scratch performance with extra steps.

5. Dropping uncertainty quantification out of the optimization loop. A surrogate without calibrated variance will send Bayesian optimization into regions where it is confidently wrong. The GP layer in PYTHIA is doing real work, and swapping it for a point estimate changes the search behavior, not just the reported error bars.

One thing to remember

Both frameworks depend on a domain-adjacency assumption that neither paper fully characterizes: PYTHIA needs a pretraining target "physically close" to the prediction target, and RheOPT needs a source fluid whose transport structure survives a change of constitutive law. That assumption is where the method will succeed or quietly fail on your problem, and it is the one thing you cannot check from a benchmark table. Run a small pilot on your easiest target case and look at whether the frozen features help before committing a compute budget.

The bottom line

You have a novel material family and a few hundred DFT evaluations to spend: adopt the PYTHIA pattern, freeze a foundation model trained on nearby chemistry, and put a Gaussian process on top so the Bayesian optimization loop knows where it is guessing. The uncertainty layer is what turns a surrogate into a decision tool.

If your target fluid's stress field spans orders of magnitude beyond anything in your pretraining corpus, do not assume the Newtonian transfer will hold. Switch to per-instance normalization, and if the gain is still inconsistent, budget for training from scratch on more target samples rather than fighting the transfer.

One thing to watch: nobody has published a map of which source domains transfer to which targets. The next useful paper in this area is not a bigger model, it is a systematic study of adjacency, and I would expect that within a year given how many groups are now running the freeze-and-align pattern on flow and materials problems alike.