A Comprehensive Review of Different Surrogate Architectures

TL;DR

• In our first blog, we explored why conventional surrogates struggle when the mesh or geometry changes. The second introduced neural operators and how they learn physical fields across different discretizations.
• In this post, we compare nine architectures. They represent and exchange information differently, and each approach makes assumptions about the underlying physics.
• Understanding those choices tells us what use case a model is likely to work well for, what its limitations are, and what it needs to predict a new design.
• In an upcoming post, we will evaluate a selection of these models against identical engineering benchmarks using Curlscape Sift, the platform that we built to train surrogate models.

The common pattern

Many point-based neural operators follow three stages: lifting, information exchange, and projection.

• Lifting transforms input features into a higher-dimensional representation.
• Information exchange allows the model to capture relationships across the domain
• Projection converts the learned features into physical quantities such as velocity and pressure.

The lifting and projection stages typically use weights shared across points. The model can therefore process different numbers of evaluation points without changing those weight dimensions.

The main architectural difference lies in how information is exchanged.

Consider a CFD case with 400,000 evaluation points. Each point has seven input features: coordinates (x, y, z), wall distance (sdf), and a three-component wall-direction vector (Nx, Ny, Nz).

The predicted outputs are velocity components (Ux, Uy, Uz) and pressure (p).

This gives an illustrative input shape of (1, 400000, 7) and output shape of (1, 400000, 4).

Not every architecture uses this representation. FNO operates on structured grids, conventional DeepONet uses a fixed-dimensional branch input, and graph networks process nodes and edges. The interactive explorer shows these differences in detail.

Why information exchange matters

In steady incompressible flow, pressure is coupled across the domain through a Poisson equation:

∇²p = −ρ (∂uᵢ/∂xⱼ)(∂uⱼ/∂xᵢ)

A change in one region can therefore influence the solution elsewhere. A surrogate predicting steady flow needs some way to represent these long-range relationships.

Full attention between every pair of points would be one approach. But with 400,000 points, this requires 1.6 × 10¹¹ attention scores. A single half-precision score matrix would occupy approximately 320 GB.

Memory-efficient attention can reduce storage requirements, but the quadratic arithmetic cost remains.

Different architectures address this problem in different ways.

Graph networks exchange information through local neighbors. FNO uses Fourier modes. DoMINO processes geometry through a structured latent representation, while Transolver uses learned slices and AB-UPT uses sampled spatial anchors.

These approaches reduce the need for direct point-to-point attention, but each introduces different trade-offs in computation and physical representation.

1. Physics-Informed Neural Networks (PINNs)

PINNs incorporate governing equations directly into training.

For a fluid-flow problem,

• A network receives spatial coordinates and predicts quantities such as velocity and pressure.
• Automatic differentiation provides the derivatives needed to evaluate the governing equations, and the loss penalizes violations of those equations and the boundary conditions.
• A conventional PINN does not explicitly exchange information between points during its forward pass. The governing equations constrain the learned solution through optimization.

The trade-off: Conventional PINNs generally learn a solution for a particular problem configuration. Changing the geometry or boundary conditions often requires further optimization, while stiff and high-Reynolds-number problems can be difficult to train.

They are useful for inverse problems, sparse observations, and cases where the governing equations provide information beyond the available data.

In Curlscape Sift, we use a related idea through an optional physics loss on data-driven operators. Local estimates of field gradients allow us to penalize constraints such as continuity while retaining supervised training.

2. Deep Operator Networks (DeepONet)

DeepONet separates the input function from the locations where predictions are evaluated.

• A branch network processes the input function and produces coefficients.
• A trunk network takes a spatial coordinate and produces basis-function values. Their outputs are combined to reconstruct the field at that location.
• The same input-function representation can therefore be evaluated at different spatial points.

The trade-off: Conventional DeepONet expects a fixed-dimensional branch input. This works well when a consistent set of parameters or sensor measurements describes the problem, but can become restrictive when geometry or topology changes substantially.

DeepONet is useful for parametric studies on relatively fixed domains and for reconstructing fields from sparse measurements. PhysicsNeMo also includes DeepONet variants.

3. Fourier Neural Operator (FNO)

The Fourier Neural Operator exchanges information globally through Fourier-space operations.

• Each spectral layer transforms the latent field into the frequency domain.
• Applies learned transformations to selected Fourier modes, and transforms the result back into physical space.
• The learned spectral weights depend on the retained modes rather than the number of grid points, allowing evaluation at different grid resolutions.

The trade-off: Standard FNO operates on structured grids. Unstructured CFD data generally needs to be mapped onto a compatible representation, while retaining a limited number of Fourier modes restricts the spectral information available to the global operator.

Fine geometric features and sharp gradients can therefore be challenging, particularly when the grid itself does not resolve them.

FNO is well suited to regularly discretized problems, including weather modeling and periodic flows. Extensions such as Geo-FNO address more complex geometries.

4. Graph Neural Networks (GNNs)

Graph networks represent a physical domain using nodes and edges, often taken directly from mesh connectivity.

• Each layer computes messages along edges, aggregates them at neighboring nodes, and updates the node features.
• Aggregation operations such as summation preserve permutation equivariance.
• This resembles the local information exchange used in finite-volume discretization.

The trade-off: Information propagates through graph neighborhoods. Capturing long-range physical relationships may require additional message-passing layers or multiscale graph structures.

GNNs are particularly useful when mesh connectivity provides a meaningful representation of the physics and local interactions dominate the problem.

5. MeshGraphNet

MeshGraphNet builds on graph message passing through an encode–process–decode architecture.

• The encoder converts node and edge features into latent representations.
• A processor exchanges information across the graph, and a decoder produces the predicted physical quantities.
• In transient simulation, the model predicts physical state changes over a time step to iteratively generate the next step.
• Training strategies such as noise injection improve robustness against error accumulation during repeated predictions.

The trade-off: Graph construction and edge processing contribute to computational cost, while long-range interactions can require deeper or multiscale communication.

MeshGraphNet is useful for transient flows and deforming meshes, although graph-based architectures can also be applied to steady-state problems.

6. DoMINO

DoMINO combines global geometric context with local neighborhood information.

• DoMINO integrates broad spatial context with local geometric details.
• First, convolutional layers process the overall geometry after it is mapped into a structured latent space.
• Next, targeted neighborhood queries gather fine-grained spatial information around designated prediction points.
• This dual strategy enables the network to blend macro-scale domain features with localized structural dynamics.
• In external aerodynamics applications, for instance, global vehicle geometry dictates primary flow patterns, whereas minor surface contours govern localized separation and pressure gradients.

The trade-off: Grid resolution, domain bounds, neighborhood radii, and sampling determine which geometric scales the model can represent effectively.

A coarse grid may capture the overall geometry but rely on local processing for finer details. Different geometry families may also require different configurations.

DoMINO is particularly relevant to large-scale external-aerodynamics applications.

7. Transolver

Transolver reduces attention cost through learned physical slices.

• Points are dynamically mapped to a compact set of learned latent slices using adaptive weights, avoiding direct point-to-point attention.
• Each slice consolidates feature data from its assigned points; attention mechanisms operate exclusively across these slices before projecting updated information back to individual point features.
• Because these slices are learned partitions rather than rigid spatial grids, spatially distant points sharing physical characteristics can share common latent spaces.
• By holding the slice count constant, point-to-slice aggregation scales linearly with the total point count, whereas inter-slice attention remains quadratic relative to the fixed slice dimension.

The trade-off: Compressing a large physical domain into a limited number of slices creates an information bottleneck. Local variations may be weakened if the learned assignments do not represent them adequately.

Transolver is useful for field prediction on irregular point sets where global information exchange is needed without a structured grid or explicit mesh connectivity.

8. GeoTransolver

GeoTransolver extends Transolver by introducing explicit geometry-aware context.

• Instead of depending solely on the initial point representations, it generates contextual representations from domain geometry and operating conditions, making them available throughout subsequent layers.
• This provides the network with a complementary pathway to integrate structural geometric features alongside learned physical slices.
• In applications like electronic cooling modules or battery cold plates, subtle variations in component shape or channel layout can substantially impact pressure drop, flow patterns, and thermal performance.
• GeoTransolver is specifically built to retain this rich geometric context while modeling global field behaviors.

The trade-off: Additional geometry processing introduces configuration choices involving neighborhood radii, sampling, and context dimensions. How well fine features are represented still depends on the training data and configuration.

GeoTransolver is one of the primary architectures we use in Curlscape Sift for geometry-dependent fluid flow and conjugate heat transfer.

9. Anchored-Branched Universal Physics Transformer (AB-UPT)

AB-UPT scales attention through a smaller set of representative spatial points called anchors.

• Information exchange across the domain is enabled by computing attention among a smaller set of representative spatial locations called anchors, from which evaluation points subsequently extract contextual features.
• In contrast to Transolver's learned slices, AB-UPT relies on explicitly sampled spatial coordinates for its intermediate representation.
• By coupling interacting surface and volume representations, the framework is well suited for external aerodynamics, where surface geometry and surrounding flow fields are tightly interdependent.
• Provided that necessary query-point inputs and geometry preprocessing are supplied, the architecture can perform inference independently of the original CFD volume mesh.

The trade-off: Anchor count and distribution determine the balance between computational cost and spatial coverage. More anchors provide a richer representation but increase attention cost.

AB-UPT is particularly relevant to large-scale external aerodynamics. Its reference implementation is maintained by Emmi AI.

Comparing the architectural choices

Across these nine approaches, three differences matter most.

Local versus global communication. Graph networks exchange information through neighborhoods, while FNO, Transolver, and AB-UPT provide different mechanisms for broader interactions. Each approach balances the distance information can travel against computational cost.

How geometry is represented. FNO relies on structured grids, graph networks use connectivity, and architectures such as DoMINO and GeoTransolver introduce explicit geometry-processing mechanisms. These choices influence how the model handles complex shapes and local features.

What inference requires. Some models depend on a mesh or structured grid. Others can predict from processed geometry and query points without requiring the original CFD volume mesh. This can reduce preprocessing effort, although geometry preparation is still necessary.

These differences are useful for narrowing the choice of architecture. They do not establish which model will be most accurate on a particular dataset.

What a new design needs

The practical differences become clearer when predicting a new geometry.

For industrial workflows, this matters beyond prediction accuracy. Generating a high-quality volume mesh can be time-consuming, so architectures that support mesh-independent inference may reduce part of the preprocessing effort.

However, they still require inputs consistent with the representations used during training.

Where to start

A model's effectiveness depends on its representation, training distribution, physical conditions, and evaluation method. Relative error metrics alone may also overlook important local gradients or spatial structures.

The architecture helps narrow the options. Experiments determine whether those choices work.

Next: Putting the architectures to test

In the next article, we compare models on engineering problems we've worked with in Curlscape Sift:

• Air-cooled PCB and heatsink: A conjugate heat-transfer dataset covering an FR4 board, package, silicon die, and aluminium heatsink in an air channel. It contains designs from a space of 12-parameter design of experiments. We compare GeoTransolver, AB-UPT, and DoMINO on matched splits, with and without physics loss.
• Battery cold plates: Seven channel families for prismatic-cell cooling, including experiments where an entire geometry family is held out during training to test generalization.
• Public PLAID benchmarks: Five datasets spanning aerodynamics, turbomachinery, and solid mechanics, evaluated alongside published baselines such as MeshGraphNet and FNO.
• Ahmed body: An external-aerodynamics benchmark for comparing GeoTransolver and DoMINO around a bluff-body wake.

We'll examine more than aggregate error scores. In particular, we'll look at how well each model reproduces spatial variation and responds to changes in geometry.

Because the question is not simply which architecture achieves the lowest error, but which one can represent the physics reliably when the design changes.