Physics-Informed Neural Networks (PINNs) for Scientific Simulations

Key Takeaways

• Physics-Informed Neural Networks embed physical laws directly into neural network training. They can solve PDEs without traditional mesh generation or massive datasets.
• Traditional ML surrogates often need many simulation runs to train. PINNs use physics equations as constraints and can work with boundary data or sparse measurements.
• A PINN outputs predictions while minimizing the PDE residual through automatic differentiation. If predictions violate the governing physics, the network is penalized during training.
• PINNs are useful for inverse problems, sparse-data regimes, irregular geometries, and cases where mesh generation is difficult.
• PINNs are not ideal for simple problems where traditional solvers work well, or when you need rigorous error bounds and convergence guarantees.

You are trying to solve a partial differential equation. It may describe heat diffusion in a new alloy, fluid flow around a turbine blade, or phase-field evolution in a battery electrode.

Traditionally, you would discretize the domain into a mesh, set up a numerical scheme, solve the resulting system, and hope the mesh does not create numerical problems.

Physics-Informed Neural Networks offer another approach. Instead of building a mesh first, you train a neural network that learns the solution while being constrained by the governing physical equation.

Since their introduction by Raissi, Perdikaris, and Karniadakis, PINNs have moved from a specialized academic idea to an important tool in computational science. They are especially useful for problems that are awkward for both traditional numerical methods and conventional machine learning.

This guide explains the core concept, how PINNs work, how to implement a simple version in PyTorch, where they help, and where their limitations matter.

The Core Idea: Teaching a Neural Network the Laws of Physics

A standard machine learning model learns by minimizing the error between predictions and a training dataset. It is mostly data-driven. You provide examples, the model finds patterns, and you hope it generalizes.

A Physics-Informed Neural Network uses a different approach. It trains on both data and physics.

The neural network predicts the solution of a PDE. The loss function measures not only how close the prediction is to available data, but also how well the prediction satisfies the governing differential equation.

In other words, the network is penalized when it violates the physics.

Consider a PDE written in residual form:

F(u, x, ∂u/∂x, ∂²u/∂x², …) = 0

Here, u is the solution, and the derivatives are taken with respect to the spatial or temporal coordinates. A PINN builds a loss function with two main components:

1. Data loss. This measures the mean squared error between network predictions and available data, such as initial conditions, boundary conditions, or sparse measurements.
2. Physics loss. This measures the mean squared PDE residual at collocation points sampled across the domain.

The model optimizes both terms at the same time. It learns predictions that are consistent with data and with the governing physics.

How PINNs Actually Work: The Architecture

The neural network architecture for PINNs is usually a multi-layer perceptron.

The input layer receives spatial coordinates such as x, y, and z, and possibly time t. For a 2D steady-state problem, there may be two inputs. For a transient 1D problem, the inputs are often x and t.

The hidden layers usually contain 2 to 5 layers with 50 to 200 neurons each. The tanh activation function is often preferred over ReLU because many PDE solutions are smooth, and smooth activations can better represent continuous physical fields.

The output layer contains one neuron per unknown. For example, it may output temperature, concentration, pressure, or velocity. Multi-physics problems can use multiple output neurons with a combined loss function.

Automatic differentiation is the key mechanism. Instead of approximating derivatives through finite differences, automatic differentiation computes derivatives of the network output with respect to its inputs. This lets the model evaluate the PDE residual directly inside the loss function.

The Loss Function: Two Terms, One Goal

A typical PINN loss function looks like this:

L_total = λ_data × L_data + λ_physics × L_physics

The weighting factors balance the two terms. This balance matters. If the physics loss dominates too heavily, the model may ignore data constraints. If the data loss dominates, the physics constraint may become too weak.

L_data measures prediction error against available measurements, initial conditions, or boundary conditions.

L_physics measures the PDE residual. The network prediction is plugged into the differential equation, squared, and averaged over collocation points sampled inside the domain.

The key idea is that the model does not need a fixed mesh. By sampling points throughout the domain, the network learns to satisfy the PDE where the physics is defined. There is no element connectivity, mesh quality issue, or traditional grid requirement.

A Worked Example: Solving the Heat Equation

Consider the 1D heat equation:

∂u/∂t = ∂²u/∂x², with u(x,0) = sin(πx)

This problem has a known analytical solution:

u(x,t) = e^-π²t × sin(πx)

Here is a simple PINN setup for this problem in PyTorch:

import torch
import torch.nn as nn

class PINN(nn.Module):
    def __init__(self):
        super().__init__()
        # Two hidden layers, 50 neurons each, tanh activation
        self.fc1 = nn.Linear(2, 50)   # inputs: x, t
        self.fc2 = nn.Linear(50, 50)
        self.fc3 = nn.Linear(50, 1)   # output: u(x,t)
        self.activation = torch.tanh
        
    def forward(self, x, t):
        x = self.activation(self.fc1(torch.stack([x, t], dim=-1)))
        x = self.activation(self.fc2(x))
        return self.fc3(x).squeeze()

# Create the network
model = PINN()

# Automatic differentiation for physics loss
x = torch.tensor(x_data, requires_grad=True)
t = torch.tensor(t_data, requires_grad=True)

u = model(x, t)
u_t = torch.autograd.grad(u.sum(), t, create_graph=True)[0]
u_x = torch.autograd.grad(u.sum(), x, create_graph=True)[0]
u_xx = torch.autograd.grad(u_x.sum(), x, create_graph=True)[0]

# Physics loss: |∂u/∂t - ∂²u/∂x²|²
physics_loss = torch.mean((u_t - u_xx)**2)

# Data loss: boundary or initial conditions
data_loss = torch.mean((u - u_initial)**2)

# Total loss
total_loss = physics_loss + data_loss

During training, the network backpropagates through the automatic differentiation graph. It adjusts weights to minimize both the PDE residual and the boundary or initial condition error.

After training, the model can evaluate the solution at any point in the domain, including points that were not used directly during training.

PINNs vs Traditional ML Surrogates

PINNs are related to machine learning surrogates, but they are not the same. A traditional surrogate learns from many input-output examples produced by simulations. A PINN learns by enforcing governing equations during training.

The fundamental distinction is this: a traditional surrogate learns a mapping from inputs to outputs by seeing many examples. A PINN learns a solution constrained by the governing equations. You do not need thousands of simulation runs if you already know the physics and boundary conditions.

When to Use PINNs: The Decision Matrix

PINNs are not a universal replacement for traditional solvers. They are useful in specific scenarios.

When PINNs Are a Strong Fit

• Inverse problems. PINNs can help discover unknown parameters such as conductivity, diffusivity, or reaction rates from sparse observations.
• High-dimensional parameter spaces. PINNs can treat parameters as network inputs and learn across parameter ranges.
• Irregular or complex geometries. Since no mesh generation is required, you can sample points in the domain and let the network learn from the collocation point distribution.
• Sparse-data regimes. Physics constraints act as regularization when data alone would be insufficient.
• Multi-fidelity problems. PINNs can combine lower-fidelity and higher-fidelity information through weighted loss terms.

When to Avoid PINNs

• Simple problems where traditional solvers already work well. If a standard solver converges quickly and accurately, a PINN may add unnecessary complexity.
• Problems that require provable error bounds. PINNs do not yet have the same convergence theory as mature finite element or finite volume methods.
• High-accuracy requirements near singularities. Standard PINNs can struggle with sharp gradients, discontinuities, and boundary layers.
• Cases where you already have a validated, optimized solver. If your existing FVM or FEM solver is fast and accurate, replacing it may not help.

Real Scientific Applications

PINNs have been applied across many scientific domains.

In plasma physics and astrophysics, PINNs have been used to solve Grad-Shafranov equations for magnetic equilibria and Lane-Emden equations for stellar structure. These problems can involve mixed boundary conditions that are challenging for some traditional solvers.

In fluid dynamics, PINNs have been used to infer full velocity fields from sparse measurement data by enforcing Navier-Stokes equations as constraints.

In biomedical imaging, physics-informed models have been used to filter and reconstruct flow data while respecting mass conservation and momentum equations.

In electrochemistry, PINNs are being explored for coupled diffusion-reaction systems, including battery electrode modeling where analytical solutions are unavailable.

Common Pitfalls and How to Avoid Them

1. Loss Imbalance

The most common training failure is one loss term dominating another. The physics loss may be much larger than the data loss, or the opposite can happen. When this occurs, the model may ignore one constraint entirely.

Fix this by normalizing both loss terms. Use adaptive weighting schemes that adjust λ_data and λ_physics based on the relative magnitude of each term. Monitor both losses separately during training.

2. Local Minima and Training Stalls

PINNs can get stuck in poor local minima. The network may satisfy the physics loss while producing an incorrect solution, or it may fit boundary data while failing inside the domain.

Fix this with curriculum-based training. Start with a simplified version of the problem, such as a steady-state form or a simpler PDE, then gradually increase complexity. You can also begin with fewer collocation points and increase coverage over time.

3. Sharp Gradients and Discontinuities

Standard PINNs often struggle with sharp gradients, discontinuities, and boundary layers. Smooth activation functions can make it difficult to represent abrupt changes.

Fix this with adaptive collocation point refinement. Concentrate points where the residual is largest. You can also use hard-constraint formulations that embed boundary conditions exactly, or composite PINNs with separate networks on different regions.

4. Overconfidence Without Guarantees

PINNs can produce visually plausible results that are still wrong. Without rigorous error bounds, it can be difficult to know where the solution is trustworthy.

Fix this by validating PINN predictions against known benchmarks when possible. Train multiple networks with different initializations to estimate prediction variance. Cross-check against traditional solvers on simple test cases before trusting results on complex problems.

Getting Started: A Practical Workflow

Use the following workflow when implementing PINNs in a research project.

Step 1: Define the Problem Formally

Write the PDE in residual form. Identify all parameters, boundary conditions, and initial conditions. Clarify what data you have and what you want to discover.

Step 2: Choose Your Framework

PyTorch is widely used in PINN tutorials and implementations. TensorFlow is also common. If you already use FiPy for traditional solvers, you can integrate PINNs at the workflow level by using them for inverse problems or rapid exploration, then verifying results with FiPy.

Step 3: Set Up the Network Architecture

Start simple: 2 to 3 hidden layers, 50 to 100 neurons, and tanh activation. You can refine later. Do not over-engineer the architecture at the beginning because the physics constraints do much of the work.

Step 4: Sample Collocation Points

Draw random points uniformly inside the domain. You may need hundreds or thousands of points depending on problem complexity. For multi-scale problems, use non-uniform sampling concentrated where the solution changes most.

Step 5: Implement the Loss Function

Build separate data loss and physics loss terms. Add normalization so their magnitudes are comparable. Monitor both loss terms during training.

Step 6: Train and Validate

Train first with Adam, often using a learning rate near 1e-3, then switch to L-BFGS for fine-tuning. Validate against analytical solutions or traditional solvers on benchmark problems before applying the model to novel problems.

The Future of PINNs in Scientific Computing

The PINN field is evolving quickly. Several developments are worth watching.

Operator Learning

Instead of learning a single solution, operator learning methods aim to learn the mapping from initial conditions and parameters to full solution fields. This means one trained model may handle a family of problems rather than one fixed case.

PhysicsNeMo

NVIDIA rebranded its Modulus framework as PhysicsNeMo, signaling broader industry interest in production-grade physics-informed AI tools.

Domain Decomposition

Extending PINNs to very large problems through parallel domain decomposition is an active research area. This matters for moving PINNs from academic benchmarks to real scientific simulation workloads.

Hybrid Approaches

Hybrid workflows combine PINNs with traditional solvers. PINNs can support rapid exploration or inverse problem solving, while verified FVM or FEM solvers can provide final validation.

Related Guides

For deeper coverage of related topics in computational science workflows:

• Machine Learning Surrogates for Scientific Simulations — Broader coverage of data-driven model acceleration, including Gaussian processes, polynomial chaos, and neural network surrogates.
• Verification vs Validation in Scientific Simulations — How to validate PINN predictions against benchmarks.
• Using FiPy for Phase-Field Modeling — When to use traditional FVM methods instead of PINNs for phase-field problems.
• Performance Profiling for Python PDE Solvers — Benchmarking PINNs against traditional solvers.

Need Help Implementing PINNs for Your Simulation?

PINNs are powerful, but they are not trivial to implement correctly. They may be useful if you are working on:

• Inverse problems with hidden physical parameters.
• Sparse-data modeling where traditional ML would overfit.
• Rapid exploration of parameter spaces that are too expensive for direct simulation.
• Integrating physics constraints into existing ML pipelines.

Our team can help build physics-informed models tailored to scientific workflows, including integration with FiPy and other PDE solvers. Contact us to discuss your project’s specific needs.

References

1. Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 724–741.
2. Karniadakis, G. E., Kevrekidis, I. G., Lu, L., et al. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 416–428.
3. Baty, H. (2024). A hands-on introduction to Physics-Informed Neural Networks for solving partial differential equations. Astronomical Observatory, Université de Strasbourg. arXiv:2403.00599.
4. Cuomo, D., et al. (2022). A comprehensive review on physics-informed machine learning for computational mechanics. Computers & Structures.
5. Karniadakis, G. E., et al. (2021). Physics-informed machine learning: A review. Nature Reviews Physics.
6. Wolf, T. (2024). Physics-informed Neural Networks: A simple tutorial with PyTorch. Medium.
7. Oxford Department of Computer Science. (2025–2026). Physics Informed Neural Networks course materials and syllabus.