Physics-Informed Machine Learning

Physics-Informed Machine Learning (PIML) is an emerging paradigm that integrates scientific domain knowledge (physical laws, constraints, or models) into machine learning workflows. In engineering systems—especially dynamic systems governed by differential equations—purely data-driven models often struggle with generalization and physical consistency. PIML addresses these challenges by blending first-principles physics with data-driven learning, thereby reducing required data, improving extrapolation, and ensuring that model outputs obey known physical rules. This survey outlines key methods to inject physics into ML models, presents code examples for simple dynamic systems (e.g. damped oscillators, reactors, thermal systems), and reviews both foundational and recent literature. Optimization applications, such as model predictive control (MPC), are emphasized alongside forward simulation.

I. Methods of Injecting Physics into ML Models

PIML techniques can be broadly categorized into several approaches which, when combined appropriately, yield models that are both accurate and interpretable.

A. Physics-Based Feature Engineering

Feature engineering uses physical insights—such as conservation laws, symmetry, and nondimensional numbers—to construct input features that better capture underlying dynamics.

For example, consider a pendulum where the true acceleration depends on sin(θ) rather than θ itself. Using sin(θ) as a feature allows even a linear regression model to capture the nonlinear dependence of acceleration on the angle.

import numpy as np
from sklearn.linear_model import LinearRegression

# Generate synthetic data for a pendulum
g, L = 9.81, 1.0
theta = np.linspace(-np.pi, np.pi, 100).reshape(-1, 1)  # angles from -π to π
acc_true = -(g/L) * np.sin(theta)                        # true angular acceleration

# Model 1: Linear regression on raw angle (theta)
model_raw = LinearRegression().fit(theta, acc_true)
pred_raw = model_raw.predict(theta)

# Model 2: Linear regression on sin(theta) as feature
X_feat = np.sin(theta)                                   # physics-inspired feature
model_feat = LinearRegression().fit(X_feat, acc_true)
pred_feat = model_feat.predict(X_feat)

# Evaluate at a sample angle
test_angle = np.array([[50 * np.pi / 180]])  # 50 degrees in radians
print("True acceleration:", -(g/L) * np.sin(test_angle))
print("Linear model (raw theta) prediction:", model_raw.predict(test_angle))
print("Linear model (sin(theta)) prediction:", model_feat.predict(np.sin(test_angle)))
[$[Get Code]]
B. Custom Neural Network Architectures with Physical Structure

Neural network architectures can be designed so that they inherently respect physical laws. Examples include Hamiltonian Neural Networks (HNNs) and Lagrangian Neural Networks. These models embed conservation laws into their topology, such that—for instance—a Hamiltonian network is constructed so that   dq/dt = ∂H/∂p   dp/dt = –∂H/∂q are satisfied automatically.

import torch

# Define a simple HNN model: input (q, p) -> output H (the system energy)
class HNN(torch.nn.Module):
    def __init__(self, hidden_dim=32):
        super().__init__()
        self.net = torch.nn.Sequential(
            torch.nn.Linear(2, hidden_dim),
            torch.nn.Tanh(),
            torch.nn.Linear(hidden_dim, 1)  # output: scalar H
        )
    def forward(self, q, p):
        # Stack q and p to form the input vector and return H(q, p)
        return self.net(torch.stack([q, p], dim=-1)).squeeze(-1)

hnn = HNN(hidden_dim=16)
optimizer = torch.optim.Adam(hnn.parameters(), lr=0.01)

# Synthetic data: small oscillations of a mass-spring system (q: position, p: momentum)
t = torch.linspace(0, 10, steps=101)
q_data = torch.cos(t)
p_data = -torch.sin(t)
dq_data = torch.gradient(q_data, spacing=(t,))[0]  # true dq/dt = p for m=1
dp_data = torch.gradient(p_data, spacing=(t,))[0]  # true dp/dt = -q for k=1

# Training loop to enforce Hamilton's equations as loss
for epoch in range(1000):
    idx = torch.randint(0, len(t), (32,))
    q, p = q_data[idx], p_data[idx]
    H = hnn(q, p)
    # Compute gradients of H with respect to p and q via autograd
    dH_dp = torch.autograd.grad(H, p, grad_outputs=torch.ones_like(H),
                                retain_graph=True, create_graph=True)[0]
    dH_dq = torch.autograd.grad(H, q, grad_outputs=torch.ones_like(H),
                                retain_graph=True, create_graph=True)[0]
    dq_pred = dH_dp         # predicted dq/dt
    dp_pred = -dH_dq        # predicted dp/dt
    dq_true = dq_data[idx]
    dp_true = dp_data[idx]
    loss = torch.mean((dq_pred - dq_true)**2 + (dp_pred - dp_true)**2)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

q_test, p_test = 1.0, 0.0
H_pred = hnn(torch.tensor(q_test), torch.tensor(p_test)).item()
print("Learned Hamiltonian at (q=1, p=0):", H_pred)
print("True Hamiltonian at (q=1, p=0):", 0.5 * q_test**2 + 0.5 * p_test**2)
[$[Get Code]]
C. Physics-Based Constraints in the Loss Function (Soft Constraints)

Physics-Informed Neural Networks (PINNs) integrate the governing equations directly into the loss function as penalty terms. The loss penalizes the residual of the underlying PDE or ODE, so that the neural network learns a solution that satisfies both the data and the physics.

import torch
# Known parameters for a damped oscillator: m*ddx + c*dx + k*x = 0, with m=1, k=1, c=0.1
c, k = 0.1, 1.0

# Neural network representing x(t)
model = torch.nn.Sequential(
    torch.nn.Linear(1, 20),
    torch.nn.Tanh(),
    torch.nn.Linear(20, 1)
)

# Collocation points in time domain
t_colloc = torch.linspace(0, 10, steps=100, requires_grad=True).reshape(-1, 1)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

for epoch in range(1000):
    x_pred = model(t_colloc)
    # Compute first and second derivatives with respect to time
    dx_pred = torch.autograd.grad(x_pred, t_colloc, grad_outputs=torch.ones_like(x_pred),
                                  retain_graph=True, create_graph=True)[0]
    ddx_pred = torch.autograd.grad(dx_pred, t_colloc, grad_outputs=torch.ones_like(dx_pred),
                                   retain_graph=True, create_graph=True)[0]
    # Compute ODE residual: ddx + c*dx + k*x should be zero
    ode_residual = ddx_pred + c * dx_pred + k * x_pred
    phys_loss = torch.mean(ode_residual**2)
    # Enforce initial conditions: x(0) = 1, dx(0) = 0
    x0_pred = model(torch.zeros(1, 1))
    dx0_pred = torch.autograd.grad(x0_pred, torch.zeros(1, 1),
                                   grad_outputs=torch.ones(1, 1), create_graph=True)[0]
    ic_loss = (x0_pred - 1.0)**2 + (dx0_pred - 0.0)**2
    loss = phys_loss + 100 * ic_loss
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
[$[Get Code]]
D. Hybrid Modeling with Differential Equations or Simulators

Hybrid modeling—also known as gray-box modeling—combines known physics with data-driven ML components. This can take the form of serial, parallel, or component-replacement strategies.

For instance, one can replace an unknown reaction rate term in a chemical process model with a neural network learned from data, and then embed this hybrid model into an ODE solver.

import numpy as np
from sklearn.neural_network import MLPRegressor
from gekko import GEKKO, ML

# Create training data for the unknown physics f_true(x) = -0.2*x^2
X = np.linspace(0, 2, 21).reshape(-1, 1)
y = -0.2 * X.ravel()**2
mlp = MLPRegressor(hidden_layer_sizes=(5,), activation='tanh', max_iter=5000).fit(X, y)

# Set up GEKKO model and import the trained MLP
m = GEKKO(remote=False)
m.time = np.linspace(0, 5, 101)  # simulate 5 seconds
x = m.Var(value=1.0)
# Create scaling for GEKKO's ML interface (custom scaler with min/max values)
data = np.concatenate([X, y.reshape(-1, 1)], axis=1)
# Assume the scaler is defined with known min/max (details omitted for brevity)
scaler = ML.CustomMinMaxGekkoScaler(data, features=[0], label=[1])
mma = scaler.minMaxValues()
ml_model = ML.Gekko_NN_SKlearn(mlp, mma, m)
f = ml_model.predict([x])

# Define the hybrid ODE: dx/dt = -0.5*x + f(x)
m.Equation(x.dt() == -0.5 * x + f)
m.options.IMODE = 4  # dynamic simulation mode
m.solve(disp=False)
print("Hybrid model final x:", x.value[-1])
[$[Get Code]]
E. Data Assimilation and Regularization with Physical Priors

Data assimilation techniques—such as Kalman filters or Moving Horizon Estimation (MHE)—continuously update the model state or parameters by blending model predictions with measurements. Regularization with physical priors (e.g., smoothness, monotonicity) further guides the learning towards physically plausible solutions.

from gekko import GEKKO
import numpy as np

# True system: dx/dt = -k_true * x, with k_true = 0.4
k_true = 0.4
t_points = np.linspace(0, 5, 51)
x_true = np.exp(-k_true * t_points)
x_meas = x_true[-1]  # measurement at t=5

# GEKKO model for parameter estimation (MHE)
m = GEKKO(remote=False)
m.time = np.linspace(0, 5, 51)
k = m.FV(value=1.0, lb=0, ub=2)
k.STATUS = 1  # allow optimization to adjust k
x = m.Var(value=1.0)
m.Equation(x.dt() == -k * x)
# Define controlled variable for measurements
y = m.CV(value=np.nan * np.ones(len(m.time)))
y.FSTATUS = 0
y.MEAS = np.nan * np.ones(len(m.time))
y.FSTATUS[-1] = 1  # provide measurement feedback at final time
y.MEAS[-1] = x_meas
m.Equation(y == x)
m.options.IMODE = 5  # MHE mode
m.options.EV_TYPE = 1  # L2 norm in objective
m.solve(disp=False)
print("Estimated k:", k.value[0])
print("Predicted x(5):", x.value[-1], "Measured x(5):", x_meas)
[$[Get Code]]

II. Comparison of PIML Techniques

ApproachStrengthsLimitations
Feature EngineeringSimple to implement; leverages domain insight; improves interpretabilityRequires accurate a priori identification of key physical features; may not capture complex dynamics fully
Custom Neural ArchitecturesGuarantees adherence to physical laws (e.g. conservation) by design; yields physically interpretable representationsRequires specialized knowledge for design; less flexible if system deviates from assumed physics
Physics-Based Loss (PINNs)General and flexible; learns solutions that satisfy PDE/ODE constraints even with limited data; naturally addresses inverse problemsTraining can be slow and challenging; balancing physics and data loss requires careful tuning
Hybrid ModelingCombines the strengths of first-principles and ML; improved accuracy with limited data; high interpretability of known physics componentsIntegration can be complex; interfacing ML with simulators requires careful scaling and calibration
Data Assimilation & RegularizationAdaptive correction with real-time data; enhances robustness; gently guides the solution towards physical plausibilityAdditional computational overhead; tuning of assimilation parameters is often problem-dependent

III. Conclusion

Physics-informed machine learning represents a significant advancement in bridging the gap between black-box AI and traditional engineering practices. By embedding physical laws into the ML modeling process—whether through feature engineering, custom neural architectures, physics-based loss functions, hybrid modeling, or data assimilation—engineers obtain models that are more data-efficient, generalizable, and interpretable. This synthesis of physics and data empowers more reliable dynamic optimization and control strategies, as demonstrated by the presented MPC case study for a chemical reactor.

Looking ahead, active research is exploring not only the parameters but also the discovery of entire physical laws via data-driven methods, further enhancing the robustness and interpretability of PIML in complex engineering applications.

Case Study

Physics-Informed Learning of Thermophysical Properties

IV. References

Home | Physics-Informed Machine Learning
Streaming Chatbot
💬