Dynamic Optimization

'''Case Study''': [[https://apmonitor.com/pds/index.php/Main/ThermophysicalProperties|Physics-Informed Learning of Thermophysical Properties]]

!!!! V. References

* Babaei, M.R., Stone, R., Knotts, T.A., Hedengren, J.D., Physics-Informed Neural Networks with Group Contribution Methods, Journal of Chemical Theory and Computation, American Chemical Society, 2023, DOI: 10.1021/acs.jctc.3c00195. [[https://pubs.acs.org/doi/10.1021/acs.jctc.3c00195|Article]]

* Patel, D. et al. (2024). Model Predictive Control Using Physics Informed Neural Networks for Process Systems. ADCHEM 2024, IFAC-PapersOnLine, 57(9), 276–281.

* Karniadakis, G. E., Kevrekidis, I. G., et al. (2021). Physics-informed machine learning. Nat. Rev. Phys., 3(6), 422–440. 
  [[https://www.avantipublishers.com/index.php/jaacm/article/view/1246|A Gentle Introduction to Physics-Informed Neural Networks]] | [[https://encyclopedia.pub/entry/51383|NSGA-PINN for Physics-Informed Neural Network Training]]

* Karniadakis, G. E., Kevrekidis, I. G., et al. (2021). *Physics-informed machine learning*. Nat. Rev. Phys., 3(6), 422–440. 
  [A Gentle Introduction to Physics-Informed Neural Networks](https://www.avantipublishers.com/index.php/jaacm/article/view/1246) | [NSGA-PINN for Physics-Informed Neural Network Training](https://encyclopedia.pub/entry/51383)
* Willard, J., Jia, X., et al. (2022). *Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems*. ACM Comput. Surv. 55(4): 73.  
  [Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems](https://ar5iv.org/pdf/2003.04919#:~:text=commercial%20applications%2C%20are%20beginning%20to,black%20box%20ML%20models%20has)
* Raissi, M., Perdikaris, P., & Karniadakis, G. (2019). *Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear PDEs*. J. Comput. Phys., 378, 686–707.
* Greydanus, S., Dzamba, M., & Yosinski, J. (2019). *Hamiltonian Neural Networks*. NeurIPS 2019.  
  [Hamiltonian Neural Networks](https://ar5iv.labs.arxiv.org/html/1906.01563#:~:text=the%20basic%20laws%20of%20physics,is%20perfectly%20reversible%20in%20time)
* Karpatne, A., Watkins, W., Read, J., & Kumar, V. (2017). *Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling*. SIAM Int. Conf. on Data Mining (SDM). 
  [Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems](https://ar5iv.org/pdf/2003.04919#:~:text=Karpatne%20et%20al,water%2C%20a%20known%20monotonic%20relationship)
* Parish, E. & Duraisamy, K. (2016). *A paradigm for data-driven predictive modeling using field inversion and machine learning*. J. Comput. Phys., 305, 758–774.
* GEKKO Documentation (2020). *Machine Learning in GEKKO*. 
  [Machine Learning — GEKKO 1.3.0 documentation](https://gekko.readthedocs.io/en/latest/ml.html#:~:text=,informed%20hybrid%20modeling)
* Sanyal, S. & Roy, K. (2023). *RAMP-Net: Robust Adaptive MPC for Quadrotors via Physics-informed Neural Networks*. IEEE Int. Conf. Robotics and Automation (ICRA).
* Zheng, Y. & Wu, W. (2023). *Physics-informed recurrent neural network based MPC for chemical processes*. J. Process Control, 118, 65–77.
* Patel, D. et al. (2024). *Model Predictive Control Using Physics Informed Neural Networks for Process Systems*. ADCHEM 2024, IFAC-PapersOnLine, 57(9), 276–281.

(:title Physics-Informed Machine Learning for Dynamic Optimization in Engineering:)

|| **Feature Engineering** || Simple to implement; leverages domain insight; improves interpretability || Requires accurate *a priori* identification of key physical features; may not capture complex dynamics fully ||
|| **Custom Neural Architectures** || Guarantees adherence to physical laws (e.g. conservation) by design; yields physically interpretable representations || Requires specialized knowledge for design; less flexible if system deviates from assumed physics ||
|| **Physics-Based Loss (PINNs)** || General and flexible; learns solutions that satisfy PDE/ODE constraints even with limited data; naturally addresses inverse problems || Training can be slow and challenging; balancing physics and data loss requires careful tuning ||
|| **Hybrid Modeling** || Combines the strengths of first-principles and ML; improved accuracy with limited data; high interpretability of known physics components || Integration can be complex; interfacing ML with simulators requires careful scaling and calibration ||
|| **Data Assimilation & Regularization** || Adaptive correction with real-time data; enhances robustness; gently guides the solution towards physical plausibility || Additional computational overhead; tuning of assimilation parameters is often problem-dependent ||

- Defining the CSTR ODEs with the learned kinetics.
- Setting the cooling jacket heat removal (Q_cool) as the manipulated variable.
- Specifying controlled variables for reactor temperature and concentration.
- Solving the optimization problem with GEKKO’s control modes (IMODE=6 or 8) to maintain safe operating conditions and optimal conversion.

(:title Physics-Informed Machine Learning for Dynamic Optimization:)

(:description Survey of Physics-Informed Machine Learning methods for dynamic simulation and optimization in engineering, with Python examples using PyTorch, GEKKO, and scikit-learn.:)

Physics-Informed Machine Learning (PIML) combines scientific domain knowledge with machine learning techniques, creating models that respect physical laws and constraints. These methods are particularly valuable in dynamic optimization, enabling accurate and interpretable predictions and control decisions for engineering systems.

Key methods to inject physics into ML models include:

-> '''I''': Physics-based Feature Engineering
-> '''II''': Custom Neural Network Architectures
-> '''III''': Physics-based Constraints (Soft Constraints)
-> '''IV''': Hybrid Modeling (Physics + ML)
-> '''V''': Data Assimilation and Regularization

!!!! I. Physics-based Feature Engineering

Using features derived from known physical laws improves model interpretability and efficiency.

(:toggle hide feature_engineering button show="Feature Engineering Example":)
(:div id=feature_engineering:)

# Pendulum dynamics
theta = np.linspace(-np.pi, np.pi, 100).reshape(-1,1)
acc_true = -(9.81) * np.sin(theta)

# Linear model using sin(theta) as feature
X_feat = np.sin(theta)

!!!! II. Custom Neural Network Architectures

Architectures embedding physical structures (e.g., Hamiltonian Neural Networks) ensure adherence to physics by design.

(:toggle hide hnn_example button show="Hamiltonian Neural Network Example":)
(:div id=hnn_example:)

   def __init__(self):

!!!! III. Physics-based Constraints (Soft Constraints)

Physics-Informed Neural Networks (PINNs) incorporate physics directly into the loss function.

(:toggle hide pinn_example button show="PINN Damped Oscillator Example":)
(:div id=pinn_example:)

!!!! IV. Hybrid Modeling (Physics + ML)

Combines first principles with ML models, improving generalization and interpretability.

(:toggle hide hybrid_example button show="Hybrid Modeling Example with GEKKO":)
(:div id=hybrid_example:)

mlp = MLPRegressor().fit(X, y)
m = GEKKO(); x = m.Var()
ml_model = ML.Gekko_NN_SKlearn(mlp, scaler_minmax, m)
m.Equation(x.dt() == -0.5*x + ml_model.predict([x]))
(:sourceend:)
(:divend:)

!!!! V. Data Assimilation and Regularization

Continuous model refinement using new data ensures accuracy and robustness.

(:toggle hide assimilation_example button show="Data Assimilation Example with GEKKO":)
(:div id=assimilation_example:)
(:source lang=python:)
from gekko import GEKKO
m = GEKKO()
k = m.FV(value=1.0); k.STATUS=1

m.Equation(x.dt()==-k*x)
m.options.IMODE=5; m.solve()

!!!! Case Study: MPC for Chemical Reactor using Hybrid PIML

Hybrid physics-informed model for reactor kinetics enhances MPC accuracy.

(:toggle hide mpc_example button show="Chemical Reactor MPC Example":)
(:div id=mpc_example:)

!!!! Summary of Methods