PINN Error Analysis for Burgers’ Equation

ISEF Category: Mathematics

Subcategory: Analysis  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A neural network can solve a fluid equation, but that does not mean it solves it well. PINNs can look smooth and smart while hiding large errors. Your project asks a hard question, how close is the network to the real solution when you measure it the right way?

What Is It?

Physics-informed neural networks, or PINNs, are neural networks trained to fit data and obey the rules of a differential equation at the same time. For this project, the equation is the 1-D viscous Burgers' equation, a classic model that acts like a toy version of fluid flow with both spreading and steepening behavior.

Think of it like trying to sketch a river using both photos and physics. A regular neural network only matches examples. A PINN also tries to respect the equation that governs the river. Your job is to test how network width and depth change the error, which means how far the prediction strays from the true solution in a Sobolev norm. A Sobolev norm measures not just the function value, but also how its derivatives behave, so it checks whether the shape and slope are right, not just the curve itself.

Why This Is a Good Topic

This is a strong science fair topic because it turns a trendy machine learning method into a precise math question. You can change network size, loss balance, or training setup and measure how the error changes. That makes the project testable, quantitative, and honest about failure. It also connects to real work in scientific computing, where people want neural nets that obey physics instead of just fitting data.

Research Questions

• How does network width affect the Sobolev-norm error of a PINN for the 1-D viscous Burgers' equation? ?
• How does network depth affect the convergence rate when width is held fixed? ?
• What is the effect of changing the balance between PDE residual loss and data loss on final error? ?
• To what extent does random initialization change the spread of error across training runs? ?
• Which activation function gives the smallest derivative error for the same network size? ?
• How does collocation point density affect the stability of the learned solution? ?

Basic Materials

• Laptop or desktop computer with enough memory to train small neural networks.
• JAX installed in Python.
• NumPy for numerical arrays.
• Matplotlib for plots.
• Access to a reference solution for the 1-D viscous Burgers' equation from a textbook, paper, or numerical solver.
• Spreadsheet or notebook for tracking hyperparameters and results.

Advanced Materials

• Workstation or university cluster access for repeated training runs.
• JAX with GPU or TPU support.
• Automatic differentiation tools for higher-order derivatives.
• Scientific Python stack, including SciPy, NumPy, and Matplotlib.
• Reference solver for Burgers' equation, such as a finite difference or spectral method implementation.
• Version control system for experiment tracking.
• LaTeX for formal write-up and equation formatting.

Software & Tools

• JAX: Computes gradients and higher-order derivatives needed for PINN training and error checks.
• Python: Organizes simulation code, training loops, and data analysis.
• NumPy: Handles arrays for grids, residuals, and evaluation data.
• Matplotlib: Plots solution profiles, residuals, and convergence trends.
• SciPy: Supports numerical comparisons, interpolation, and error metrics.

Experiment Steps

1. Define the exact PDE, the domain, and the error norm you will measure so your project has one clear target.
2. Choose the network sizes you will compare, then fix every other training choice so width and depth are the main variables.
3. Build or source a trusted reference solution so you can compare predictions against a known answer.
4. Plan how you will sample collocation points, boundary points, and initial condition points so the training data are balanced.
5. Design your evaluation method around one or more norms, then decide how you will estimate convergence from repeated runs.
6. Set up plots and tables before training starts, so you can track error, runtime, and variance in a clean way.

Common Pitfalls

• Measuring only pointwise error, which can hide derivative mistakes that a Sobolev norm would catch.
• Changing the training schedule between trials, which makes width and depth comparisons unfair.
• Using too few collocation points, which can make the PINN look accurate in one region and fail elsewhere.
• Comparing against a low-quality reference solution, which can make the error analysis meaningless.
• Reporting one lucky training run, which ignores the large run-to-run variation that PINNs often show.

What Makes This Competitive

A competitive version of this project goes beyond a simple training demo. You would give a careful error theory, compare several network scales, and report uncertainty across multiple runs. Strong work also separates approximation error from optimization error, so you can say whether the network failed because it was too small or because training got stuck. Clear figures, clean scaling laws, and a thoughtful discussion of when the bound matches reality would make the project much stronger.

Project Variations

• Test the same error analysis on a different PDE, such as the heat equation, to compare how stiffness changes convergence.
• Compare tanh, sine, and ReLU activations to see which one best controls derivative error in the Sobolev norm.
• Add noisy boundary or initial data and study how noise changes the gap between theory and numerical behavior.

Learn More

• MIT OpenCourseWare, Numerical Analysis: Search MIT OpenCourseWare for courses on numerical methods, PDEs, and scientific computing.
• Stanford Encyclopedia of Philosophy, Neural Networks and Approximation: Search for background on function approximation and universal approximation ideas.
• arXiv: Search for recent preprints on physics-informed neural networks and Burgers' equation.
• PubMed: Search for review articles on scientific machine learning and physics-informed models, then filter for open-access papers.
• NIST Digital Library of Mathematical Functions: Use it for background on special functions, norms, and mathematical notation when you need definitions.