Home
Numerical Methods
Structured course notes on floating-point arithmetic, polynomials, numerical differentiation, and more.
When Numbers Go Wrong
The 500 Million USD Typo
On June 4, 1996, the European Space Agency’s Ariane 5 rocket lifted off on its maiden flight. Thirty-seven seconds later, it veered off course and self-destructed in a spectacular fireball, destroying 500 million USD worth of satellites.
The investigation uncovered a single line of software: a 64-bit floating-point number was being converted to a 16-bit signed integer. The value, representing the rocket’s horizontal velocity, exceeded 32,767, the maximum a 16-bit integer can hold. The overflow triggered an unhandled exception. The exception crashed the guidance computer. The rocket thought it was tumbling and destroyed itself.
The Ariane 4 rocket, for which the software was originally written, had never produced a value that large. Nobody thought to re-validate the assumption on a faster rocket. One unchecked numerical conversion ended a decade of engineering work.
The Patriot Missile Failure
During the Gulf War, a U.S. Patriot missile battery in Dhahran, Saudi Arabia failed to intercept an Iraqi Scud missile. The Scud struck a barracks, killing 28 soldiers.
The cause was a rounding error in time. The system’s internal clock counted time in units of second, stored as a 24-bit fixed-point number. But has no exact binary representation just like has no exact decimal representation. The tiny error accumulated with every tick.
After 100 hours of continuous operation, the accumulated drift was 0.34 seconds enough for a Scud traveling at Mach 5 to move over 500 meters. The battery tracked the incoming missile, calculated where it was, and fired where it had been. A number that was almost right caused a catastrophic miss.
Why Study Numerical Methods?
Every time a computer solves an equation, simulates physics, renders a 3D scene, or trains a neural network, it is doing numerical computation. And every numerical computation involves approximation.
The question is never “is there an error?” there always is. The question is “how large is the error, and does it matter?” Numerical methods give you the tools to answer that question rigorously.
In this course you will learn:
- How floating-point arithmetic works and where it silently introduces error
- How to approximate derivatives and integrals when closed forms don’t exist
- How to solve equations and linear systems that have no algebraic solution
- How to fit models to data in a mathematically optimal way
- How to bound and control error in all of the above
Chapters
Floating Point Arithmetic
How computers store and manipulate real numbers: fixed point, IEEE 754, rounding errors, and loss of significance.
Polynomial Interpolation
Polynomial representation, Vandermonde, Lagrange, Newton divided differences, Hermite interpolation, and Runge’s phenomenon.
Numerical Differentiation
Finite difference approximations, truncation error, rounding error, and Richardson extrapolation.
Root Finding
Iterative algorithms for solving non-linear equations: bisection, fixed-point iteration, Newton’s method, Aitken acceleration, and the secant method.
Linear Equations
Direct methods for solving square systems: Gaussian elimination, LU decomposition, and pivoting strategies for numerical stability.
Least Squares
Solving overdetermined systems by minimizing squared residuals: normal equations, orthonormality, and QR decomposition via Gram-Schmidt.
Numerical Integration
Newton-Cotes formulas, trapezoidal and Simpson’s rules, composite methods, and error analysis.
How to Use This Site
- Use the sidebar on the left to navigate between chapters and individual sections.
- Topics within a chapter build on each other; reading in order is strongly recommended.
- Each chapter ends with practice problems that include full worked solutions.
- Key formulas are highlighted in boxes throughout each section.