Numerical Differentiation

Differentiation is one of the most fundamental operations in mathematics, yet in numerical computation we often cannot apply standard calculus rules directly. When $f(x)$ is defined only by a table of measured values, or is too complex for analytical treatment, we must approximate the derivative using only function values at discrete points. This chapter develops the three classical finite difference formulas, rigorously quantifies their errors, and introduces Richardson extrapolation as a technique for systematically improving accuracy.

---

Learning Objectives

By the end of this chapter you should be able to:

1. State the geometric interpretation of the derivative and explain why a secant line approximates the tangent slope.
2. Write down and apply the forward, backward, and central difference formulas for a given function and step size.
3. State the truncation error order for each formula and predict how the error changes when $h$ is halved or divided by 10.
4. Prove the forward difference truncation error using Lagrange interpolation.
5. Compute an upper bound on the truncation error using the appropriate derivative and an interval-maximisation argument.
6. Explain why reducing $h$ beyond an optimal value increases the total error through catastrophic cancellation.
7. Write the total error formula combining truncation error and rounding error, and identify the optimal $h$.
8. Derive the first-level and second-level Richardson extrapolation formulas from Taylor series.
9. Apply Richardson extrapolation to compute a derivative approximation of order $O(h^4)$ or $O(h^6)$.
10. Deduce a Richardson-type formula for a non-standard step ratio.

---

Chapter Sections

#	Section	Key Concepts
1	Differentiation Basics	Power rule, tangent slope, motivation for numerical methods
2	Difference Formulas	Forward, backward, and central difference; worked examples; comparison
3	Truncation Error	Error orders $O(h)$ vs $O(h^2)$; Lagrange proof; upper bound example
4	Rounding Error and Total Error	Catastrophic cancellation; total error formula; optimal $h$
5	Richardson Extrapolation	$D_h^{(1)}$ and $D_h^{(2)}$ derivations; generalized formula; worked example; custom step ratio