Newton's Method

Newton’s method (also known as the Newton-Raphson method) is the workhorse of numerical root-finding. By using the tangent line at the current point to predict where the function crosses zero, it achieves super-linear convergence, meaning $\lambda = 0$. In practice this translates to very few iterations, often converging in under 10 steps even for demanding problems.

---

Derivation

At the current iterate $x_k$, draw the tangent line to $f(x)$. The tangent has slope $f'(x_k)$ and passes through the point $(x_k,\, f(x_k))$. The next iterate $x_{k+1}$ is defined as the x-intercept of this tangent line, that is, where $y = 0$.

Looking at the right-angled triangle formed by the points $(x_{k+1}, 0)$, $(x_k, 0)$, and $(x_k, f(x_k))$, the “rise” is $f(x_k)$ and the “run” is $(x_k - x_{k+1})$. Because the slope of this line is $f'(x_k)$, we can write:

$f'(x_k) = \frac{\text{rise}}{\text{run}} = \frac{f(x_k) - 0}{x_k - x_{k+1}}$

Rearranging to solve for $x_{k+1}$:

$x_k - x_{k+1} = \frac{f(x_k)}{f'(x_k)}$

$\boxed{x_{k+1} = x_k - \frac{f(x_k)}{f'(x_k)}}$

This is the Newton iteration formula. Each step moves from $x_k$ in the direction that the tangent line points toward zero.

Once $x_{k+1}$ is found, the process repeats. As shown by the green line in the figure, we drop down from the x-axis to the curve to find the new coordinate $(x_{k+1}, f(x_{k+1}))$. We draw a new tangent line there, which shoots down to the x-axis to find $x_{k+2}$. With each cycle, the x-intercept closes in rapidly on the true root.

---

Proof of Super-Linear Convergence

Newton’s method can be viewed as a fixed-point iteration with:

$g(x) = x - \frac{f(x)}{f'(x)}$

The convergence rate is $\lambda = |g'(x^*)|$. Differentiating $g(x)$ using the quotient rule on the second term:

$g'(x) = 1 - \frac{f'(x) \cdot f'(x) - f(x) \cdot f''(x)}{[f'(x)]^2} = \frac{[f'(x)]^2 - [f'(x)]^2 + f(x)f''(x)}{[f'(x)]^2} = \frac{f(x)f''(x)}{[f'(x)]^2}$

At the root $x^*$, by definition $f(x^*) = 0$:

$\lambda = |g'(x^*)| = \left|\frac{f(x^*)f''(x^*)}{[f'(x^*)]^2}\right| = \left|\frac{0 \cdot f''(x^*)}{[f'(x^*)]^2}\right| = 0$

Since $\lambda = 0$, Newton’s method achieves super-linear convergence at every simple root (where $f'(x^*) \ne 0$).

Note

Super-linear convergence means that the error shrinks dramatically with each iteration. Once the iterates are close to the root, each step roughly squares the number of correct decimal places. This is far faster than linear convergence where each step only multiplies the error by a fixed factor $\lambda$.

---

Drawbacks

Newton’s method has two important failure modes:

1. Division by zero at a turning point. If $f'(x_k) = 0$ at any iterate, the formula produces a division by zero and the method breaks down entirely. This happens when the iteration lands exactly on a local maximum or minimum of $f$.

2. Cycling near a turning point. If $x_0$ is chosen close to a turning point (where $f'$ is small), the tangent line is nearly flat, projecting $x_1$ far away from the root. The next iterate may overshoot to the other side, and the sequence can enter an infinite loop bouncing back and forth without converging.

Caution

These failure modes mean that a good initial guess is important for Newton’s method. If the derivative is small in the region of interest, consider using the bisection method first to get a rough bracket, then switch to Newton’s method once the initial guess is close to the root.

---

Worked Example

Problem: Find the root of $f(x) = \dfrac{1}{x} - 0.5$ starting from $x_0 = 1$ with error bound $10^{-5}$.

Step 1 — Compute the derivative

$f(x) = x^{-1} - 0.5 \implies f'(x) = -x^{-2} = -\frac{1}{x^2}$

Step 2 — Set up the iteration

$x_{k+1} = x_k - \frac{f(x_k)}{f'(x_k)} = x_k - \frac{\frac{1}{x_k} - 0.5}{-\frac{1}{x_k^2}}$

Step 3 — Iterate

$k$	$x_k$	$f(x_k)$	$f'(x_k)$
0	1.00000000	0.50000000	$-$1.00000000
1	1.50000000	0.16666667	$-$0.44444444
2	1.87500000	0.03333333	$-$0.28444444
3	1.99218750	0.00196078	$-$0.25196463
4	1.99996948	0.00000763	$-$0.25000763
5	1.99999999	0.00000000	$-$0.25000000
6	2.00000000	0.00000000	$-$0.25000000

Step 4 — Confirm

The iteration converges rapidly to:

$\boxed{x^* = 2.00000}$

The super-linear convergence is evident: after just a few iterations the function value is essentially zero. The true root can be confirmed analytically: $\frac{1}{x} = 0.5 \implies x = 2$.