Part 1 of 2
Numerical Methods — Roots of Equations & Linear Systems
Last reviewed 16 Sept 2026 · 7 min read
Errors
| Error | Definition |
|---|---|
| Absolute error | |
| Relative error | |
| Percentage error | Relative error × 100 |
| Round-off error | Due to finite number of digits in computation |
| Truncation error | Due to approximating an infinite process by a finite one (e.g. truncating a Taylor series) |
| Inherent error | Present in the data/model itself |
- Significant digits — digits that carry meaning; a number correct to decimal places has error at most .
- Errors propagate: for sums, absolute errors add; for products/quotients, relative errors add (approximately).
Roots of non-linear equations
Intermediate value theorem
If is continuous on and , there is at least one root in .
1. Bisection method
- Choose with .
- ; replace the end point whose function value has the same sign as .
- Repeat until the interval is small enough.
- Always converges (bracketing), but slowly — linear convergence, error halves each step.
- Iterations for accuracy ε: .
2. Regula falsi (method of false position)
Keeps a bracket like bisection but uses the chord intercept — usually faster; linear convergence (one end may remain fixed).
3. Secant method
No bracketing required; superlinear convergence (order ≈ 1.618); may diverge.
4. Newton–Raphson method
- Quadratic convergence (order 2) near a simple root — the number of correct digits roughly doubles each step.
- Requires ; fails or is slow if , near multiple roots (convergence becomes linear), or with poor starting values.
- Square root of :
- Reciprocal of :
5. Fixed-point iteration
Rewrite as and iterate .
- Converges if near the root (linear convergence with rate ).
Order of convergence summary
| Method | Order | Bracketing |
|---|---|---|
| Bisection | 1 (linear) | Yes |
| Regula falsi | 1 (linear) | Yes |
| Secant | ≈ 1.618 | No |
| Newton–Raphson | 2 (quadratic) | No |
| Fixed-point | 1 (if ) | No |
Linear systems — direct methods
Gauss elimination
- Forward elimination — reduce to upper triangular form.
- Back substitution.
- Partial pivoting — swap rows so the largest absolute element in the column becomes the pivot — reduces round-off errors and avoids division by zero.
- Computational effort ≈ multiplications for large .
Gauss–Jordan method
Eliminates above and below pivots to obtain the identity matrix — gives the solution directly (more work than Gauss elimination); used to find inverses.
LU decomposition
, then solve (forward substitution) and (back substitution) — efficient for many right-hand sides.
| Method | Form |
|---|---|
| Doolittle | has unit diagonal |
| Crout | has unit diagonal |
| Cholesky | for symmetric positive definite matrices (about half the work) |
Thomas algorithm
A simplified Gauss elimination for tridiagonal systems (common in finite differences for beams, heat conduction) — work proportional to .