Mathematics
Condition Number: Measuring Numerical Stability
Quick fact
A problem with a condition number of 10^k can amplify input errors by a factor of about 10^k, meaning that a 1% input error could become a 50% output error for a condition number of around 50.
Why this is interesting
Have you ever noticed that even a tiny change in a measurement can sometimes cause a wildly different result? Why do some calculations seem to magnify small errors while others don't?
Read the full explanation
Understanding Condition Number: Measuring Numerical Stability
Imagine you are baking a cake, and the recipe says 200 grams of flour. If you accidentally use 205 grams, the cake still turns out fine. But if the recipe is extremely sensitive—say, 200 grams of a chemical that must be exact—then a tiny change could ruin the result. In mathematics, the condition number measures exactly this kind of sensitivity: it tells us how much the output of a problem changes when the input is nudged slightly. For example, solving a system of equations can be thought of as a machine: you feed in coefficients and constants, and it produces a solution. If the machine is well-conditioned, a small tweak to the numbers you feed in produces only a small change in the solution. If it is ill-conditioned, a tiny tweak can produce a huge change. The condition number is a single number that summarizes this behavior. If it is close to 1, the problem is well-conditioned; if it is large (like 10^6), the problem is ill-conditioned. To get a feel for it, consider the equation x + y = 2 and x + 1.00001y = 2.00001. The solution is x=1, y=1. But if you change the second equation slightly to x + 1.00001y = 2.00002, the solution becomes very different—x=2, y=0. Here, a tiny change in a coefficient leads to a dramatic shift in the solution, so this system has a high condition number.
A deeper explanation
Mathematically, the condition number of a problem is the ratio of the relative change in the output to the relative change in the input. For a function f, it is roughly |x f'(x) / f(x)| for a single variable, but for solving linear systems Ax = b, it is defined as ||A|| ||A^{-1}||, where ||·|| is a matrix norm. This quantity is always at least 1, and it captures the worst-case amplification of input errors. The key insight is that condition numbers are intrinsic to the problem itself—they do not depend on the algorithm you use. No matter how accurate your computation, if the condition number is huge, the computed solution can be wildly off because the input data already contains small errors (from measurement or rounding). This is why numerical analysts emphasize that a large condition number means the problem is ill-conditioned and thus difficult to solve accurately. In practice, when you solve a linear system on a computer, the input is rounded to floating-point numbers. That introduces a small relative error, say about 10^{-16} on a typical machine. If the condition number is 10^12, then the error in the solution can be around 10^{-4}, which is significant. Understanding conditioning is crucial because it tells you whether an inaccurate result is the algorithm's fault or the problem's inherent sensitivity. It guides you to choose stable algorithms (like using partial pivoting) and to recognize when a given problem is simply too sensitive to be solved with confidence. Condition numbers appear everywhere: in optimization, differential equations, and even in machine learning where data matrices are often ill-conditioned.