Mathematics
Using Lagrange Multipliers to Optimize Across Constraints
Quick fact
The method of Lagrange multipliers was introduced by Joseph-Louis Lagrange in the 18th century and is based on the insight that at an optimal point under a single equality constraint, the gradients of the objective function and the constraint are parallel. This fundamental principle generalizes to multiple constraints and has become a cornerstone of multivariable optimization.
Why this is interesting
Imagine you're a farmer with a fixed length of fence and you want to enclose the largest possible rectangular field. How do you find the best dimensions? This puzzle is a taste of constrained optimization, and Lagrange multipliers is the magic tool that solves it elegantly.
Read the full explanation
Understanding Using Lagrange Multipliers to Optimize Across Constraints
Let's start with a simple problem: maximize f(x,y) = xy subject to the constraint that x + y = 10. You could solve this by substitution, but when constraints become more complex, we need a better way. The core idea is to look for points where the level curves of f are tangent to the constraint line. At such a point, the gradient of f (which points in the direction of steepest increase) is perpendicular to the level curve, and the gradient of the constraint (also perpendicular to its level curve) must be parallel to it. So we introduce a new variable, λ (the Lagrange multiplier), and write the condition: ∇f = λ∇g. This gives us a system of equations to solve for x, y, and λ. In our example: ∂f/∂x = y, ∂f/∂y = x, and ∂g/∂x = 1, ∂g/∂y = 1. So we get y = λ, x = λ, and x + y = 10. Solving gives x = y = 5. That's the optimum!
A deeper explanation
Why does this work? The level curves of f (curves where f is constant) and the constraint curve g(x,y)=0 are tangent at an optimum. Since the gradient is perpendicular to level curves, both gradients point in the same direction (or opposite), meaning they are linearly dependent: ∇f = λ∇g. This is the fundamental theorem behind the method. For multiple constraints, say g1=0 and g2=0, we write ∇f = λ1∇g1 + λ2∇g2. The method turns a constrained problem into an unconstrained one by forming the Lagrangian L = f - λ(g - c) and taking partial derivatives with respect to all variables and λ. Setting them to zero yields the necessary conditions for a local extremum. This technique is widely used in economics to maximize utility under a budget, in physics to find equilibrium configurations, and in engineering to optimize designs with material limits. The Lagrange multiplier λ also has an economic interpretation: it measures the sensitivity of the optimum to a small change in the constraint, e.g., the marginal utility of income.