Bellman equation
The Bellman equation is a recursive relationship that expresses the value of a decision problem at a certain state in terms of the payoff from an immediate action plus the value of the resulting successor state. Named after Richard Bellman, it forms the mathematical backbone of Dynamic programming and Reinforcement learning, enabling computers to find optimal policies by breaking problems into overlapping subproblems.
The equation embodies a principle of optimality: if a policy is optimal, then every future decision within that policy must also be optimal. This insight transforms seemingly intractable problems into solvable recursive structures. It appears across domains—from optimal control in engineering, to solving Markov Decision Processes in Artificial intelligence, to pricing financial derivatives.
The beauty of the Bellman equation lies in its simplicity and universality. Whether you're training a game-playing AI, routing packets through networks, or allocating resources, you're likely using some form of this equation. It connects Cause and effect, Path dependence, and Recursion into a unified framework for understanding how rational agents should behave over time.