Loss function

A loss function is a mathematical function that measures how far a model's predictions deviate from actual values. It quantifies error in a single number, serving as a compass for improvement. During training, optimization algorithms adjust model parameters to minimize this loss—the lower the loss, the better the model performs.

Loss functions are foundational to neural networks, Machine learning, and statistical modeling. Different tasks demand different loss functions: Regression problems often use mean squared error, while Classification tasks favor cross-entropy. The choice of loss function shapes what the model learns to optimize for, making it a critical design decision.

Think of it as a scoreboard: if your model predicts tomorrow's temperature as 75°F but it's actually 68°F, the loss function penalizes that gap. Over thousands of examples, the model refines itself to minimize total penalty. This elegantly transforms the vague goal of "be accurate" into a concrete, computable target.

Loss functions also appear in game-theoretic contexts, economic modeling, and anywhere systems learn from feedback.

Related

Machine learning, Neural networks, Gradient descent, Optimization (mathematics), Model training, Regression, Classification