Hyperparameter

A hyperparameter is a setting or configuration value that you choose before training a Machine Learning model, rather than one the model learns from data. Think of it as a knob you turn to guide how the learning process unfolds.

Common examples include the learning rate (how big each step of improvement is), the number of layers in a Neural Network, or the batch size (how many examples to process together). Unlike regular parameters—which the model adjusts internally to fit your data—hyperparameters shape the entire learning strategy from the outside.

Choosing good hyperparameters is part craft, part science. Too high a learning rate and your model overshoots; too low and it crawls. The process of finding optimal values is called Hyperparameter tuning, often done through Grid search, Random search, or more sophisticated methods like Bayesian optimization.

Hyperparameters matter across domains: in neural networks, Support Vector Machines, Decision trees, and Ensemble methods like Random forests. They're why Machine Learning remains as much art as engineering—no universal formula exists; you must experiment, measure, and refine for your specific problem.

Related

Machine Learning, Overfitting, Validation, Computational model, Algorithmic bias