Stratified validation

Stratified validation is a computational and statistical technique for assessing model performance across distinct subgroups or strata within data. Rather than treating all observations equally, this approach ensures that each meaningful segment—defined by categories like income level, geographic region, or demographic group—receives proportional representation during validation.

The method divides data into homogeneous layers before splitting into training and testing sets, preserving the distribution of each stratum. This is especially valuable when certain groups are rare or when performance must be reliable across all populations, not just the majority. It guards against skewed results that might emerge if one stratum dominates the validation set.

Stratified validation appears across domains: software engineering uses it to test performance across user demographics; biomedical research applies it across patient populations; ecological studies employ it across habitat types or trophic levels. The approach embodies a commitment to equitable assessment—ensuring that models work fairly for everyone they're meant to serve.

Related concepts include failure analysis through subgroups, ratio-based sampling, and robustness testing.

Related

Machine learning validation, Cross-validation, Statistical sampling, Data representation, Demographic parity, Model fairness