Normalization and standardization

Put every feature on a common scale before the model ever runs.

Distance doesn't care about meaning — only magnitude. Fix the ruler, not the data.

The problem

Glucose lives in the hundreds; diabetes-pedigree lives between 0 and 2.4.
A 30-unit glucose gap contributes 900 to squared distance; a 0.5 pedigree gap contributes 0.25.
To k-NN or k-means, "nearest" secretly means "similar glucose."
Nobody chose that. It's an accident of units — and it wrecks distance-based models.

Eight features, one shared axis. Insulin and glucose stretch across; the rest are crushed near zero.

Min-max to a fixed [0, 1] box:

x' = \dfrac{x - x_{\min}}{x_{\max} - x_{\min}}

Standardization — the z-score, mean 0 and std 1:

z = \dfrac{x - \mu}{\sigma}

Robust scaling — median and IQR, outlier-proof:

x' = \dfrac{x - \operatorname{median}(x)}{\operatorname{IQR}(x)}

Each scaler is a fit (read a statistic) plus a transform (apply it).
Fit on the training split only. Apply those frozen numbers to the test split.
Never scale the whole dataset before splitting — that leaks test statistics into training.
A leaked scaler inflates your held-out score into a number you can't reproduce.

Same query, same data. Only the ruler changes: raw space → standardized space.

Raw: 15 neighbors vote 4 diabetes / 11 no → no diabetes. Standardized: 8 / 7 → diabetes.

Fixed k-NN, same split, four runs. Test accuracy — fraction of held-out patients called right (unitless, 0–1).

Raw 0.745 → standardized 0.771. No new information, just a change of units.

Scale for distance (k-NN, k-means, SVM, PCA), gradient descent (linear/logistic, nets), and penalties (ridge, lasso).
Don't bother for trees — threshold splits are immune to monotonic rescaling.
Standardization is the default; min-max for a bounded range; robust when outliers dominate.
Always fit on train, transform everything. No test statistics in the transform.