Naive Bayes

Score a patient against a statistical portrait of each class. Least surprising class wins.

Bayes' theorem, 1763. The "naive" shortcut, 1960s. Still the baseline spam filter to beat.

The whole idea

• Keep a mean and variance per feature per class — that's the model.
• Each feature votes its own likelihood; multiply the votes (the "naive" part).
• Class priors break the tie. Argmax the posterior.
• Work in log-space: products of tiny densities become sums.

The data

768 Pima patients, eight measurements, a diabetes label. All eight in play.

Below: glucose vs BMI. The clouds overlap, but the diabetic cloud leans up and right.

The whole model

Bayes' rule, drop the constant denominator:

Naive assumption — features independent given the class:

Gaussian likelihood per feature, then argmax in log-space:

The model, made visible

The fitted bell curves — one per class per feature. This is all the model stores.

Glucose separates cleanly; BMI overlaps. Naive Bayes weights each by its separation.

Watch the posterior tip

Each test patient scored against both Gaussians. Rings are 1σ and 2σ; the bar is the verdict.

Colored by prediction, red-ringed when wrong. The mistakes cluster in the overlap.

Low glucose: pinned near 0.95 healthy. Climb into the seam and the bar crosses 0.5 — the boundary is where the two bells pull equally.

Scratch vs. library

GaussianNB is the same model. Both score 0.7727 and agree on all 154 test patients.

Not "close to" the library — bit-for-bit the same fitted parameters.

Takeaways

• Fast, thrifty, tiny-data-friendly. The baseline text classifier to beat.
• The independence assumption is wrong and useful: it double-counts correlated features and still ranks well.
• Beats the 65% majority-class floor by a dozen points on Pima.
• Bad at calibration — a good sort key, a bad probability. Logistic regression fixes that later.