The simplest classifier
One feature. One line. Everyone above it, sick; below it, healthy.
The dumbest model that could work — and the bar everything else has to clear.
Why start here
- Before a neural network, know what one line scores. That's the floor.
- Most people never measure the floor. Then they ship a model that barely clears it.
- A one-feature threshold is a decision stump: a tree with one split.
- It's also the atom bigger models are built from — boost stumps → AdaBoost, stack splits → a forest.
The data
768 diabetes patients, eight measurements each. We use one: blood glucose.
The two groups overlap, but the diabetic readings lean high.
The whole model
Predict diabetic when glucose clears a cutoff:
y^i(θ)=1[gi≥θ]
"Training" is one line of search — the cutoff with the highest accuracy:
θ∗=argθmaxN1i∑1[y^i(θ)=yi]
That's it. An argmax over one integer.
Watch the search
Every frame is one real iteration. Red rings are the cutoff's current mistakes;
the lower panel traces accuracy as the line moves.
The trade-off you can't escape
Low cutoff: catch every diabetic, flag half the healthy. High cutoff: the
reverse. No setting zeroes both mistakes.
Where you put the line is a decision about which mistake you'd rather make.
Scratch vs. library
DecisionTreeClassifier(max_depth=1) is the same model. The bars land on top
of each other.
Takeaways
- Fit the stump first, every time. It's the number your real model has to beat.
- Free feature ranker: hand it every column, the split it picks is your top signal.
- The atom of the course: boosting and trees are stumps assembled.
- Reach for something heavier when one straight cut leaves accuracy on the table — which is most real problems. That's why the course keeps going.