A weighted sum, a hard threshold, and a rule that learns only from its mistakes.
The first neuron — the one that started neural networks, and the one that hit the wall.
The fix (layers) was imaginable in 1969. Training the stack took until the 1980s.
80 points, two classes, a clear gap. Built separable on purpose — that gap is the precondition for convergence.
Net input, then take the sign:
z=w⊤x+by^=sign(z)∈{−1,+1}On a misclassified point, nudge the weights toward it:
w←w+ηyxb←b+ηyCorrect points get nothing. Labels in {−1,+1} are what fold both cases into one line.
Converges in ≤(R/γ)2 updates if the data is separable. Otherwise, never.
One frame per correction. Orange line is the boundary; the red ring is the point being fixed; the bars count mistakes per epoch down to zero.
Zero weights → no line → a crude cut → swing until a pass is clean. 5 passes, 26 corrections.
No straight line separates these. Opposite diagonals share a class.
Separable drops to zero and stops. XOR bounces forever — no clean pass, no end.
Separable: both hit 100% — convergence theorem as a number. XOR: both at chance.
Separable 1.0 / 1.0. XOR 0.525 / 0.5125. Two identical bars at chance — the case for going deeper.