The perceptron

A weighted sum, a hard threshold, and a rule that learns only from its mistakes.

The first neuron — the one that started neural networks, and the one that hit the wall.

Where it comes from

McCulloch & Pitts, 1943: a threshold unit is a logic gate. Weights wired by hand.
Rosenblatt, 1958: give it a learning rule. The Perceptron — real hardware, real hype.
Minsky & Papert, 1969: one line can't solve XOR. The first AI winter.

The fix (layers) was imaginable in 1969. Training the stack took until the 1980s.

80 points, two classes, a clear gap. Built separable on purpose — that gap is the precondition for convergence.

Net input, then take the sign:

z = w^{\top}x + b \qquad \hat{y} = \operatorname{sign}(z) \in \{-1, +1\}

On a misclassified point, nudge the weights toward it:

w \leftarrow w + \eta\, y\, x \qquad b \leftarrow b + \eta\, y

Correct points get nothing. Labels in $\{-1, +1\}$ are what fold both cases into one line.

Converges in $\le (R/\gamma)^2$ updates if the data is separable. Otherwise, never.

One frame per correction. Orange line is the boundary; the red ring is the point being fixed; the bars count mistakes per epoch down to zero.

Zero weights → no line → a crude cut → swing until a pass is clean. 5 passes, 26 corrections.

No straight line separates these. Opposite diagonals share a class.

Separable drops to zero and stops. XOR bounces forever — no clean pass, no end.

Separable: both hit 100% — convergence theorem as a number. XOR: both at chance.

Separable 1.0 / 1.0. XOR 0.525 / 0.5125. Two identical bars at chance — the case for going deeper.

The atom of neural nets: weighted sum, activation, error-driven update. Everything since is this scaled.
A real convergence guarantee — but only on separable data, and it stops at any separator, not the best one.
The limit is the lesson: one line can't do XOR. When a model can't express the answer, change the model.
The step's flat derivative blocks stacking; a line can't bend. Fix both → differentiable activation + layers.
That's the multilayer perceptron. XOR is where this chapter ends and the next begins.