Skip to content
← The Lab

The Learning Machine

A neural network learning live in your browser. Pick a shape; watch the boundary form.

Pick a shape, scatter some labelled dots, and press go. A neural network, training live in your browser, gropes for the boundary between the classes, gets it wrong, and then, over a few hundred steps, bends a curve into exactly the right place. No magic, no pretrained weights. You are watching learning happen.

The concept

A network is a stack of linear maps with a nonlinearity between them, and "learning" is just adjusting its weights to make its output match the labels. It starts random. Each step, it measures how wrong it is, computes which direction every weight should nudge to be less wrong, and takes a small step that way. Repeat, and a featureless blob of parameters becomes a decision boundary.

The math

Each layer computes $\mathbf{a} = \sigma(W\mathbf{x} + \mathbf{b})$, with $\sigma$ a nonlinearity. Training minimises a loss $L$ by gradient descent:

W←W−η ∂L∂WW \leftarrow W - \eta\, \frac{\partial L}{\partial W}

The gradient $\partial L / \partial W$ for every weight, in every layer, comes from one pass of the chain rule, backpropagation, sweeping the error backward from the output. The learning rate $\eta$ sets the step size: too big and it thrashes, too small and it crawls. The experiment runs this hand-rolled MLP and draws the boundary re-forming after each step.

Why it stays strange

Nothing in the setup knows what a "spiral" or a "circle" is. The network discovers the boundary purely by descending an error surface it cannot see the shape of, and the same three lines, predict, measure error, step downhill, scale up to models that write and reason. Watching it here strips the mystique: intelligence, at this layer, is a ball rolling downhill in a very high-dimensional valley.

Further reading

  • Rumelhart, Hinton, Williams (1986), "Learning representations by back-propagating errors."
  • Goodfellow, Bengio, Courville, Deep Learning.
@article{rumelhart1986,
  author  = {Rumelhart, David E. and Hinton, Geoffrey E. and Williams, Ronald J.},
  title   = {Learning Representations by Back-Propagating Errors},
  journal = {Nature},
  volume  = {323},
  pages   = {533--536},
  year    = {1986}
}