Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

NN01 - Edge Detection Intuition: A Single Neuron as Pattern Matching

A hands-on guide to understanding how a single neuron detects patterns in images

Have you ever wondered how neural networks can recognize faces, read handwriting, or detect objects in photos? It all starts with something surprisingly simple: pattern matching.

In this post, we’ll build an edge detector from scratch to understand the fundamental operation at the heart of all neural networks. By the end, you’ll have an intuitive grasp of:

No prior deep learning knowledge required — just basic math intuition.

The Big Picture: What Does a Neuron Actually Do?

A neuron in a neural network does three things:

  1. Multiply each input by a learned weight

  2. Add all the products together (plus a bias term)

  3. Apply an activation function (we’ll use ReLU)

Mathematically:

z=∑ixi⋅wi+b=(x1⋅w1)+(x2⋅w2)+…+(xn⋅wn)+bz = \sum_{i} x_i \cdot w_i + b = (x_1 \cdot w_1) + (x_2 \cdot w_2) + \ldots + (x_n \cdot w_n) + b
(1)
output=ReLU(z)=max⁡(0,z)\text{output} = \text{ReLU}(z) = \max(0, z)
(2)

That’s it! The magic is in what values the weights take — they determine what pattern the neuron responds to.

Intuition: Think of each input as an expert’s opinion, and the weights as how much you trust each expert. A large positive weight means “this input is very important.” A negative weight means “if this input is high, I’m less interested.”

What is ReLU and Why Do We Need It?

ReLU (Rectified Linear Unit) is the simplest activation function:

ReLU(z)=max⁡(0,z)={zif z>00if z≤0\text{ReLU}(z) = \max(0, z) = \begin{cases} z & \text{if } z > 0 \\ 0 & \text{if } z \leq 0 \end{cases}
(3)

It acts as a gate: positive signals pass through unchanged, negative signals get blocked.

Why is this useful? Without an activation function, stacking layers would be pointless — the whole network would collapse into a single linear transformation. ReLU introduces non-linearity, which allows the network to learn complex patterns.

<Figure size 1200x400 with 2 Axes>

Building an Edge Detector

Now let’s see how a neuron can detect a specific pattern: a vertical edge.

Imagine a tiny 5×5 grayscale image (25 pixels). A neuron connected to this image has:

Designing the Weights

To detect a vertical edge (dark on left, bright on right), we set:

When an image with this exact pattern arrives, the positive and negative contributions reinforce each other, giving a high score. When the pattern doesn’t match, contributions cancel out or go negative.

<Figure size 700x600 with 2 Axes>

Testing Our Detector on Different Images

Let’s create several test images to see how our detector responds:

ImageDescriptionExpected Response
Perfect EdgeDark left, bright rightStrong (pattern matches!)
Shifted EdgeEdge in wrong positionReduced (partial match)
Uniform GrayNo edge at allZero (no contrast)
Inverted EdgeBright left, dark rightBlocked (opposite of what we want)
Horizontal EdgeEdge rotates 90°Weak (wrong orientation)
<Figure size 1400x700 with 8 Axes>

Computing the Neuron’s Response

For each image, we compute:

z=∑i,jimage[i,j]×weight[i,j]+bz = \sum_{i,j} \text{image}[i,j] \times \text{weight}[i,j] + b
(4)
output=max⁡(0,z)\text{output} = \max(0, z)
(5)

Note: We’re setting bias b=0b = 0 for now to keep things simple. This lets us focus purely on how the weight pattern matches the input. We’ll explore bias later!

<Figure size 1400x500 with 2 Axes>

Visualizing the Math: Step by Step

Let’s look inside the calculation to see exactly how the score emerges from element-wise multiplication.

<Figure size 1600x400 with 4 Axes>
<Figure size 1600x400 with 4 Axes>
<Figure size 1600x400 with 4 Axes>

The Flattened View: It’s Just a Dot Product!

The 5×5 matrix representation is convenient for visualization, but the actual computation is simply a dot product of two 25-element vectors:

z=x⃗⋅w⃗=x0w0+x1w1+…+x24w24z = \vec{x} \cdot \vec{w} = x_0 w_0 + x_1 w_1 + \ldots + x_{24} w_{24}
(6)

Let’s see what this looks like when we “unroll” the matrices into strips:

<Figure size 1400x500 with 4 Axes>
<Figure size 1400x500 with 4 Axes>

Key Insight: Alignment Determines Response

The neuron’s output depends entirely on how well the input aligns with the weight pattern:

ScenarioWhat HappensResult
Perfect matchDark pixels × negative weights → positive contributions
Bright pixels × positive weights → positive contributions
All reinforce!		High output
Inverted patternDark pixels × positive weights → small contributions
Bright pixels × negative weights → negative contributions		Negative → blocked
No patternEverything averages out~Zero

The Role of Bias: Adjusting Sensitivity

So far we’ve set bias b=0b = 0. But what does bias actually do?

z=∑(x⋅w)+bz = \sum(x \cdot w) + b
(7)

Think of bias as the neuron’s “baseline mood” — a positive bias means it’s eager to fire, while a negative bias means it needs more convincing.

<Figure size 1400x500 with 2 Axes>

Notice how:

In a trained network, each neuron learns its own optimal bias along with its weights.

Different Detectors for Different Patterns

Our vertical edge detector is just one example. By changing the weights, we can detect completely different patterns:

<Figure size 1400x700 with 8 Axes>

From Single Neurons to Deep Networks

A real neural network has thousands or millions of neurons arranged in layers:

This hierarchy — from simple to complex — is why deep learning works so well. And it all starts with the simple pattern matching we’ve explored here!

The key insight: We don’t design these weights by hand. During training, the network automatically discovers what patterns are useful for the task. Backpropagation adjusts each weight to reduce errors, and useful detectors emerge naturally.

Try It Yourself!

Modify the image below and re-run to see how the detector responds:

<Figure size 1600x400 with 4 Axes>

Summary

We’ve built an edge detector from scratch and discovered the core principles that power all neural networks:

ConceptWhat It DoesAnalogy
WeightsDefine what pattern the neuron detectsA template to match against
Weighted SumMeasure how well input matches the patternA similarity score
BiasShift the activation thresholdThe neuron’s “eagerness” to fire
ReLUBlock negative responsesA gate that only passes matches

The Big Ideas

  1. A neuron is a pattern matcher. High output = good match, zero output = poor match.

  2. Weights encode knowledge. The specific values determine what the neuron “looks for.”

  3. Learning = finding good weights. Training adjusts weights until useful patterns emerge.

  4. Depth creates hierarchy. Simple detectors combine into complex recognizers.

This simple mechanism — multiply, sum, activate — repeated billions of times with learned weights, is how neural networks learn to see, read, translate, and even generate art.

Now you understand the atom of deep learning. Everything else is scale and clever architecture!

Neural Networks From Scratch
Neural Networks From Scratch: Understanding Deep Learning From First Principles
Neural Networks From Scratch
NN02 - Training an Edge-Detection Neural Network from Scratch