> ML SERIES :: 12 CNNs

Page 12 of 14 · Neural networks & security

Convolutional networks: learning to see

A 1000 × 1000 photo has a million pixels. Connecting each one to every neuron in a dense layer would need billions of weights, and the network would have to relearn "edge" separately for every position in the image. Convolutional neural networks (CNNs) solve both problems by sliding one small filter across the whole image.

Sinclair College - Dr. Mike Libassi

Slide a filter across an image

The left grid is a tiny 12 × 12 grayscale image; select pixels to draw. A 3 × 3 filter (also called a kernel) slides over every position. At each stop it multiplies the 9 pixels under it by its 9 weights and adds them up. The results form a feature map that lights up wherever the image matches the filter's pattern.

input image (12 × 12)

filter (3 × 3)

feature map (10 × 10)

cyan = strong positive match, amber = strong negative match
Press "Step the filter" to see the calculation at one position.

Pooling: keep the strongest signal

After convolution, a ReLU zeroes out negative values, and a pooling layer shrinks the map. Max pooling looks at each 2 × 2 block and keeps only the largest value. The map gets four times smaller, and small shifts in the image barely change the result.

after ReLU (10 × 10)

2 × 2 max pool (5 × 5)

This updates live as you draw or change the filter above.

A whole CNN

In the demo you chose the filter. In a real CNN, the network learns the filter weights through backpropagation (page 11), and each layer has dozens or hundreds of filters.

image→conv + ReLU→pool→conv + ReLU→pool→dense→softmax: "cat" 0.93

Early layers learn simple filters such as edges and color blobs. Middle layers combine those into textures and shapes. Late layers respond to whole objects such as wheels, faces, or text. This hierarchy is what AlexNet demonstrated in 2012.

weight sharing

One 3 × 3 filter has 9 weights no matter how big the image is. A dense layer connecting the 144 input pixels to the 100 outputs here would need 14,400.

translation invariance

The same filter scans everywhere, so a pattern is recognized wherever it appears in the image.

CNNs in security

CNNs are not only for photos. Malware researchers convert executable files into grayscale images, one byte per pixel, and train CNNs to spot family resemblances, since code from the same family often has a visually similar layout. CNNs also power face recognition, the reading of CAPTCHAs (which is why simple text CAPTCHAs no longer work), and the detection of phishing pages that imitate a bank's logo.

import torch.nn as nn
model = nn.Sequential(
    nn.Conv2d(1, 16, kernel_size=3), nn.ReLU(), nn.MaxPool2d(2),   # 16 learned 3x3 filters
    nn.Conv2d(16, 32, kernel_size=3), nn.ReLU(), nn.MaxPool2d(2),
    nn.Flatten(),
    nn.Linear(32 * 5 * 5, 10))                                 # 10 classes, e.g. digits 0-9 on 28x28 input