A convolutional neural network looks at a photo of a panda and correctly identifies it. Add a calculated noise pattern, invisible to the human eye, and the same network identifies the same panda as a gibbon with 99.3% confidence. The noise was not random, it was computed by following the gradient of the model’s loss function with respect to the input pixels, the same gradient mechanism we covered in the previous article on neural networks. Beyond just gradients, these attacks succeed because CNNs use a hierarchical structure of abstract feature detectors that provides an attacker with multiple levels of potential manipulation.

The previous article covered how neural networks learn through backpropagation and why gradient-based learning creates the fundamental attack surface for adversarial machine learning. The CNN architecture adds its own set of vulnerabilities to the standard list by relying on spatial assumptions and hierarchical features that remain susceptible to manipulation across various layers and pooling stages.

Why CNNs exist

A fully connected architecture requires every neuron in one layer to connect to every neuron in the next as we saw in the previous article. Feed a 256×256 colour image into a fully connected network and the first layer alone needs roughly 50 million parameters. That is computationally absurd, but the real problem is not speed. It is that the network has no concept of spatial structure. Pixel (0,0) and pixel (255,255) are treated as equally related. A fully connected network processing images is like reading a book by shuffling every word into a random order and hoping the meaning survives.

CNNs solve this by enforcing a structural assumption: nearby pixels matter more than distant pixels, and the same pattern is worth detecting regardless of where it appears in the image. These two principles, locality and translation invariance, are what make CNNs work. They are also what make them breakable in specific and predictable ways.

How convolution works

A convolutional layer slides a small filter (typically 3×3 or 5×5 pixels) across the input image, computing the dot product between the filter weights and the pixel values at each position. A 2D grid known as a feature map serves as the output by using individual values to represent the intensity of patterns detected at specific locations.

The architecture employs several specialised filters including those that recognise vertical edges, horizontal edges, and diagonal transitions. The network learns these filters during training by adjusting their weights through backpropagation, exactly as described in the previous article. The only difference is that instead of learning individual neuron weights, the network learns filter weights that are shared across every spatial position in the input.

This weight sharing is what gives CNNs their efficiency. A 3×3 filter has 9 learnable parameters regardless of whether the input image is 32×32 or 1024×1024. CNNs possess translation invariance which means a filter will identify a particular feature in the center of an image just as easily as in the top-left corner.

For an adversary, weight sharing means something specific. If you know (or can infer) what a filter detects, you know it detects that pattern everywhere. A perturbation designed to confuse one filter will confuse it at every spatial position. You do not need to craft location-specific noise. One adversarial patch works across the entire feature map.

The feature hierarchy and why it matters for attacks

In a hierarchical build, the first convolutional layer of a CNN identifies basic textures and gradients alongside various edges to establish a foundation for the model. The second layer combines those edge detections to recognise corners, curves, and more complex textures. Deeper layers combine those mid-level features into high-level representations: object parts, shapes, entire objects.

Consider a CNN trained to recognise handwritten digits. The first layer detects the straight edges and curves that form the strokes. “In the second layer, the network learns to identify stroke connections including the specific way a horizontal bar crosses a vertical line to create the number seven. Deeper layers encode the full shape of each digit as a composite of these structural features.

This hierarchy creates a layered attack surface. An adversary can choose where in the feature hierarchy to apply pressure.

Targeting early layers means manipulating edge-level features. Small, high-frequency perturbations (the kind used in FGSM and PGD attacks) operate here. These perturbations are typically imperceptible to humans because human vision is relatively insensitive to small pixel-level changes. The network’s early filters, however, are extremely sensitive to them. A perturbation of a few pixel values can flip the output of an edge detector, which cascades through every subsequent layer.

Targeting deeper layers means manipulating the high-level features the network uses for its final classification. Style transfer attacks and feature-space adversarial examples operate at this level. Instead of optimising for raw pixel perturbation, the attacker optimises for a specific internal representation. The perturbed input might look dramatically different from the original to a human, but the network’s deep layers see the same abstract feature pattern as the target class.

Targeting intermediate layers is where things get subtle. An attacker who understands the network’s mid-level feature detectors can craft adversarial examples that are structurally coherent at the edge level and visually plausible at the object level, but that disrupt the specific combination of mid-level features the network relies on for discrimination. These attacks are harder to detect because the input looks “right” at both the pixel level and the object level. The corruption lives in the abstract relational structure between them.

Pooling

After each convolutional layer, a pooling layer downsamples the feature map. Max pooling takes the maximum value within a small window (typically 2×2). Average pooling takes the mean. Either way, the spatial resolution is halved and information is permanently discarded.

Pooling serves two purposes. Beyond lowering computational requirements for deeper layers, pooling creates spatial invariance so that minor shifts in the input data do not significantly alter the output of the network. A cat shifted three pixels to the right still activates the same pooled feature map.

For a red teamer, pooling is interesting because it defines what the network does not see. Max pooling retains only the strongest activation in each window. If an adversarial perturbation falls in a region where the pooling operation discards it, it has no effect on the downstream computation. Conversely, if a perturbation lands on the maximum activation within a pooling window, it propagates through every subsequent layer.

Effective adversarial perturbations against CNNs are not uniformly distributed across the input. They are concentrated on the spatial positions that survive pooling. An attacker with access to the model architecture avoids guesswork by computing exactly which positions are retained by each pooling layer and concentrating their perturbation efforts on those specific points. The result is a smaller, more efficient, and harder-to-detect adversarial input.

Fully connected layers

The output of the final pooling layer is flattened into a one-dimensional vector and fed into one or more fully connected layers. These are identical to the layers in the standard neural networks covered in the previous article. They perform the final classification by mapping the high-level feature representation to class probabilities.

From an adversarial perspective, the fully connected layers are the least interesting component of a CNN. The lack of security stems from the fact that these models are susceptible to the same vulnerabilities already mentioned which involve gradient-based attacks and weight extraction through repeated querying along with a high sensitivity to training data distribution. The CNN-specific attack surfaces live in the convolutional and pooling layers. The fully connected layers just inherit whatever corruption made it through.

The assumptions CNNs depend on, and what happens when they break

Every model operates on assumptions, and those assumptions define where the model fails. CNNs make four that matter for adversarial work.

Locality. CNNs assume that local pixel relationships carry more information than global ones. This is why convolutional filters are small. But adversarial patches violate this assumption deliberately. A small, visually conspicuous patch placed anywhere in an image can override the network’s classification by generating feature activations so strong that they dominate the global representation. Researchers have demonstrated printed patches that, when held in front of a camera, cause object detectors to miss people entirely. The patch does not need to be near the person. It just needs to generate features that overwhelm the network’s representation of a human shape.

Translation invariance. CNNs assume a cat is a cat regardless of where it appears in the frame. Weight sharing enforces this. But an attacker can exploit translation invariance by ensuring that an adversarial perturbation is effective at every position, because the same filter processes every position identically. A perturbation designed to confuse one spatial location confuses all of them.

Sufficient and representative training data. CNNs require large, labelled datasets to learn meaningful features. If the training data is biased, the learned features encode that bias. A facial recognition CNN trained predominantly on one demographic will learn feature representations optimised for that demographic and perform poorly on others. An adversary does not need to tamper with the model if the training data already did the work.

Normalised input distribution. CNNs expect input pixel values within a specific range (typically 0 to 1 or -1 to 1). Input outside this range can cause unstable activations, gradient explosion, or silent classification failures. This is not a theoretical concern. Production image pipelines that resize, crop, or recompress images before feeding them to a CNN can inadvertently shift the input distribution far enough from the training distribution to degrade accuracy. An attacker who controls the image processing pipeline upstream of the CNN can degrade model performance without ever touching the model itself.

What this means in practice

CNN-based systems are deployed in environments where classification errors have operational consequences: facial recognition at border crossings, autonomous vehicle perception, medical imaging analysis, malware detection from binary visualisation, phishing page detection from screenshots.

The attack surfaces described above are not theoretical. Adversarial patches that cause object detectors to ignore pedestrians have been demonstrated in physical environments. Adversarial eyeglass frames that defeat facial recognition systems were published by researchers at Carnegie Mellon in 2016. Adversarial perturbations that cause medical imaging classifiers to reverse their diagnoses have been demonstrated on production-grade models.

The common thread is that the attacker does not need to compromise the model’s weights, intercept its traffic, or access its training data. The model’s architecture, its convolutional filters, its pooling decisions, its feature hierarchy, provides enough information to construct inputs that reliably produce wrong outputs. The attack surface is the architecture itself.

Defending CNN-based systems

Input preprocessing. Techniques like JPEG compression, spatial smoothing, and input quantisation can destroy high-frequency adversarial perturbations before they reach the model. These defences are cheap to deploy and effective against basic attacks like FGSM. They are less effective against optimisation-based attacks (PGD, C&W) that account for preprocessing in their perturbation generation.

Adversarial training. Augmenting the training dataset with adversarial examples forces the network to learn features that are robust to known perturbation types. This is the most widely studied defence and the most computationally expensive. It typically reduces clean accuracy (the model performs slightly worse on unperturbed inputs) and provides no guarantee against novel attack methods the model was not trained against.

Feature squeezing. Reducing the colour depth of input images or applying spatial smoothing at inference time can eliminate subtle perturbations. The defence works by comparing the model’s output on the original input versus the squeezed input. If the outputs diverge significantly, the input is flagged as potentially adversarial.

Certified defences. Randomised smoothing and interval bound propagation provide mathematical guarantees that the model’s output will not change for perturbations within a specified L-norm bound. These defences are provably correct within their specified radius but that radius is often too small to cover practical attack scenarios.

Monitoring feature map activations. Adversarial inputs often produce unusual activation patterns in intermediate layers, activations that fall outside the distribution observed during training. Monitoring these activations at inference time can detect adversarial inputs that bypass input-level defences. This approach is architecturally specific and requires per-model calibration.

None of these defences are complete. Each one trades off clean accuracy, computational cost, and robustness against specific attack classes. A defence strategy that protects against FGSM may fail against C&W. A defence that catches pixel-level perturbations may miss adversarial patches. The most honest assessment is that there is no known defence that makes a CNN provably robust against all adversarial inputs at practical perturbation magnitudes.

Closing

The interesting vulnerability in CNNs is not that adversarial examples exist. That is a property inherited from neural networks generally. The CNN-specific vulnerability is that the architecture’s spatial assumptions, its locality, its translation invariance, its hierarchical feature extraction, give an attacker a structured map of what the network pays attention to and what it ignores. The convolutional structure of a CNN does more than process images by telling an attacker exactly where to apply pressure which makes these attacks architectural in nature unlike the high-dimensional noise searches required for fully connected networks.