Fully Convolutional Network (FCN)
Also known as: FCN, fully convolutional network, FCN-32s, FCN-16s, FCN-8s, dense prediction network
The Fully Convolutional Network (FCN), introduced by Long, Shelhamer, and Darrell at CVPR 2015, was the first end-to-end deep learning architecture trained to produce dense pixel-wise semantic segmentation maps from images of arbitrary size. By replacing the fully connected layers of a classification CNN with convolutional layers and adding learned upsampling through transposed convolutions and skip connections, FCN enabled the direct prediction of a class label for every pixel in an image, establishing the template for all subsequent segmentation architectures including U-Net and DeepLab.
Key highlights
- First architecture to train a deep network end-to-end for pixel-wise semantic segmentation, removing hand-crafted post-processing pipelines.
- Accepts arbitrary input spatial dimensions at inference time because no fully connected layer constrains the input size.
- Transfer learning from ImageNet-pretrained classifiers greatly reduces the labelled data required for segmentation.
- Skip connections at multiple pooling scales recover spatial detail lost during downsampling, improving boundary accuracy.
- The FCN design principle (fully convolutional + upsampling + skip connections) is the direct architectural template for U-Net, SegNet, DeepLab, and most modern segmentation networks.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
FCN is applicable whenever a dense, pixel-level class prediction is required from image data — for example, road-scene parsing, biomedical tissue segmentation, satellite land-cover mapping, or industrial inspection. It is appropriate when a labelled pixel-wise annotation dataset is available (or can be bootstrapped from a classification dataset with weak supervision), when the problem can be cast as assigning one of C mutually exclusive classes to each pixel, and when a GPU with sufficient memory is available for training. FCN assumes that the label of each pixel depends on its local spatial context and that this relationship can be modelled by a convolutional hierarchy. It is less suited to very high-resolution imagery where multi-scale tiling strategies or memory-efficient variants are needed, or to instance-level segmentation (where Mask R-CNN and its descendants are preferred).
Strengths & limitations
- First architecture to train a deep network end-to-end for pixel-wise semantic segmentation, removing hand-crafted post-processing pipelines.
- Accepts arbitrary input spatial dimensions at inference time because no fully connected layer constrains the input size.
- Transfer learning from ImageNet-pretrained classifiers greatly reduces the labelled data required for segmentation.
- Skip connections at multiple pooling scales recover spatial detail lost during downsampling, improving boundary accuracy.
- The FCN design principle (fully convolutional + upsampling + skip connections) is the direct architectural template for U-Net, SegNet, DeepLab, and most modern segmentation networks.
- Fixed receptive field per layer means the model can struggle with objects that are very large relative to the receptive field or very small relative to the stride.
- Transposed convolution upsampling can produce checkerboard artefacts in predicted maps if not carefully initialised or regularised.
- The skip-connection fusion (add rather than concatenate) discards complementary channel information; later architectures (U-Net) use concatenation for richer fusion.
- FCN-32s/16s/8s variants require sequential training stages; joint end-to-end training of all skip levels requires careful learning-rate scheduling.
- Global context is limited because standard convolutions and pooling do not model long-range dependencies (addressed in later work by dilated convolutions and attention mechanisms).
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the difference between FCN-32s, FCN-16s, and FCN-8s?
All three variants share the same convolutional backbone. FCN-32s upsamples the stride-32 deep feature map directly back to the image resolution in a single step, giving coarse predictions. FCN-16s first upsamples by 2× and adds the stride-16 (pool4) feature map before a final 16× upsample, recovering some spatial detail. FCN-8s additionally incorporates the stride-8 (pool3) feature map for the finest resolution. Each step improves boundary sharpness at the cost of an additional training stage.
How does FCN differ from U-Net?
U-Net, introduced by Ronneberger et al. (2015) shortly after FCN, adopts the same encoder-decoder principle but uses concatenation (rather than addition) to merge skip connections, retaining more channel information in the expanding path. U-Net also uses a symmetric, mirror-like decoder architecture and was designed specifically for biomedical images where training data are scarce. FCN is the architectural predecessor; U-Net is the dominant variant in medical imaging.
What evaluation metric should I use for semantic segmentation?
Mean Intersection-over-Union (mIoU), averaged across all classes, is the standard metric in semantic segmentation benchmarks (Pascal VOC, Cityscapes, ADE20K). Pixel accuracy is a secondary metric but misleading when classes are imbalanced because the model can score high accuracy by predicting only the dominant background class.
Do I need a large labelled dataset to train an FCN?
Training from scratch typically requires tens of thousands of pixel-labelled images. In practice, the standard approach is to initialise the convolutional backbone from an ImageNet-pretrained classifier (e.g., VGG-16) and fine-tune on the target segmentation dataset. This transfer-learning strategy makes FCN feasible on datasets with a few hundred to a few thousand labelled images, though more data consistently helps.
Sources
- 1.Long, J., Shelhamer, E., & Darrell, T. (2015). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3431–3440.
- 2.Shelhamer, E., Long, J., & Darrell, T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.
- 3.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning (Ch. 9). MIT Press.ISBN 978-0-262-03561-3
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Fully Convolutional Network (FCN). ScholarGate. https://scholargate.app/deep-learning/fully-convolutional-network