AlexNet
AlexNet (Krizhevsky–Sutskever–Hinton Deep Convolutional Neural Network) · Also known as: AlexNet, Krizhevsky net, SuperVision CNN, ImageNet CNN 2012, deep convolutional neural network (AlexNet)
AlexNet is a deep convolutional neural network (CNN) introduced by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton in 2012. It won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012) with a top-5 error rate of 15.3%, outstripping the runner-up by more than 10 percentage points and reigniting broad interest in deep learning. The architecture introduced or popularised several techniques — ReLU activations, dropout regularisation, and multi-GPU training — that became standard practice across the field.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
AlexNet is appropriate when the task involves visual recognition (image classification, feature extraction) and sufficient labelled image data are available. As a historical architecture it is now largely superseded for production use by deeper and more efficient networks such as ResNet, VGG, and EfficientNet. However, AlexNet remains valuable as a teaching model and as a lightweight feature extractor when computational resources are very constrained. The core design principles it established — deep stacked convolutions, ReLU, dropout, large-scale GPU training — apply to virtually all modern CNN design. A dataset of at least several thousand labelled images is needed for training from scratch; transfer learning can mitigate data requirements substantially.
Strengths & limitations
- Demonstrated definitively that deep CNNs trained end-to-end on raw pixels can outperform hand-crafted feature pipelines by a large margin on a large-scale benchmark.
- ReLU activation enables training of deep networks without the vanishing-gradient problem, dramatically accelerating convergence.
- Dropout regularisation provides an effective, low-overhead technique to reduce overfitting in large fully connected layers.
- Multi-GPU training strategy introduced a practical parallelism scheme that influenced the design of subsequent large models.
- Architecture is simple enough to serve as a clear teaching model and a baseline for understanding deeper architectures.
- Computationally expensive to train from scratch; requires a GPU and large labelled datasets.
- Superseded in accuracy by deeper architectures (VGGNet, GoogLeNet, ResNet, EfficientNet) on most modern benchmarks.
- Local Response Normalisation is less effective than batch normalisation, which was introduced later and is now standard.
- The fully connected layers contain the vast majority of the model's ~60 million parameters, making it memory-heavy and prone to overfitting on small datasets.
- Fixed input size (224×224) and no built-in mechanisms for multi-scale recognition.
Frequently asked
Why did AlexNet have such a large impact in 2012?
The gap between AlexNet's error rate (15.3%) and the runner-up (26.2%) on ILSVRC 2012 was unprecedented. It demonstrated conclusively that deep neural networks trained on raw pixels with large datasets and GPU compute could dramatically outperform the best hand-crafted feature pipelines, overturning a decade of assumptions in computer vision.
What is the difference between AlexNet and a standard CNN?
AlexNet was, at its time, an unusually deep CNN (8 learnable layers). Its distinctive contributions were using ReLU activations throughout (instead of sigmoid/tanh), applying dropout in the fully connected layers to combat overfitting, using local response normalisation, and distributing computation across two GPUs. These design choices together — not the convolutional structure per se — produced its gains.
Should I use AlexNet for a new image classification task today?
For most practical tasks, no. Modern architectures such as ResNet-50, EfficientNetV2, or Vision Transformers achieve higher accuracy with fewer parameters. AlexNet is best used as a teaching tool or as a lightweight baseline when hardware resources are very limited. Transfer learning from pre-trained ResNet or EfficientNet weights will almost always outperform a from-scratch AlexNet.
How does dropout in AlexNet prevent overfitting?
During each forward pass in training, each neuron in the large fully connected layers is independently set to zero with probability 0.5. This prevents neurons from co-adapting in ways that only work on the training data, effectively training an exponential number of different thinned network architectures simultaneously. At test time all neurons are active, with their weights halved to maintain the same expected output.
Sources
- Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25, 1097–1105. (Republished: Communications of the ACM, 60(6), 84–90, 2017.) DOI: 10.1145/3065386 ↗
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning (Ch. 9: Convolutional Networks). MIT Press. ISBN: 978-0-262-03561-3
- LeCun, Y., Bengio, Y., & Hinton, G. E. (2015). Deep Learning. Nature, 521, 436–444. DOI: 10.1038/nature14539 ↗
How to cite this page
ScholarGate. (2026, June 3). AlexNet (Krizhevsky–Sutskever–Hinton Deep Convolutional Neural Network). ScholarGate. https://scholargate.app/en/deep-learning/alexnet
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
Compare side by side →