Multilayer Perceptron (MLP)
Multilayer Perceptron (Fully Connected Feedforward Neural Network) · Also known as: MLP, feedforward neural network, fully connected neural network, vanilla neural network, dense neural network
A Multilayer Perceptron is a classic fully connected feedforward neural network trained with the backpropagation algorithm, as formalised by Rumelhart, Hinton & Williams in their landmark 1986 Nature paper. Composed of an input layer, one or more hidden layers of neurons, and an output layer, the MLP learns nonlinear mappings from input features to target outputs and serves as the foundational building block of modern deep learning.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+4 more
When to use it
An MLP is appropriate when the relationship between inputs and outputs is nonlinear and cannot be captured by logistic or linear regression, when you have sufficient labelled data (typically at least several hundred to a few thousand samples per class for modest architectures), and when features are in tabular or fixed-length vector form. The method assumes that all relevant features are presented as numeric inputs and that the training set is representative of the deployment distribution. For structured tabular data with limited samples, gradient-boosted trees often outperform MLPs; for image, audio, or sequence data, specialised architectures (CNNs, RNNs, Transformers) are generally preferred.
Strengths & limitations
- Universal approximation: a sufficiently wide single hidden layer can approximate any continuous function to arbitrary precision.
- Learns complex nonlinear feature interactions automatically, without manual feature engineering.
- Scales well to large datasets; training on GPUs is straightforward with modern frameworks.
- Flexible output layer allows the same architecture to handle binary classification, multiclass classification, and regression by changing the activation and loss function.
- The foundational architecture whose principles — forward pass, backpropagation, weight update — transfer directly to all modern deep learning models.
- Requires substantially more data than linear models; small datasets (<200 samples) lead to overfitting even with regularisation.
- Hyperparameter choices (depth, width, learning rate, regularisation, optimiser) have a large effect on performance and require systematic tuning.
- Training is sensitive to weight initialisation and can converge to poor local minima or stall due to vanishing gradients in deep networks.
- Model outputs are not naturally interpretable; understanding what the network learned requires additional post-hoc tools.
- Computationally more expensive than simpler models; inference latency can be a constraint in resource-limited environments.
Frequently asked
How many hidden layers and neurons should I use?
Start small and simple. A single hidden layer with 64–256 neurons often suffices for tabular datasets; add depth only if the validation loss is still high after tuning. Each doubling of depth increases expressive power but also training difficulty and overfitting risk, so combine added depth with dropout or batch normalisation.
What activation function should I use in hidden layers?
ReLU (Rectified Linear Unit) is the default for most applications because it avoids vanishing gradients, is computationally cheap, and trains quickly. Variants such as Leaky ReLU or GELU are preferred in specific contexts (e.g., GELU in Transformer blocks). Sigmoid and tanh are generally reserved for output layers in binary and bounded-range regression tasks.
How do I prevent overfitting in an MLP?
The most effective strategies are dropout (randomly zeroing neurons during training), L2 weight regularisation, early stopping on the validation loss, and reducing model capacity. Collecting more training data is the most reliable long-term solution. Batch normalisation can also help by acting as a mild regulariser and stabilising training.
When should I choose an MLP over gradient boosting or logistic regression?
Prefer logistic regression for small datasets with near-linear relationships where interpretability is required. Prefer gradient-boosted trees (XGBoost, LightGBM) for moderate-sized tabular datasets, where they typically outperform MLPs with less tuning. Choose an MLP when you have large amounts of data, expect complex nonlinear interactions, or plan to extend to a larger deep learning pipeline.
Sources
- Rumelhart, D. E., Hinton, G. E. & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. DOI: 10.1038/323533a0 ↗
- Goodfellow, I., Bengio, Y. & Courville, A. (2016). Deep Learning (Ch. 6–8). MIT Press. ISBN: 978-0-262-03561-3
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning (Ch. 5). Springer. ISBN: 978-0-387-31073-2
How to cite this page
ScholarGate. (2026, June 3). Multilayer Perceptron (Fully Connected Feedforward Neural Network). ScholarGate. https://scholargate.app/en/deep-learning/multilayer-perceptron
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Logistic RegressionResearch Statistics↔ compare
- Random ForestMachine learning↔ compare
- Recurrent Neural NetworkDeep learning↔ compare
- XGBoostMachine learning↔ compare