مقایسهٔ روش‌ها

	بهینه‌ساز گرادیان کاهشی تصادفی (SGD) با مومنتوم / بهینه‌ساز Adam	نرمال‌سازی دسته‌ای (Batch Normalization)
حوزه	یادگیری عمیق	یادگیری عمیق
خانواده	Machine learning	Machine learning
سال پیدایش	2015	2015
پدیدآور	Rumelhart, Hinton & Williams (momentum SGD, 1986); Kingma & Ba (Adam, 2015)	Ioffe, S. & Szegedy, C.
نوع	First-order adaptive stochastic optimizer	Normalization technique (applied per mini-batch during training)
منبع بنیادین	Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR 2015). arXiv:1412.6980.	Ioffe, S. & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning (ICML), PMLR 37, 448–456.
نام‌های دیگر	Adam, Adam optimizer, SGD with momentum, momentum SGD	BatchNorm, BN, batch norm, mini-batch normalization
مرتبط	1	1
خلاصه	Stochastic Gradient Descent (SGD) with momentum and its adaptive descendant Adam are the foundational parameter-update algorithms used to train virtually every modern deep learning model. Momentum SGD was formalised by Polyak (1964) and brought into neural network training by Rumelhart, Hinton, and Williams (1986). Adam, introduced by Kingma and Ba at ICLR 2015, extended the momentum idea by also maintaining a running average of squared gradients, producing per-parameter adaptive learning rates that make it the default optimizer in contemporary deep learning practice.	Batch Normalization is a training technique introduced by Sergey Ioffe and Christian Szegedy in 2015 that normalizes the pre-activation outputs of each layer using the mean and variance computed over the current mini-batch. By stabilizing the input distribution to each layer throughout training, it substantially reduces internal covariate shift, enabling the use of higher learning rates and making deep networks train faster and more reliably.