方法证据记录
SGD with Momentum / Adam Optimizer
Stochastic Gradient Descent (SGD) with momentum and its adaptive descendant Adam are the foundational parameter-update algorithms used to train virtually every modern deep learning model. Momentum SGD was formalised by Polyak (1964) and brought into neural network training by Rumelhart, Hinton, and Williams (1986). Adam, introduced by Kingma and Ba at ICLR 2015, extended the momentum idea by also maintaining a running average of squared gradients, producing per-parameter adaptive learning rates that make it the default optimizer in contemporary deep learning practice.
源记录
引文逐字复制自方法源记录。这些引文不代表任何层级的验证。
Stochastic Gradient Descent with Momentum and Adaptive Moment Estimation (Adam)
分类方法记录 · ml-model / deep-learning
- Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR 2015). arXiv:1412.6980. · URL
- Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. · DOI 10.1038/323533a0
- Polyak, B. T. (1964). Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4(5), 1–17. · DOI 10.1016/0041-5553(64)90137-5
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning (Ch. 8: Optimization for Training Deep Models). MIT Press. · ISBN 978-0-262-03561-3
精选声明
声明已持久化到证据分类账中，每个声明都有自己的评估。
尚无精选声明
当分类账中没有声明时，此视图不会自行创建声明评估。
相关方法
从方法图中生成，显示为机器建议的关系 — 不推断任何证据声明。