方法对比

	微调强化学习	微调Transformer
领域	深度学习	深度学习
方法族	Machine learning	Machine learning
起源年份	2017–2022	2017–2019
提出者	Christiano, P. et al.; Ouyang, L. et al.	Vaswani et al. (architecture); fine-tuning paradigm popularised by Howard & Ruder, Devlin et al.
类型	Policy adaptation via fine-tuning	Transfer learning / supervised fine-tuning
开创性文献	Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.	Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
别名	RL fine-tuning, policy fine-tuning, RLHF, reinforcement learning from human feedback	Transformer fine-tuning, pre-trained transformer fine-tuning, task-adaptive transformer, downstream-tuned transformer
相关	5	4
摘要	Fine-Tuned Reinforcement Learning adapts a pre-trained policy or model to a new task or behavioral objective using reinforcement signals — including human feedback — rather than retraining from scratch. Popularized by RLHF, it is the core technique behind aligning large language models and adapting deep RL agents to specialized environments with minimal additional data.	Fine-tuning a Transformer adapts a large pre-trained model — such as BERT, GPT, or ViT — to a specific downstream task by continuing gradient-based training on a labelled target dataset. This two-stage paradigm (pre-train then fine-tune) consistently achieves state-of-the-art results across NLP and computer vision tasks with far less task-specific data than training from scratch.