Machine learningDeep Learning, Language Models, RLHF Alternatives
直接偏好优化
直接偏好优化（Direct Preference Optimization, DPO）是由 Rafailov 等人于 2023 年提出的一种训练方法，它可以在不显式构建奖励模型的情况下，使语言模型与人类偏好对齐。通过直接优化偏好对（更优响应 vs. 更劣响应），DPO 相较于基于人类反馈的强化学习（Reinforcement Learning from Human Feedback, RLHF）简化了训练流程。
阅读完整方法
仅限会员
登录使用免费账户登录即可阅读本节。
Method map
The neighbourhood of related methods — select a node to explore.
来源
- Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290. link ↗
如何引用本页
ScholarGate. (2026, June 3). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. ScholarGate. https://scholargate.app/zh/deep-learning/direct-preference-optimization
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
Compare side by side →