Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, Boris Hanin
October 2024
PDF Code Poster
Type
Conference paper
Publication
International Conference on Learning Representations (ICLR), 2025
Language Models Direct Preference Learning DPO Likelihood Displacement Alignment

Related