Reinforcement Learning from Human Feedback

What Makes a Reward Model a Good Teacher? An Optimization Perspective