On RL Objectives and Credit Assignment for LLMs

less than 1 minute read

Published:

Maximum Entropy RL :

\[J(\theta) = \sum_t \mathbb{E}_{\tau \sim \pi_\theta(\tau)} \left[ r(s_t, a_t) + \beta H\!\left(\pi_\theta(\cdot \mid s_t)\right) \right]\]