Iterative response generation

Recently context language models proposed giving models a text file to freely edit which is synced to be its conversation history at each tool call. This allows the model to edit and preserve its context history as it chooses when it moves from one call to the next.

Can we generalize this for response generation itself? We want the final response to be a text file which the model can iteratively build and edit as it reasons via its chain of thought. Intuitively, this allows the model to refine its final response freely without needing to commit to a fully structured final response at the end of its COT budget.

Option 1 : Diffusion drafter drafts COT upto reasoning forks into output file, content is appended to COT

Option 2 :

Append-only reasoning history ──┐
                                ├── Model → reasoning or edit action
Mutable document memory ────────┘                │
          ↑                                      │
          └──── apply edit and save file ────────┘