Exploring On Policy Distillation
If you are looking for information about On Policy Distillation, you have come to the right place.
- https://rllm-project.com/post.html?post=opd.md rLLM
- I recently met Sasha Rush and he started giving me an impromptu lecture on how targeted
- ... we do just distillation trying to fit the teacher performance the teacher trajectories and we do it with either
- Title:
- In this video, we break down knowledge
In-Depth Information on On Policy Distillation
00:00 - Intro 01:39 - Motivation: Why Blog-post: https://thinkingmachines.ai/blog/ Slides: https://docs.google.com/presentation/d/1iwAyhXMdLl-506HquRaoT192w4k0uBk0LTlhmiBsMno/edit?usp=sharing. This lecture starts slow, but covers key trends and training methods that came out of advancements in synthetic data. The core of ...
Paper: Fast and Effective
We hope this detailed breakdown of On Policy Distillation was helpful.
