Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Microsoft paper that distills GPT-4 into a 13B Llama model by training on rich step-by-step explanation traces rather than terse SFT pairs.
- Publisher
- Microsoft Research
- Year
- 2023
- Venue
- preprint
- Authors
- 6
- Hosting
- External sourcelicense unknown
Cite
Notes
Only stored in your browser.
Introduces 2 artifacts - 1 tool, 1 model
TL;DR
Semantic Scholar
Orca is developed, a 13-billion parameter model that learns to imitate the reasoning process of LFMs, indicating that learning from step-by-step explanations, whether these are generated by humans or more advanced AI models, is a promising direction to improve model capabilities and skills.
Artifacts
2Tools
Models