Characterizing Narrative Content in Web-scale LLM Pretraining Data
17 Jun 2026
The narrative composition of web-scale LLM pretraining corpora remains largely unexplored, even though narrative is a fundamental mode of human communication. We present the first fine-grained study of narrative features in Dolma, a 3-trillion-token open pretraining corpus.
20.0/h