Suchin Gururangan

LESS: Selecting Influential Data for Targeted Instruction Tuning

arXiv 2024

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

arXiv 2024

Language models scale reliably with over-training and on downstream tasks

arXiv 2024

AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters

arXiv 2024

Scaling Expert Language Models with Unsupervised Domain Discovery

arXiv 2023

SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore

arXiv 2023

Time is Encoded in the Weights of Finetuned Language Models

arXiv 2023

Editing Models with Task Arithmetic

arXiv 2022

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models

arXiv 2022

M2D2: A Massively Multi-domain Language Modeling Dataset

arXiv 2022

DEMix Layers: Disentangling Domains for Modular Language Modeling

NAACL 2022 7

Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

don-t-stop-pretraining-adapt-language-models-1

Noah A. Smith

Luke Zettlemoyer

professor

Margaret Li

Mike Lewis

Gabriel Ilharco

Hannaneh Hajishirzi

professor

Luca Soldaini

Ludwig Schmidt

professor

Mitchell Wortsman

Tim Althoff