Stanford Center for Research on Foundation Models (CRFM)

Stanford CRFM's wide-coverage evaluation framework - dozens of scenarios scored on accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency.

A safety benchmark evaluating language models against risk categories derived from government regulations and company policies.

Stanford's hand-curated 1,000-problem reasoning dataset that, paired with budget forcing at inference, produced o1-competitive results for ~$50 of compute.

Stanford CRFM's open-source Python framework for holistic, reproducible, transparent evaluation of foundation models across many benchmarks and metric axes.