Prior work establishes that instruction-tuned LLMs exhibit left-of-center political bias, but measures it exclusively through abstract questionnaires. We show this does not generalize to concrete policy decisions. We introduce a dual-instrument methodology grounded in Swiss direct democracy: the Smartvote questionnaire (75 policy questions), administered to 66 LLMs and compared to 184 elected members of the Swiss National Council, and 48 real federal referenda (Volksabstimmungen) put to 9 flagship LLMs in four national languages and three information conditions, compared to actual outcomes and party recommendations (Parolen). The instruments disagree. (1) The left-to-right agreement gradient that dominates Smartvote (replicating prior work; Cohen's d = 3.64) shifts to center-peaked on referenda, where models align most with centrist Die Mitte and FDP rather than leftist SP and Gruene (Wilcoxon p = 0.008). (2) For some models the language of a question changes the answer more than its content does: cross-linguistic consistency ranges from 50% (Mistral) to 98% (GPT-5.4). (3) Two models vote Nein on 83-94% of referenda at similar rates on progressive and conservative proposals (binomial p < 0.0001), a status-quo aversion rather than a left-right bias. What prior work measured as "leftward bias" may not extend beyond abstract instruments: confronted with real decisions, LLMs behave less like coalition partners of the left than like cautious civil servants, centrist and inconsistent across languages.
Progressive in Principle, Centrist in Practice: LLM Political Bias Is Instrument-Dependent
Prior work establishes that instruction-tuned LLMs exhibit left-of-center political bias, but measures it exclusively through abstract questionnaires. We show this does not generalize to concrete policy decisions.
- Preview

- Year
- 2026
- Hosting
- Full text hostedCC-BY-4.0
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2606.00048CC-BY-4.0
- TL;DR
- Semantic Scholar