Why does machine-generated text remain detectable? We trace the answer to the decoding stage: standard strategies such as top-k and nucleus sampling restrict generation to high-probability tokens, while human writers routinely choose words that are contextually apt yet rank low under a model's next-token distribution. Truncation therefore renders a measurable share of human word choices unreachable; we call this set the truncation blind spot. Across five open models and three domains, 8--18% of human-selected tokens fall outside common truncation boundaries, with content words excluded at 2.9 times the rate of function words. As a consequence, simple classifiers using two features separate 1.8 million machine generations from human text at mean AUC-ROC near 0.97, and detectability tracks truncation intensity rather than model scale, architecture, or alignment. Probability-floor samplers substantially narrow the blind spot, and generation experiments confirm the exclusion--detectability link out of sample. Code, data, and the measurement harness are released at https://github.com/EstebanGarces/human_vs_machine.
The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices
Why does machine-generated text remain detectable? We trace the answer to the decoding stage: standard strategies such as top-$k$ and nucleus sampling restrict generation to high-probability tokens, while human writers routinely choose words that are contextually apt yet rank…
- Preview

- Year
- 2026
- Hosting
- Full text hostedCC-BY-4.0
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2603.18482CC-BY-4.0
- TL;DR
- Semantic Scholar