0

RSD: Moving Local Triangular Charts for Auditing Language-Model Hidden States

We study Relational Semantic Decomposition, abbreviated as RSD, as a moving local triangular chart audit for language-model hidden states. For repeated occurrences of one target word, RSD fits a shared three-anchor membership chart $S_t$ at layer or token-time $t$.

Preview
Year
2026
Hosting
Abstract onlyARXIV-DEFAULT

Cite

Notes

Only stored in your browser.

Attribution

Abstract & full text
arxiv.org/abs/2605.17482ARXIV-DEFAULT
TL;DR
Semantic Scholar
Attribution policy →

Abstract

We study Relational Semantic Decomposition, abbreviated as RSD, as a moving local triangular chart audit for language-model hidden states. For repeated occurrences of one target word, RSD fits a shared three-anchor membership chart S_t at layer or token-time t. The hidden-state channel uses X_t\approx S_tC_t; the invariant readout M_t=S_tS_t^\top is the induced occurrence co-membership relation, and R_t=X_t-S_tC_t records what the fitted root chart leaves outside the chart. The broader joint audit reuses the same membership chart for relation data, A_t\approx S_tB_tS_t^\top, such as an attention-derived occurrence relation. The current GPT-2 evidence is the X-channel hidden-state audit with Word-in-Context labels used as an external same-sense versus different-sense reference relation. On full WiC train, the root chart passes 16 of 53 eligible target words; this is audit coverage, not GPT-2 task accuracy. Token-time and pair-level diagnostics show the main regimes: make and break align at the target state, drive and stay improve after right context in small-count exploratory cases, and play remains a localized root-chart failure whose final same-sense pairs are not closer and have larger residual discrepancy. The resulting claim is diagnostic: RSD reports where a sense relation is visible in root co-membership and which failures become residual branch candidates or attention-channel obligations.