Directional Contextual Representations for Dependency Relations: Why Cross-Direction Pairing Fails

Splitting a bidirectional LSTM's contextual representation into a forward-only F_i (strictly a function of tokens 1..i) and a backward-only B_i (strictly a function of tokens i..n) beats either alone and beats a fused self-attention representation for dependency relation-type classification. But a specific, natural extension of this idea -- pairing a token's forward state against a candidate's backward state (``cross-direction'' pairing, F_i vs.\ B_j) -- consistently underperforms same-direction pairing, and the penalty grows, not shrinks, with token distance, both paired-bootstrap significant. We diagnose why using a frozen-trunk methodology: architectural information leakage between directions is impossible by construction (a single-layer BiLSTM, verified by code inspection); 93% of the same-vs-cross gap survives freezing the trunk and training only fresh heads, ruling out training-co-adaptation as the primary cause; linear regression shows partial representational redundancy between F_i and B_i (R^2{=}0.324 vs.\ 0.028 for a shuffled control) and a linear probe shows partial anticipatory encoding of upcoming tokens in F_i (36.5% vs.\ 17.2% majority baseline) -- real effects, but neither alone, nor combined, cleanly explains the full gap. Extended frozen-trunk diagnostics (a positional probe and a distance-decay probe) show directional information is genuinely stored but not exactly positioned, and propagates only a few tokens before decaying to baseline -- consistent with, and mechanistically underneath, the distance-growth finding.