Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
English summary
The paper probes the causal influence of chain-of-thought reasoning steps in large reasoning models. It identifies a 'commitment boundary'—a sharp transition from transient guesses to a stable, high-confidence answer, often occurring in a single step well before the reasoning block ends. Steps after this boundary are epiphenomenal, leaving the final answer probability unchanged. Attention probes can linearly decode answer-formation stages from intermediate steps with high accuracy and generalize to unseen tasks. Exploiting this signal, early-exiting at the commitment boundary reduces CoT length by up to 55% on average with negligible performance loss.
Chinese summary
本文探究大型推理模型中思维链推理步骤的因果影响，识别出一个“承诺边界”——从短暂猜测到稳定高置信度答案的急剧转变，常发生在单个步骤内，远早于推理块结束。边界之后的步骤是附带现象，不改变最终答案概率。研究者利用注意力探针从中间步骤中线性解码出答案形成阶段，精度高且能泛化至未见过任务。通过此信号在承诺边界处提前退出推理块，可在几乎不影响性能的情况下平均减少高达55%的思维链长度。
Key points
Proposes an early exit method to estimate the causal importance of each chain-of-thought step.
提出通过提前退出来估计每个思维链步骤的因果重要性。
Identifies a 'commitment boundary'—a sharp, often single-step transition from transient guesses to a stable, high-confidence final answer.
识别出“承诺边界”——从短暂猜测到稳定高置信度最终答案的急剧转变，常发生在单步内。
Shows that reasoning steps after the commitment boundary are epiphenomenal and do not alter the final answer probability.
证明承诺边界后的推理步骤是附带现象，不改变最终答案概率。
Demonstrates that answer-formation stages can be linearly decoded from intermediate steps using attention probes, with robust generalization to unseen tasks.
展示利用注意力探针可从中间步骤高精度线性解码答案形成阶段，并能强泛化至未见任务。
Early-exiting at the commitment boundary reduces chain-of-thought length by up to 55% on average with negligible impact on model performance.
在承诺边界处提前退出可平均减少思维链长度最多55%，且对模型性能影响可忽略。