Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection
English summary
The paper argues that entropy-based token selection in reinforcement learning for visual reasoning is insufficient because it misses critical contextual visual cues. The authors propose vision-anchored token selection, which forces the agent to prioritize task-relevant visual features during decision-making. Experimental results demonstrate that this method yields more robust and interpretable performance on visual reasoning tasks compared to entropy-driven baselines. The work underscores the need for more sophisticated attention mechanisms to improve RL agents' understanding of visual environments.
Chinese summary
论文指出，强化学习中的视觉推理任务仅依靠熵进行令牌选择是不够的，会遗漏关键的上下文视觉线索。作者提出视觉锚定令牌选择方法，迫使智能体在决策时优先关注与任务相关的视觉特征。实验表明，相比基于熵的基线，该方法在视觉推理任务上表现更稳健、更可解释。该工作强调需要更复杂的注意力机制，以提升强化学习智能体对视觉环境的理解。
Key points
Traditional RL for visual reasoning often uses entropy for token selection, which overlooks critical contextual visual cues.
传统视觉推理强化学习常使用熵进行令牌选择，忽略了关键的上下文视觉线索。
The proposed vision-anchored token selection forces the agent to focus on relevant visual features, enhancing reasoning ability.
提出的视觉锚定令牌选择迫使智能体关注相关视觉特征，增强了推理能力。
The method leads to more robust and interpretable outcomes in visual reasoning tasks compared to entropy-based baselines.
与基于熵的基线相比，该方法在视觉推理任务中产生了更鲁棒且可解释的结果。