Simple Recursive Model: Simplified, Single-State Reasoning with Skip Connections

Hierarchical Reasoning Models (HRM) and their variants (TRM) have shown impressive performance on a variety of reasoning tasks. These models maintain two separate states $z_L$ and $z_H$ that are designed to capture low-level and high-level representations for reasoning. However, the necessity of this dual-state strcuture remains unclear. We conjecture that the benefit of $z_H$ stems not necessarily from representing any distinct level of information, but rather from providing access to information from earlier timesteps, as $z_H$ is updated less frequently than $z_L$. To test this, we propose Simple Recursive Model (SRM): a model that maintains only a single state $z$ and adopts skip connections to earlier timesteps in replacement of the second-level state $z_H$. This approach eliminates the complicated nested loops in HRM/TRM that alternate between $z_L$ and $z_H$, leading to a more straightforward and parsimonious architecture. Through experiments on the Sudoku task, we demonstrate that the single-state SRM can achieve comparable performance to the TRM baseline. Our results suggest that the key benefit of HRM/TRM is not necessarily hierarchical state separation, but may come from the ability to access information across different timescales. Such ability can be equivalently achieved through other mechanisms such as skip connections, opening up new avenues for designing simpler and more efficient reasoning models. Code available at: https://github.com/liaoq/SimpleRecursiveModel