Less is More: Recursive Reasoning with Tiny Networks

Hierarchical Reasoning Model (HRM) is a novel approach to generate an answer to a given question using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku puzzle solving, Maze path-finding, and the "Abstract and Reasoning Corpus" for Artificial General Intelligence benchmark (ARC-AGI) while trained with small models (27M parameters) on small data (around 1000 examples). This impressive feat shows that HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach (easy to understand; requiring no biological argument nor fixed-point theorems) that achieves significantly higher generalization, while using a single tiny network with only 2 layers and 7M parameters. TRM obtains higher accuracy than HRM and most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters. This shows that scaling large models is not the only way toward intelligent AI. With recursive reasoning, it turns out that “less is more”. A tiny model pretrained from scratch, recursing on itself and updating its answers over time, can achieve high accuracy on difficult logic puzzle tasks at a low cost (< $500 USD for ARC-AGI). This paves the way toward small specialized models that are more environmentally friendly (due to the low cost of training) and that can directly run on mobile devices (unlike current large models, where your potentially private query must be transmitted to the cloud for the model to process it).