Towards Structural Understanding of LLM Overthinking
Thinking models waste 5 to 20 times more compute on simple queries without accuracy gains, driven mainly by over-verification and over-exploration in their reasoning.
It offers a structural, quotable way to detect and manage wasted reasoning compute instead of relying on raw output length.
Xinliang Frederick Zhang · Anhad Mohananey · Alexandra Chronopoulou · Pinelopi Papalampidi · Somit Gupta · Tsendsuren Munkhdalai · Lu Wang · Shyam Upadhyay — meet the researchers →
Two readings, equal authority
How to choose: The paper’s words is verbatim — use it when you need to quote, or to judge how they write. Plain language is a paraphrase written for comprehension — use it when you want the idea fast. Neither is a summary of the other; they are two doors into the same room.
“Models employing long chain-of-thought (CoT) reasoning have shown superior performance on complex reasoning tasks. Yet, this capability introduces a critical and often overlooked inefficiency—overthinking—models often engage in unnecessarily extensive reasoning even for simple queries, incurring significant computations without accuracy improvements. While prior work has explored solutions to mitigate overthinking, a fundamental gap remains in our understanding of its underlying causes. Most existing analyses are limited to superficial, profiling-based observations, failing to delve into LLMs’ inner workings. This study introduces a systematic, finegrained analyzer of LLMs’ thought process to bridge the gap, TRACE. We first benchmark the overthinking issue, confirming that long-thinking models are five to twenty times slower on simple tasks with no substantial gains. We then use TRACE to first decompose the thought process into minimally complete sub-thoughts. Next, by inferring discourse relationships among sub-thoughts, we construct granular thought progression graphs and subsequently identify common thinking patterns for topically similar queries. Our analysis reveals two major patterns for open-weight thinking models—Explorer and Late Landing. This finding provides evidence that over-verification and over-exploration are the primary drivers of overthinking in LLMs. Grounded in thought structures, we propose a utility-based definition of overthinking, which moves beyond length-based metrics. This revised definition offers a more insightful understanding of LLMs’ thought progression, as well as practical guidelines for principled overthinking management.”
Models that show their reasoning step by step often spend far more time thinking through easy questions than needed, without getting better answers. The authors built a tool called TRACE that breaks that reasoning into pieces and finds two common wasteful thinking patterns, then propose a way to define 'overthinking' based on exactly when extra reasoning stops paying off.
What this paper defines
Every definition below is the paper’s own sentence, with its locator. The plain gloss is a reading aid and is marked as one.
Overthinking (length-based definition)
“Overthinking is the generation of additional thought tokens that do not contribute to a performance gain.”Section 3, Defining Overthinking
In plain terms: A model is overthinking if it produces extra reasoning tokens that don't make its final answer any better.
Overthinking (structure-based / utility redefinition)
“Overthinking is the continuation of thought beyond the point where the marginal return (∆Performance / ∆Thought) drops below a predefined threshold ϵ.”Section 5, Novel Definition of Overthinking
In plain terms: Overthinking starts at the exact point where one more step of reasoning stops producing enough improvement to be worth it, not just after some fixed length.
Sub-thought
“We define a sub-thought according to three strict criteria: • Self-contained: It must be independently comprehensible without heavy reliance on other sub-thoughts. • Complete: It must represent a full logical step, typically including an intent, an action or calculation, and a conclusion. • Answer-bearing: It must explicitly state a proposed answer to the query.”Section F.1, Sub-thought Criteria
In plain terms: A sub-thought is the smallest complete, self-contained chunk of reasoning that states a candidate answer.
Explorer (thought progression pattern)
“Explorer. This pattern (Figure 4) is characterized by the ground-truth answer being spread out across almost all nodes (i.e., distinct answers) in the thought process.”Section 5.1, Thought Progression Patterns
In plain terms: In this reasoning style, confidence in the correct answer is spread across many candidate answers instead of converging on one.
Late Landing (thought progression pattern)
“Late Landing. This pattern follows a “convergent” reasoning trajectory. As the thought process unfolds, the model gets closer to the correct answer, and the correctness probability increasingly concentrates towards the final proposal.”Section 5.1, Thought Progression Patterns
In plain terms: In this reasoning style, the model's confidence steadily converges onto a single final answer rather than staying spread out.
Token waste
“introduce a summary metric, token waste, defined as the thinking tokens count multiplied by the performance ratio.”Table 2 caption
In plain terms: A rough single-number score that multiplies how many thinking tokens were used by how much accuracy improved, to summarize wasted computation.
Sidetrack (discourse label)
“Sidetrack is a unique label for rambling— a digressive, seemingly unrelated tangent that often adds no value to the thinking.”Section 4, Stage 2: Thought Decomposition & Label Inference
In plain terms: A label for a digressive tangent in the model's reasoning that doesn't help answer the question.
What they actually did
Each step is a synthesis. Open any step to see the paper’s own sentence it was derived from, with its locator — so nothing here floats free of the source.
- Benchmark 14 open-weight thinking models (Qwen3 and DeepSeek-R1-Distill families) head-to-head in thinking vs. non-thinking mode across six datasets spanning knowledge recall and reasoning.
Trace this step to the paper
“we benchmark a suite of 14 models trained via reinforcement learning (RL) or supervised finetuning via distillation.”Section 3.1, Experimental Setup
- Use deterministic greedy decoding (temperature 0, top-p 0) with capped generation lengths so results are reproducible.
Trace this step to the paper
“we employ a greedy decoding strategy with temperature set to 0 and top-p set to 0.”Section 3.1, Experimental Setup
- TRACE Stage 1: sample responses from four large thinking models on the benchmark queries.
Trace this step to the paper
“For stage 1, we follow the same setting as in Section 3.1 to sample responses from 4 large thinking models: Qwen3-30B-A3B, Qwen3-32B, R1-Distill-Llama-70B and Qwen3-235B-A22B.”Section 4, Stage 1: Response Sampling
- TRACE Stage 2: use gemini-2.5-pro to decompose each thought process into sequential sub-thoughts and infer the discourse relationship between them.
Trace this step to the paper
“Once responses are collected, we utilize gemini-2.5-pro (Comanici et al., 2025) to systematically decompose each thought process into sequential sub-thoughts and infer the functional relationship between them.”Section 4, Stage 2: Thought Decomposition & Label Inference
- Validate the automatic sub-thought labels against human judgment on a sample of 200 sub-thoughts.
Trace this step to the paper
“we also perform human inspection on 200 randomly sampled sub-thoughts, and we find that the automatic labels reasonable in 93% of the time.”Section 4, Stage 2: Thought Decomposition & Label Inference
- TRACE Stage 3: build a thought progression graph per query, with nodes as distinct proposed answers and edges as the inferred discourse relation between them.
Trace this step to the paper
“we construct a thought progression graph (e.g., Figure 3) for each individual thinking trace. In this graph, each node corresponds to a distinct answer proposed by the LLM during its thought process.”Section 4, Stage 3: Progression Graph Construction
- TRACE Stage 4a: group individual progression graphs by query type, difficulty, and number of distinct answers, then aggregate them into a combined graph.
Trace this step to the paper
“Graphs are first grouped by shared characteristics, i.e., query type, query difficulty, and the number of distinct answers proposed in the thought process.”Section 4, Stage 4: Thought Pattern Induction
- TRACE Stage 4b: threshold the aggregated graph to prune infrequent nodes and edges, isolating the dominant reasoning paths as the induced pattern.
Trace this step to the paper
“We apply a heuristic threshold (e.g., 0.3) to the aggregated graph to prune infrequent nodes and edges. This filtering step removes noise and highlights the dominant reasoning paths.”Section 4, Stage 4: Thought Pattern Induction
- Trace how model performance evolves sub-thought by sub-thought (utility tracing) to locate the point of diminishing returns.
Trace this step to the paper
“To quantify overthinking more structurally, we look into utility tracing by investigating the evolving performance as a function of the number of sub-thoughts.”Section 5.2, Utility Tracing
- Derive two real-time, ground-truth-free heuristics (self-looping and backtrack) that approximate the convergence point for practical overthinking management.
Trace this step to the paper
“Self-looping (SL): terminate once the model performs k consecutive self-verifications after proposing an answer (i.e., visit a node).”Section 5.3, Towards Overthinking Management
Exactly what was run, and how
| Model | Developer | Temp | Effort / reasoning | Deployment | Other settings |
|---|---|---|---|---|---|
| Qwen3 model family (0.6B to 235B-A22B) | — | 0 | not reported | local weights | top-p set to 0; max generation length 8k tokens for thinking mode (16k for temporal reasoning tasks), 2k for non-thinking mode; both thinking and non-thinking modes generated for controlled comparison; served via vLLM |
| DeepSeek-R1-Distill family (Qwen2.5 and Llama-3 backbones, 1.5B to 70B) | — | 0 | not reported | local weights | Same greedy decoding and generation-length caps as the Qwen3 family; served via vLLM |
| gemini-2.5-pro | — | not reported | not reported | unstated | — |
Source for Qwen3 model family (0.6B to 235B-A22B) settings
“we employ a greedy decoding strategy with temperature set to 0 and top-p set to 0.”Section 3.1, Experimental Setup
Source for DeepSeek-R1-Distill family (Qwen2.5 and Llama-3 backbones, 1.5B to 70B) settings
“We use vLLM (Kwon et al., 2023) to deploy aforementioned models.”Appendix A, Models Used in This Work
Source for gemini-2.5-pro settings
“Once responses are collected, we utilize gemini-2.5-pro (Comanici et al., 2025) to systematically decompose each thought process into sequential sub-thoughts and infer the functional relationship between them.”Section 4, Stage 2: Thought Decomposition & Label Inference
What they reported — and what they left out
The paper names exact model families, parameter sizes, and decoding settings (greedy, temperature 0, top-p 0, served via vLLM) for the benchmarked models, but never states which company trained Qwen3 or DeepSeek-R1-Distill, and gives no temperature, effort level, or deployment detail for gemini-2.5-pro, the model used to do the sub-thought labeling.
The numbers they report
Thinking models spend far longer reasoning on simple queries than non-thinking models, without meaningfully better answers, across a large model and domain sweep.
5 to 20x longer inference time; measured across 14 thinking LLMs and 6 data domains
See it in the paper
“Our results show that for simple queries like “7+2” or “date of 1000 days after today”, thinking models naively spend five to twenty times longer at inference than non-thinking counterparts, with little or even no improvement in performance.”Section 1, Introduction
For simple reasoning tasks, thinking only helps smaller models; the benefit nearly disappears once a model is big enough.
Thinking's performance gain is most significant below roughly 4-8 billion parameters (Figure 1)
See it in the paper
“For simple reasoning tasks, the performance gains from enabling a model’s thinking capabilities are most significant for models with fewer than 4–8 billion parameters (Figure 1).”Section 3.1, Preliminary Results
On the hardest math benchmark tested, thinking adds real accuracy but at a steep token cost, most of which is wasted.
GSM8k requires over 10x more thought tokens than non-thinking mode; 80% of that extra compute yields no measurable gain
See it in the paper
“solving GSM8k requires over 10× more thought tokens, and still 80% of that extra compute produces no measurable gain.”Section 3.2, Vertical Analysis
For easy temporal-reasoning levels, non-thinking models are already near-perfect, so added reasoning barely helps.
Temporal-L1 95.87/99.41 and Temporal-L2 95.52/97.61 (non-thinking/thinking accuracy, Table 3)
See it in the paper
“at L1–L2, where queries involve manageable duration representations, non-thinking models already achieve near-perfect accuracy, so extra reasoning adds little help.”Section 3.2, Vertical Analysis
Beyond a certain temporal-reasoning difficulty, extended thinking cannot bridge the gap and performance collapses despite heavy computation.
Thinking accuracy on hard temporal reasoning tops out around 50% even with a large reasoning workload
See it in the paper
“thinking performance collapses despite huge reasoning workload. While enabling thinking does help, boosting accuracy up to roughly 50%,”Section 3.2, Vertical Analysis
TRACE's automatically inferred discourse labels largely agree with human judgment.
93% agreement rate on 200 human-inspected sub-thoughts
See it in the paper
“we also perform human inspection on 200 randomly sampled sub-thoughts, and we find that the automatic labels reasonable in 93% of the time.”Section 4, Stage 2: Thought Decomposition & Label Inference
At the empirically identified convergence point (the eighth sub-thought), one additional thought either hurts or barely helps accuracy for both tested models.
Qwen3-235B-A22B: 63.25% to 62.05% with one more thought; Qwen3-32B: 84.76% to 85.06%
See it in the paper
“For both models, the convergence point is the completion of the eighth sub-thought, classifying any additional thinking beyond this point as overthinking. Concretely, Figure 6 shows that the introduction of an additional thought reduces performance (63.25 → 62.05) or provides negligible benefit (84.76 → 85.06).”Section 5.3, Towards Overthinking Management
For the Explorer-type model, adding a backtrack-based stopping rule keeps accuracy at the standard-thinking level while cutting output length by nearly 60%.
Qwen3-235B-A22B: length reduced to 1,100 words (from 2,722 words for standard thinking), nearly 60% efficiency savings
See it in the paper
“For the Explorer-type Qwen3-235B-A22B, adding backtracking preserves accuracy while reducing length to 1,100 words (nearly 60% efficiency savings).”Section 5.3, Towards Overthinking Management
For the Late-Landing-type model, a slightly more permissive self-looping threshold raises accuracy close to the thinking-mode baseline while cutting inference cost substantially.
Qwen3-32B: accuracy 80.18 (3 points below thinking mode) with a 40% cut in inference cost (4k to 2,463 words)
See it in the paper
“For the LateLanding-type Qwen3-32B, setting k = 3 slightly encourages verification, raising accuracy to 80.18 (3 points below the thinking mode) while cutting the inference cost by 40% (4k → 2,463).”Section 5.3, Towards Overthinking Management
A simple self-looping-only heuristic roughly halves output length for both tested models while still improving accuracy over standard thinking.
Qwen3-235B-A22B and Qwen3-32B reach 62.23 and 68.90 accuracy respectively; average output length halved (2,722 to 1,315 and 4,000 to 1,874 words)
See it in the paper
“62.23 and 68.90 accuracies for Qwen3-235B-A22B and Qwen3-32B, respectively, while average output length is halved (from 2,722 → 1,315 and 4,000 → 1,874).”Section 5.3, Towards Overthinking Management
Running the largest evaluated model requires substantial dedicated hardware.
Qwen3-235B-A22B inference requires 8 NVIDIA H100 GPUs; per-model evaluation time ranges from several minutes to 1 day
See it in the paper
“running inference with the Qwen3-235B-A22B model requires 8 NVIDIA H100s. The evaluation time for each model ranges from several minutes to 1 day, depending on the configurations (e.g., dataset, model size, thinking mode).”Limitation, GPU resources
What they assert, beside what they showed
Left is the claim in the paper’s own words. Right is the data offered for it. Where the two do not fully meet, a gold band names the distance.
The paper asserts its automatically inferred sub-thought labels are reliable enough to build the rest of the analysis on.
“we find that the automatic labels reasonable in 93% of the time.”
“we also perform human inspection on 200 randomly sampled sub-thoughts,”
Section 4, Stage 2: Thought Decomposition & Label InferenceThe paper claims it is the first to comprehensively benchmark LLM overthinking on simple queries.
“we are the first to comprehensively benchmark LLM overthinking tendencies on simple queries.”
“we benchmark a suite of 14 models trained via reinforcement learning (RL) or supervised finetuning via distillation.”
Section 3.1, Experimental SetupA figure caption states a single, crisp size threshold above which thinking becomes ineffective.
“We find that additional thinking becomes ineffective once the model scale is above the threshold of 4B.”
“For simple reasoning tasks, the performance gains from enabling a model’s thinking capabilities are most significant for models with fewer than 4–8 billion parameters (Figure 1).”
Section 3.1, Preliminary ResultsThe paper claims its structure-based redefinition enables practical, real-time overthinking management without ground truths.
“this case study showcases that our structure-based redefinition not only pinpoints overthinking in post-hoc analysis, but also enables practical heuristics for managing overthinking at realtime inference without access to ground truths.”
“For the Explorer-type Qwen3-235B-A22B, adding backtracking preserves accuracy while reducing length to 1,100 words (nearly 60% efficiency savings).”
Section 5.3, Towards Overthinking ManagementThe abstract claims the new definition offers practical guidelines for managing overthinking.
“This revised definition offers a more insightful understanding of LLMs’ thought progression, as well as practical guidelines for principled overthinking management.”
“we introduce two heuristics derived from the behavioral patterns observed in Section 5.1. They serve as proxies for estimating the convergence point.”
Section 5.3, Towards Overthinking ManagementThe paper frames its overthinking findings as holding 'at scale', across many models and domains.
“we uncover the finding at scale, across 14 thinking LLMs (Yang et al., 2025; DeepSeek-AI et al., 2025) and 6 data domains (i.a., Miao et al., 2020; Cobbe et al., 2021; Tan et al., 2023; Wei et al., 2024).”
“Table 1: Datasets used for benchmarking in horizontal analysis. We include six datasets spanning two major task domains—knowledge recall and reasoning.”
Table 1How they frame it, and what they want next
Their framing
The authors frame overthinking not as an unavoidable side effect of long reasoning but as a structurally diagnosable and manageable failure mode, arguing that a length-based view of the problem was incomplete because it never looked inside the thought process itself. They present TRACE as both an analysis tool and a stepping stone to practical, real-time fixes, positioning the two discovered patterns (Explorer, Late Landing) as the mechanism that explains why length-based interventions only partially work.
Register: The prose is confident and declarative about the main findings and definitions (bare statements like 'Overthinking is the continuation of thought...' and 'we are the first...'), while nearly all epistemic caveats are pushed into footnotes and a compact, separately labeled Limitation section rather than woven into the main argument.
Where they hedge
“Though “simple” is cognitively hard to define, we treat queries solvable by bright middle school students as simple.”Section 2, footnote 3
“4 samples were excluded due to gemini-2.5-pro parsing errors, accounting for minor discrepancies with Table 3.”Section 5.3, footnote 14
“We ignore the initial descent for one sub-thought case.”Section 5.3, footnote 13
What they say it means
- The right amount of reasoning depends on a model's own representational capacity, not just how hard a task looks from the outside.
the paper’s words
“In summary, thinking is valuable only in a narrow middle ground: trivial tasks waste it, and tasks beyond the model’s representational capacity nullify it.”Section 3.2, Vertical Analysis
- Overthinking can potentially be caught and stopped during generation itself, not only measured after the fact.
the paper’s words
“this case study showcases that our structure-based redefinition not only pinpoints overthinking in post-hoc analysis, but also enables practical heuristics for managing overthinking at realtime inference without access to ground truths.”Section 5.3, Towards Overthinking Management
- Length-based overthinking metrics are a special case of the paper's structural definition, suggesting future mitigation work could target the marginal-return threshold directly instead of penalizing length alone.
the paper’s words
“In fact, the prior definition can be viewed as a special case of our novel redefinition, where ∆ Thought is the difference in total length between thinking and non-thinking outputs.”Section 5.3, Towards Overthinking Management
What they call for next
- Extend TRACE's structural analysis to more complex, multi-step reasoning tasks (e.g., competitive coding) to test whether the same thought patterns generalize.
the paper’s words
“future work will expand TRACE to more complex settings where creative or multi-step reasoning can provide tangible benefits. This broader coverage will help assess whether the observed thinking dynamics generalize to more challenging tasks (e.g., competitive coding) without conflating productive reasoning with overthinking.”Limitation, Evaluation Domains
Limitations they state
“It is thus more time-consuming, especially the 235-billion model, than evaluating smaller previous-generation models like BART (Lewis et al., 2020), which in turn results in a significantly higher carbon footprint.”Limitation, GPU resources
“these domains do not comprehensively represent the full spectrum of NLP tasks.”Limitation, Evaluation Domains
“existing studies have typically restricted their focus to the math domain only”Limitation, Evaluation Domains
“our current evaluation analysis primarily involve relatively simple, well-posed prompts where extended reasoning may not yield clear advantages,”Limitation, Evaluation Domains
Moves worth stealing
Uses a provocative, plain-language rhetorical question as the paper's actual title, with the formal academic description demoted to a subtitle.
“Do LLMs Really Need 10+ Thoughts for “Find the Time 1000 Days Later”?”
Grounds an abstract inefficiency argument in tiny, concrete toy examples before presenting any statistics.
“for simple queries like “7+2” or “date of 1000 days after today”,”
Names its two discovered failure patterns with vivid, memorable labels rather than generic technical terms, making the taxonomy easy to cite.
“Our analysis reveals two major patterns for open-weight thinking models—Explorer and Late Landing.”
Sets its own core definitions apart from the surrounding prose as visually distinct, boxed statements, doing this twice as the definition is refined.
“Overthinking is the continuation of thought beyond the point where the marginal return (∆Performance / ∆Thought) drops below a predefined threshold ϵ.”
Quantifies the environmental cost of its own compute budget in a dedicated Limitation section rather than omitting it.
“we evaluate models on NVIDIA H100 (80GB VRAM) or A100 (80GB VRAM), both with significant CPU and memory resources.”
Where else this leads
Same territory
- Would This Change Your Answer? Evaluating Explanations of LLM Behavior in the Wild with Counterfactual Experiments Anthropic
interpretability - Verbalizable Representations Form a Global Workspace in Language Models Anthropic
interpretability - Introducing LifeSciBench OpenAI
llm-evaluation - Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations Anthropic
interpretability
Published alongside it
The nearest publications in time, across all three labs.
- Introducing GeneBench-Pro OpenAI
2026-06-30 - The Case for Globally Beneficial Technology Google DeepMind
2026-07-06 - Separating signal from noise in coding evaluations OpenAI
2026-07-08 - Bridging the Scale Gap: Augmenting Human Red-Teaming to Uncover Latent Risks in T2I Models Google DeepMind
2026-06-26
What this page was built from
Extracted from the manifest's full-grade plain-text file (main body, references, and Appendices A-H); figures and tables are represented here via their captions and surrounding prose since only plain text was provided.