Every new generation of LLMs gives agents stronger reasoning, better planning, and the ability to complete more complex tasks. But that intelligence comes at a price:
- Gemini 3.5 Flash is roughly 3× more expensive than Gemini 3 Flash for the relevant token mix.
- Claude Fable 5 is 2× the price of Opus 4.8.
An agent may make many model calls for a single user request. So the cost increase doesn’t apply once per query; it compounds across planning, retrieval, tool use, verification, retries, and answer generation. The alternative is to run the entire agent on a cheaper model. But that sacrifices exactly what users value: deeper reasoning, better judgment, and higher-quality answers.
So how do we deliver frontier level intelligence at fraction of the cost ?
The recursive decomposition
Most people think of AI agents as single models that reason through entire tasks from start to finish. In practice, enterprise workflows are rarely that simple; one request might require searching across thousands of documents, interacting with multiple enterprise systems, applying business policies, and updating downstream workflows.
So, rather than attempting to solve the entire problem in a single reasoning step, the Box Agent follows a recursive parent-child execution model:
- First the parent agent analyzes the user's request and creates an execution plan by breaking the objective into a sequence of reasoning steps and executable subtasks, and whenever the plan requires retrieving information, interacting with enterprise systems, or executing an isolated operation, the parent spawns one or more child agents.
- Each child agent receives only the context required for its assigned task, executes independently within an isolated context window, and returns the result back to the parent.
- Finally, the parent evaluates those results, performs the higher-level reasoning, determines whether additional child agents are needed, and repeats the process until the objective is complete.
Control always remains with the parent. Child agents never decide what to do next; they simply execute a single assigned task and immediately hand control back to the parent. This recursive execution model naturally separates orchestration from execution, enabling the agent to scale to complex enterprise workflows while maintaining context isolation and parallel execution.
Here’s a sample query execution flow for analyzing contracts using the recursive agent:
Adaptive model selection
The recursive parent-child architecture creates many independent reasoning units during a single user request. The parent may analyze the objective, construct a plan, spawn a child to find the relevant contracts, spawn another to locate the correct risk rubric, and then launch many children in parallel to evaluate individual contracts. Each child is capable of completing an entire subtask: planning its local execution, calling tools repeatedly, evaluating the results, reflecting on whether it has sufficient evidence, and refining its approach before returning control to the parent.
But these reasoning steps don’t all require the same model.
The parent’s initial planning may have a disproportionately large impact on everything that follows. A mistake at that early stage could produce the wrong decomposition or send every child agent in the wrong direction. Final synthesis is similarly consequential; the parent must reconcile the returned results, resolve inconsistencies, and decide whether the overall task is complete. Between those stages, the agent may execute dozens, or hundreds, of narrower calls. Some involve predictable operations such as generating search queries, selecting tools, extracting structured results, or updating workflow state. Others, like applying a nuanced legal-risk rubric, may still require substantial reasoning.
The goal, therefore, is not to assign a cheap model to every child and an expensive model to every parent, but to determine the appropriate model for the current reasoning step, regardless of where that step occurs.
Our model-selection middleware implements this strategy by intercepting each LLM call and observing signals from the agent’s execution. It tracks whether the agent is planning, executing, or synthesizing; whether child agents have returned; whether retrieval produced useful content; and whether the model or retrieval system is showing signs of failure. Based on those signals, it can substitute a lightweight model for routine execution or move to a more capable model when deeper reasoning becomes valuable.
Today, this policy relies on observable runtime signals and configurable thresholds. Over time those decisions will incorporate richer information:
- The complexity of the current subtask
- The ambiguity of available context
- The importance of the step to downstream execution
- The number and quality of previous attempts
- Conflicts in retrieved evidencef
This richer trove of data turns model selection from a static product configuration into a runtime systems decision.
The evaluation setup
To evaluate adaptive model selection, we ran both the baseline recursive agent and the adaptive variant against the Box Enterprise Work Dataset, a benchmark of around 400 representative enterprise agent tasks spanning multiple workload categories. Unlike traditional QA benchmarks, these tasks exercise the complete lifecycle of an enterprise agent. Some tasks provide the source documents directly, while others require retrieval across enterprise content before reasoning can begin. The dataset includes deep multi-document analysis, multi-step workflow execution, and multi-modal understanding. Most tasks require the recursive agent to spawn child agents, retrieve context, synthesize evidence, invoke tools, and modify workflow state before the task is considered complete. Each configuration is executed end to end by our evaluation runner using identical tasks, prompts, tools, and execution environment; the only variable is the agent being executed.
To evaluate adaptive model selection strategy, we set up two experiments. For the first we compared a recursive Gemini agent to an adaptive one:
- Recursive Gemini Agent: Gemini 3.5 Flash only
- Adaptive Gemini Agent: selects between Gemini 3.5 Flash and Gemini 3.5 Flash Lite models
For the second experiment we used Claude family models:
- Recursive Opus Agent: Claude Opus 4.8 only
- Adaptive Opus Agent: selects between Opus 4.8 and Haiku 4.5
We looked at four metrics:
- task success rate
- p50 latency
- tokens consumed
- estimated cost across the evaluation dataset
Task success rate is measured across multiple criteria defined per task, not just the final response. Each task has human judgements of the expected outcome/response, what information must be cited, what tools should be used, and what path the agent should take to complete the execution. Only when all required criteria are met by the agent response for the task, it is considered successful by the evaluation grader.
The estimated cost per task is computed based on published pricing for each provider and those specific models for input, output and cached tokens utilized by the agent when the evaluation is executed. Note that other than the model selection strategy we’re using the same underlying agent harness, so caching strategies, context compression, etc. remain the same across all experiments.
Evaluation results
Across the Opus configuration, adaptive execution achieved a 78.5% task success rate, compared to 78.2% for the recursive baseline, while reducing the benchmark’s total inference cost by 25%, from $346 to $258. Token consumption was reduced from 166 million to 147 million tokens, reflecting fewer LLM invocations during the execution.
The Gemini configuration exhibited a similar trend; adaptive execution increased task success from 71.1% to 74.6%, while reducing benchmark inference cost by 21%, from $69.88 to $55. Total token usage decreased from 105 million to 71 million.
The key observation isn’t just that adaptive execution uses cheaper models, but that it allocates frontier-model reasoning only to execution stages where model intelligence materially improves the outcome. Less demanding subtasks like tool calls or information synthesis for internal stages can often be completed better using smaller models.
By analyzing execution traces across the adaptive Gemini and Opus agents, we consistently see fewer tokens consumed per task and per LLM call, which further reduces inference cost and also leads to slight improvement in task success rate for both cases. This happens because smaller models like Haiku 4.5 or Gemini 3.5 Flash Lite are often more succinct and adhere strictly to function schemas, whereas larger models like Opus 4.8 or Gemini 3.5 Flash tend to be more verbose and often hallucinate function parameters. Model verbosity actually compounds in multi-step agent processing, leading to significantly higher token consumption. Reasoning models like Opus also consume thinking tokens which aren’t necessary for tool calls and other straightforward tasks.
The bottom line: by matching model capability to the complexity of each agentic step, the agent substantially reduces overall inference cost without sacrificing end-to-end task success.
Conclusion
The industry often frames model selection as a product decision focused on choosing the best model your latency and budget allow. Our experience suggests that this framing is too coarse for modern agentic systems. Recursive agents transform a single user request into a dynamic execution graph composed of planning, retrieval, reasoning, tool use, verification, and synthesis. Each of those reasoning steps has different computational requirements, consequences, and opportunities for optimization, so treating every step as if it requires the same level of intelligence leads to unnecessary inference cost.
Adaptive model selection changes this perspective by treating intelligence not as a fixed property of the agent but as a runtime resource that can be allocated throughout execution. More capable models are reserved for the reasoning steps where they materially influence the outcome, while smaller models execute the many intermediate tasks that don’t. As frontier models continue to become more capable, and more expensive, we believe it will become essential to identify how to orchestrate intelligence across the execution steps of a complex task, thereby delivering frontier intelligence at a fraction of the cost.
