Critical thinking is how you, a human, evaluate information objectively and make well-reasoned decisions rather than instantly accepting claims at face value. When you pause to think critically about something you’re being told or asked to do, you often make better decisions.
One of the biggest advancements in AI recently has been enabling models to pause and reason, too, rather than simply predict. Reasoning models not only retrieve or generate responses, but actively weigh options, simulate outcomes, and reflect before generating an output. Just like critical thinking in humans, this ability helps AI make more coherent, useful, context-aware decisions.
Reasoning models are the topic of the latest episode of the Box Enterprise AI Explainer Series podcast. Host Meena Ganesh sits down with Box Chief Technology Officer Ben Kus to talk about the profound capabilities of reasoning models, from nuanced human-like cognition to complex use cases in business operations.
Key takeaways:
- Reasoning models are a new class of AI models that have the ability to generate higher quality responses to more complex tasks
- Reasoning models represent a huge leap forward for enterprise AI, but must be used in the right context — in tasks that demand deep, analytical evaluation
- Reasoning models work in tandem with RAG and other techniques to accurately source external data
What are reasoning models?
Reasoning models, a new class of AI models like ChatGPT o3 or Claude Sonnet 3.7, are characterized by the idea that they think more. As Kus explains, unlike traditional AI models, which rely on one-shot responses to queries, reasoning models “reason about what you're prompting them to do."
This distinction makes reasoning models more suited to tasks that demand deeper analysis or thoughtful judgment — something traditional AI models can struggle with.
The benefits and trade-offs of reasoning models
While reasoning models are a significant leap forward in AI, they come with their own set of challenges and trade-offs. On the plus side, Kus highlights their capability for delivering nuanced answers: "These thinking models give you typically much higher-quality answers when they’re well thought through." Enterprises can benefit from such enhanced responses for tasks where precision and depth matter most, such as legal or financial analysis.
But the trade-off lies with time. Using reasoning models requires patience and can get in the way of rapid work — one of the primary reasons organizations tend to adopt AI in the first place. In high-paced scenarios where immediate outputs are ideal, reasoning models might not be the best solution.
Kus underscores another limitation of reasoning models that’s reminiscent of human thinking: "At some point, when you ask them to think more, sometimes it doesn't make them better." In other words, there are diminishing returns to overthinking, even in AI models, making it essential to use them strategically.
The enterprise application of reasoning models
So where do reasoning models shine? The real magic off these new models emerges when tasks are more nuanced — for instance, looking through a legal contract to analyze subtle but critical language. When tasks demand deep, analytical evaluation within enterprise environments, reasoning models offer increased depth and accuracy.
For simpler tasks, traditional AI models still excel. If you’re simply looking for a straightforward answer you know will be found somewhere within extensive documentation, a straightforward generative AI model works well. These types of situations don’t require the heavy cognitive lifting that reasoning models are designed for, and they perform much more quickly within the flow of work.
The role of RAG
One important aspect of the conversation, which Kus and Ganesh touch on in their conversation, is how reasoning models work in tandem with techniques like retrieval-augmented generation (RAG). RAG enables reasoning models to source external data, enriching their responses with broader context and accurate information. This combination opens even more doors for enterprise applications like knowledge management, risk analysis, and customer support.
As reasoning models interact with RAG and other advanced frameworks, businesses can expect significant improvements in both the breadth and quality of AI-driven insights.
Catch the full episode
Reasoning models hold tremendous transformative potential for enterprises. These AI systems represent a shift from reactive to reflective processing, enabling solutions that mirror human-like cognition for complex problems. But their use comes with critical trade-offs, particularly around time-sensitive scenarios. To make the most of reasoning models, organizations need strategic approaches that match the right tools to the right tasks. The deeper the problem, the more reasoning models can shine.
As reasoning models evolve and integrate with complementary technologies like RAG, the enterprise AI landscape is poised to enter an era of smarter and more nuanced capabilities.
Ready to dive deeper into this discussion? Don’t miss Episode 6 of the Box AI Explainer Series.Subscribe now to stay informed and get inspired about the AI-first era. Start listening today to learn practical, actionable strategies for integrating AI into your organization or industry.
