AI agents have moved from technological concept to viable business application in record time. Asking an AI chatbot to answer a question may soon feel quaint — before you've finished writing the prompt, the bot will have already answered it.
According to Box’s State of AI in the Enterprise report, nearly nine out of ten organizations have begun integrating simple AI agents into their workflows. Just over 40% are experimenting with fully autonomous AI operations, though few enterprises are close to deploying these agents at scale.
But as agents take on more sophisticated tasks, they’re running into stumbling blocks. A recent study found that generic LLM agents, AI systems built on foundational models without specialized training or access to enterprise-specific data, successfully complete only 58% of simple tasks and just 35% of more complex ones. Another study found that agents failed to complete real-world office tasks nearly 70% of the time.
As enterprises struggle with high failure rates for AI experiments, technology leaders need to step back and ask themselves why so many pilots fail to gain altitude. The primary reason: insufficient direction and context for agents to complete tasks effectively.
Context is critical for organizations to be successful with AI. For agents to succeed, you must establish well-structured goals, point them to relevant data, and provide thorough grounding in company workflows. But the foundation of effective AI agents isn't just process — it's data.
A recent study found that agents failed to complete real-world office tasks nearly 70% of the time.
Any data science model is only as good as the data it processes, and for an AI agent to have the context it needs to be effective, it must access and reference your enterprise's proprietary data and content. Your company's unique data — from internal documents and communications to customer interactions and operational records — is what transforms a generic agent into a powerful, context-aware business tool.
Ultimately, model choice isn’t where companies will win with AI. The context layer is what will allow enterprises to differentiate and rise above the competition.
Making agents context aware
Many AI agents suffer from one of two problems: Either they don’t have enough information to produce a relevant response, or they have too much information and don’t know which data sources deserve the most attention. As a result, users waste valuable time engineering their prompts to get a useful answer, or they end up getting a response that’s incomplete or unreliable.
As Box CTO Ben Kus points out, "If an AI model isn’t giving you the information you need, it may not be because the model isn’t smart enough; it may be because the model hasn’t been taught how to find the data it needs to answer your questions.”
If an AI model isn’t giving you the information you need, it may not be because the model isn’t smart enough; it may be because the model hasn’t been taught how to find the data it needs to answer your questions.
Context engineering — building agents that draw on tightly focused pools of domain knowledge, with specific data, workflows, and tools already in place — keeps agents from going off the rails.
But context is about more than just pointing agents towards the right data sources. Agentic workflows also need to be granular and specific. Does the agent need to produce an answer quickly, or can it take more time? If it’s unable to find the answer immediately, should it look at secondary sources or come back and seek clarification? Once the agent finds the information it needs, what actions should it take? And so on.
“Context engineering is increasingly the most critical component for building effective AI Agents in the enterprise right now" says Box CEO and co-founder Aaron Levie. "We need AI Agents that can deeply understand the context of the business process that they’re tied to. This means accessing the most important data for that workflow, using the appropriate tools at the right moment, having proper objectives and instructions, and understanding the domain that they’re in.”
Context engineering is increasingly the most critical component for building effective AI Agents in the enterprise right now
As agents scale from boosting individual productivity to helping teams or entire companies, getting context engineering right will matter more than ever. There will be a huge reward for the individuals, teams, and companies that are able to give agents the best context to do their work.
Unlocking context from unstructured data
Aside from making agents more effective and business users more productive, well-executed context engineering has two other enormous benefits for the enterprise.
The first is that it unlocks the context layer that makes each organization unique. Every enterprise is sitting on enormous volumes of proprietary data — everything from documents and spreadsheets to audio and video files, presentation slides, chat transcripts, and much more. Because multi-modal large language models are designed to ingest and analyze these types of information, agents are engineered to draw upon pools of previously inaccessible data. That helps drive insights that were previously unavailable to most organizations.
Real-world applications:
- Financial services firms deploy agents that combine news reports, social media posts, and earnings call transcripts to anticipate market disruptions and build sophisticated risk models.
- Law firms use agents to analyze massive volumes of contracts, case law, internal communications, and regulatory requirements to identify legal risks and compliance gaps.
- Healthcare providers deploy agents that incorporate clinical notes, radiology reports, and research literature to diagnose diseases and suggest treatments months sooner than traditional methods.
“Getting the best context to the model is core to AI product strategy" says Levie. "If you don’t have a differentiated context you don’t have a differentiated product. This is where user history, workflows, and data are going to be insanely important.”
If you don’t have a differentiated context you don’t have a differentiated product. This is where user history, workflows, and data are going to be insanely important.
Moving from individual agents to systems of intelligence
The second major benefit emerges when agents work in concert to create systems of intelligence. Dividing labor among specialized sub-agents, each with domain expertise, enables enterprises to build robust, reliable agentic ecosystems.
For example, a team of software development agents could replicate human workflows: The product manager agent defines project requirements, the architect agent designs system structure, developer agents write the code, QA agents test it, and the final agents in the chain handle deployment and monitoring.
A financial services firm might use one type of agent to analyze market conditions, another to assess potential risk, a third to place trades, and a fourth to rejigger the firm's portfolio. These agents would continuously share data and adapt as market conditions evolve.
At first, these agents would recommend actions for human approval. Over time they may be entrusted to act more autonomously, leaving people to weigh in when recommendations are unclear or involve high levels of risk.
Building a context layer into each agent also flips the debate about whether we’ll ever achieve Artificial General Intelligence (AGI). Instead of a single enormous AI model able to answer any question or perform any task, millions of small AI agents with deep knowledge on specific domains will collaborate to provide answers and perform work.
Intelligent content management is the key element
Organizations can’t just press a button and turn on the context layer for AI agents. CIOs and CTOs will first need to answer the following questions:
- Is your data ready for agents? To get the full benefit of agents, you may need to modernize your environment and adopt a centralized platform that handles a wide range of unstructured data.
- Do you have the right data for each task? Making sure your access controls align with what you’re asking the agent to do is critical. Deciding whether to use RAG, web search, or just put all of the data into the context window will matter a lot.
- Do you have the right data governance controls in place? Agents are notoriously bad at keeping secrets. If you give an agent contextual information to help it respond to prompts, there is inevitably a way to pull that data out of the agent, intentionally or otherwise.
According to Box VP of Engineering Tamar Bercovici, "For AI to be truly valuable within a business context, it needs access to your proprietary internal information. But you need to make sure the AI is only accessing information it’s been authorized to see, and that the results of its work are only surfaced to people who are authorized to see it. That means you need to keep your data in a platform with strong security and compliance features built in."
You need to make sure the AI is only accessing information it’s been authorized to see, and that the results of its work are only surfaced to people who are authorized to see it.
IT teams must deeply understand what humans need to perform tasks and collaborate with business stakeholders to map agent actions to trusted sources and systems.
Line of business owners — from sales, legal, and HR — must ensure agents serving their customers are effective, secure, and have appropriate content access.
Enterprises will want a data platform that combines Intelligent Content Management with agents that retrieve information using APIs. Such platforms employ AI to help find relevant content and automatically enforce policies governing data access and usage, greatly simplifying the process of building context-aware agents.
The window is now
We're in a critical window where AI agents will be built for every vertical and domain.
Connecting agents to relevant data sources and tools, and enabling them to operate seamlessly within existing workflows, is the key to agentic success. This will allow existing platforms to become even more useful, and create new agent platforms to emerge in spaces that never had software before.
“The race for the next few years in AI in the enterprise is to see who best to deliver the right context for any given workflow", according to Levie. "This will determine the winners and losers in the AI race.”
The race for the next few years in AI in the enterprise is to see who best to deliver the right context for any given workflow. This will determine the winners and losers in the AI race.
The enterprises that master context engineering today will define tomorrow's competitive landscape.
