A few weeks ago, at a dinner table in London, I asked a few of my fellow operators the following question: “Are you paying for ‘personal productivity’ agents, and if so, how do you track their impact?”
The answer was yes, but impact and ROI were “very hard to tell.” It was almost impossible to know what team members were using AI agents for; were they researching a customer? Planning a wedding? I even heard, “We don't know what they’re doing, but if it's personal and I can help them get it done more quickly, then they should turn to work sooner.”
In a world where budgets still have to come from somewhere, we’re constantly weighing these sorts of trade-offs. Here’s one statistic that stood out for me in the recent State of AI in the Enterprise report: “86% of enterprise companies have deployed agents.”
What exactly does this figure mean? What are individuals doing with these agents? If I had to guess, I’d say the majority of that usage is personal productivity agents. But is that the right approach? Should we set aside $X per month for team members to use on personal productivity agents, or what I call “just-for you” agents? Or should we use those resources elsewhere and ask team members to use “team agents?” Or perhaps a hybrid of the two?
As context, at Box we have over 60 Box AI agents that have been hand-selected from Boxers’ proposals and experimentations, optimized and refined, then put into production. They have Box-specific instructions and context, and their output meets Box’s quality bar, falls into Box templates, and pulls information from content in an efficient, secure and well-permissioned way. These agents are executing repeatable tasks at scale, and any Box team member can use them.
At Box, we have over 60 Box AI agents that have been hand-selected from Boxers’ proposals and experimentations, optimized and refined, then put into production.
Of course, these 60 agents don’t do all tasks; they represent the most common activities for non-engineering teams. There’s an SOW Box agent for the professional services sales team, an RFP Box agent for the software sales team, a blog-writing Box agent for the content marketing team, and so on. Imagine 3-4 agents per sub-function. We can see usage by agent and team member, and since we know what the task was, we can estimate time saved, number of actions completed, higher quality achieved, etc., — all leading metrics that help us begin to calculate ROI.
On the subject of ROI: just-for-you agents, on the other hand, perform tasks for one individual and take a lot of work to set up. Even if we assume no personal tasks are assigned, we know that, at a minimum, everyone writes prompts differently, and may have to prompt and re-prompt before getting something that works. We also know that individuals rarely change models to optimize token consumption; why would they (and are they even given the ability to easily choose within and across model families/providers)? Over time, individual people giving individual agents individualized instructions consume a huge amount of company resources, as each person instructs their own agents to fulfill their own set of needs.
That’s where the question of individual versus team productivity comes in.
Scalable, sustainable AI transformation for most enterprises will primarily come from efficient use of agents. And full transformation will come when these role-specific agents can be wired into more complex, multi-stage, cross-functional, and multi-user workflows.
So in a time when AI spending is challenging everyone's budgets, should a responsible P&L owner encourage just-for-you agents, or insist on leapfrogging into cross-functional group workflows based on established agents that we've already vetted and know the ROI for? Are we better able to manage agent security and efficiency if we focus on a clear portfolio of agents that have been reviewed and tested?
Your company is probably having similar discussions:
- How much should we spend on AI?
- Should we spend it on “personal AI” or AI that drives team and role-based agents?
- Do we say, “Okay, everyone can spend $x a month with XYZ AI agent (the same one or two across the company) however they want?”
- Can we assume that resource optimization naturally occurs? Namely, will people use just-for-you agents for the tasks that matter most to them and therefore result in optimal allocation of resources?
Week by week I become more convinced that the right answer is a hybrid one, where team agents are the big rocks and just-for-you agents are the pebbles. I know, controversial. Basically it’s a pareto distribution, and I’m proposing that from a resourcing perspective, team agents are the 80% and “just-for-you” agents are the 20%:
Big rocks first: Make sure you’re building, and funding, role-specific agents for the clear, repeatable tasks that your teams benefit from. This includes creating agents for tasks that individuals wouldn’t otherwise spend time on due to their difficulty, but for which there are real business returns. These big-rock tasks will account for the majority of work that can efficiently leverage AI and can meaningfully scale.
Pebbles as fillers: Then, with a smaller budget, fund just-for-you agents — your company’s little rocks. They’ll fill in what standardized role-based agents won't do and provide a source of innovation. After all, those pebbles may turn into big rocks one day.
Let me know what you’re seeing — as operators, we’re all on this journey together.
