How does a tech giant like IBM tackle the challenge of integrating generative AI to tangibly improve operations and drive value? Matt Leitzen, CIO of Technology Platforms at IBM, shares his firsthand experience on the AI First podcast hosted by Jon Herstein, Chief Customer Officer at Box.
To ensure scalable and secure adoption of AI tools like watsonx and Box AI across a workforce of 280,000, IBM’s approach blends technology innovation with governance and ethics. From addressing employee anxiety about AI adoption to IBM’s goal of "making AI real" across its operations, this conversation in this episode provides insights for organizations navigating the complexities of AI-driven transformation.
Key takeaways:
- IBM is transforming its core technology platforms to make AI practical for every employee through tools like Ask IBM and Ask HR that seamlessly combine deterministic and non-deterministic workflows
- Strong AI guardrails are critical, and IBM establishes comprehensive security protocols and ethics reviews before implementing any AI vendor solution (including Box)
- AI advocates have to articulate the value of AI beyond mere hours saved; successful AI adoption requires measuring impact through financial metrics like unit costs
Eliminate. Simplify. Automate.
“IBM is undergoing a massive transformation right now, of course,” says Leitzen. “One of the things we realized a little less than two years ago, as part of this transformation — we need to double down on what we're doing with technology” and “make AI real for every single IBMer.”
IBM’s AI strategy revolves around eliminating inefficiencies, simplifying processes, and automating routine tasks. Leitzen describes the general AI mantra as: "Eliminate, simplify, automate." This includes driving productivity across domain-specific workflows as well as what he calls “everyday AI” — the tools that make IBMers productive in their day-to-day tasks on the job.
For instance, Ask IBM is a generative AI tool, trained on IBM’s intranet data, that employees can use in an ad hoc way. An employee might prompt Ask IBM: “I’m having a meeting with my client Rodrigo from Acme today. Draft me a summary agenda I can use to pitch our AI strategy.” The tool will then pull the most recent information from the intranet and draft an up-to-date presentation.
A year or two ago, that employee would have been up late prepping for the call and perhaps creating a deck. Now, Ask IBM can do it for them very quickly. This wouldn’t be possible with an external generative AI tool like ChatGPT, because accurate results depend on access to IBM’s specific corpus of information, which lies within its private intranet.
Deterministic vs. non-deterministic workflows
Leitzen describes another AI tool IBM has developed called Ask HR, which has both generative capabilities and workflow process automation built in. If an employee logs in and asks “How many vacation days do I have left?” the tool needs to:
- Recognize the asker
- Look up their particular vacation policy
- Determine how many days they’ve already taken this year
- Translate the answer into conversational language
While seemingly simple, this transaction requires the AI app to access more than one source of information while using natural language processing. Leitzen articulates the challenge of designing user experiences for these kinds of AI-based workflows, because they require a synthesis of deterministic (structured) and non-deterministic (flexible) systems.
This trick is to combine these capabilities in order to simplify interactions for IBMers without overwhelming them. Leitzen’s team learned valuable lessons from failed attempts. At one point, they tried to migrate a 40-field web form into a single digital assistant prompt, and that attempt to streamline did not translate to good user experience. As Leitzen recalls, “We realized that was a bad idea. It forced us to reexamine the overall process.”
Through iterative refinements and a focus on user experience, IBM ensures technology enables users rather than hindering them. IBM also fosters adoption through initiatives like the watsonx Challenge, a hands-on program exposing employees to AI tools.
Strong AI guardrails are critical
While using AI to accelerate workflows across their hybrid cloud environments, IBM has proactively established guardrails to ensure safe and scalable adoption. When evaluating where to allocate their AI efforts, Leitzen looks directly at the end-to-end workflow of all use cases under consideration to measure the efficacy of security and governance efforts against those workflows.
His team also vets AI partners carefully. “Every vendor is adding AI capabilities to their products,” he says. “The challenge becomes, which ones are we comfortable with?” It starts with the workflows themselves — asking what will make the job easier and better for employees? — but in tandem, IBM conducts rigorous AI ethics reviews to evaluate vendor trust and data-handling practices.
During any vendor evaluation that includes an AI component, the team assesses whether and how the vendor will handle IBM’s data from a cybersecurity risk perspective. Box stands out from this perspective because, as Herstein explains, “Every tool we embed in Box AI, every platform we integrate, is rigorously vetted to ensure it adheres to the privacy, security, and ethical guidelines that our customers expect. Governance isn’t just a checkpoint — it’s foundational.”
Understanding workforce fears and opportunities
While the sense of innovation AI brings is palpable at IBM and in every industry, a general air of uncertainty still surrounds AI’s impact on the workforce. Leitzen candidly addresses this anxiety, noting the mixed mindsets among employees: “There are some people who are gung ho, and some people that, quite frankly, have a lot of anxiety about what AI is going to mean to them personally — their roles, maybe their careers.”
But as he reminds us, “None of us really knows.” While leaders can speculate, AI's evolution continues to surprise industries. For instance, a year ago, prompt engineering was a new skill in the workforce. Today, seemingly everyone is a prompt engineer, but he says: “Just like in other massive revolutions, there are roles out there we haven’t even begun to discover that we need.”
Leitzen encourages organizations to approach the workforce transitions AI will inevitably bring with empathy, addressing concerns while keeping an eye on the valuable emerging opportunities.
Articulating the value of AI beyond mere hours saved
To capitalize on the right emerging opportunities, Herstein says, “It’s not just about having AI for the sake of saying you have AI. It’s about embedding it directly into your workflows where it can drive measurable value and make jobs simpler, faster, and more impactful.”
Leitzen stresses the need for technology leaders to articulate AI’s value in terms that resonate with CFOs and other business leaders. He explains that simply saying “it saves time” may underwhelm decision-makers: “You try to have a conversation with a CFO about hours saved, it’s like, what did you really do for me today?”
Instead, organizations should highlight financial metrics like unit costs, process velocity, or aggregate savings. He says,“If we say we can produce a supplier brief in about five minutes, that translates into something a CFO can recognize in terms of what this is doing to our aggregate cost structure. Because really, when we’re articulating value, it’s got to come to the bottom line.”
One of the ways IBM has found to communicate bottom-line value is by comparing per-unit cost to the output of the workflow. For instance, what does it cost an employee or a team to produce an invoice, versus the cost of spinning up a virtual machine that can use AI to do it?
Being honest about AI's impact and ROI is crucial. Organizations shouldn’t overpromise financial benefits, but should clearly pinpoint operational improvements enabled by AI.
Strategic partnership is key to be an AI leader
Even huge technology companies like IBM aren’t necessarily in the business of building their own Intelligent Content Management platform. So partnership with Box is integral to IBM’s AI efforts in this arena. Box AI leverages watsonx to handle unstructured data securely while enabling seamless workflows.
“There are certain areas within an organization where you want to leverage a core capability like Box rather than building your own,” Leitzen explains. “Box embedding the watsonx platform underneath gives me a sense of security, especially when we start using the trusted LLMs to perform the actions.”
From the Box perspective, Herstein says, “Box AI embedded with watsonx gives our customers a trusted solution as we move into areas like generative AI where security, governance, and trust are non-negotiable. This partnership allows businesses to unlock the power of AI while staying confident in the reliability of the solutions they use.”
Such partnerships allow IBM to focus on higher-value activities, empowering employees to concentrate on their core responsibilities. Herstein confirms the Box POV on this: "Our role is to provide the technologies and tools and platforms that just allow customers to go do that. Sellers need to sell, builders need to build."
The key to long-term enterprise AI success
Even as AI reshapes industries, Herstein and Leitzen underscore the importance of striking a balance between technology and humanity. By focusing on user experience, ethical governance, and measurable impact, leaders can harness AI’s potential for long-term success in the enterprise landscape.
Ready to dive deeper into this discussion? Don’t miss Executing AI-driven changes at IBM on the AI First podcast.Subscribe now to stay informed and get inspired about the AI-first era. Start listening today to learn practical, actionable strategies for integrating AI into your organization or industry.
