Update: Gemini 2.5 Pro (I/O Edition) and Box AI

Google recently released early access to Gemini 2.5 Pro Preview (I/O edition), an updated version of 2.5 Pro that has significantly improved capabilities for coding, especially building compelling interactive web apps. Box evaluated the previous version of Gemini 2.5 Pro, so we’re updating this evaluation to reflect the latest release.

Our latest Box AI Enterprise Eval specifically tested Gemini’s reasoning capabilities and how they can be applied to data extraction tasks which is a major sticking point for organizations today. The core finding is clear: this latest iteration of Gemini 2.5 Pro delivers significant performance gains, particularly on extensive datasets and sophisticated Q&A, demonstrating its increasing power for demanding, real-world business applications within the Box Intelligent Content Management platform.

The evolution of Gemini 2.5 Pro

Looking at the I/O version of Gemini 2.5 Pro and comparing it to the March 25 release of 2.5 Pro, the I/O version of Gemini 2.5 Pro demonstrates noteable advancements over its immediate predecessor. The I/O version of Gemini 2.5 Pro achieved an F1 score of 0.86 and 86.0% correctness on the full CUAD dataset. The initial release of Gemini 2.5 Pro achieved an F1 score of 0.82 and 80.00% correctness on the full CUAD dataset. In multi-doc Q&A, the latest version of Gemini 2.5 Pro rose to 84.72% accuracy up from 81.7% on the initial version. While the hard CUAD extraction subset and single-doc Q&A evals were at relative parity with the original version of Gemini 2.5 Pro, the significant gains in broader dataset comprehension and multi-document synthesis highlight its enhanced capabilities for complex enterprise workloads.

Read our first look on Gemini 2.5 Pro here, and get started today with Gemini 2.5 Pro, available in Box AI Studio and with Box AI APIs.