Geekbench 6.7 Flags Intel BOT-Optimized Scores as Invalid

The latest version of Geekbench 6.7 has introduced a controversial change that could reshape how we evaluate Intel's newest processors. The popular benchmarking tool now detects when Intel's Binary Optimization Tool (BOT) is enabled and automatically flags those results as invalid, preventing them from being compared to other CPUs in the database.

This decision comes after Primate Labs, the developer behind Geekbench, conducted extensive testing of BOT and found that while it delivers consistent performance improvements, those gains paint an "unrealistic picture" of daily usage scenarios.

What Is Intel's Binary Optimization Tool?

BOT is a feature exclusive to Intel's latest Core Ultra 200 Plus (Arrow Lake refresh) and Core Ultra 300 (Panther Lake) processors. The tool works by running checksums on executables to identify them and then compiling optimized binaries specifically for those applications.

When BOT detects a supported application, it replaces the standard code with these specially tuned binaries, theoretically delivering better performance. However, the catch is that BOT support is limited to only a handful of applications at present.

Testing Reveals Selective Performance Boosts

Primate Labs' investigation revealed that BOT's impact varies dramatically depending on the workload. While overall multi-core and single-core scores saw modest improvements of around 5.5%, certain specific tests experienced much more dramatic gains.

For instance, HDR processing workloads achieved approximately 30% better results with BOT enabled. This selective optimization demonstrates how BOT can artificially inflate scores in specific scenarios while having minimal impact on others.

The fundamental issue, according to Geekbench, is that their benchmark is designed to evaluate performance across a broad range of workloads that accurately represent real-world usage. BOT's approach of replacing varied code with specifically tuned binaries for only two desktop CPU families (currently) undermines this goal.

How Geekbench 6.7 Handles BOT Detection

With the release of version 6.7, Geekbench has implemented a detection mechanism that identifies when BOT is active during testing. Once detected, the benchmark marks the run as invalid and excludes it from the database comparisons.

Interestingly, BOT doesn't yet have optimized binaries for Geekbench 6.7 itself, so users won't see any performance difference when running the benchmark with BOT enabled. However, the detection system is in place for future compatibility.

This means that even when Intel eventually creates BOT-optimized binaries for Geekbench 6.7, any scores generated with BOT enabled will be marked as incomparable to other CPUs in the database.

Broader Implications for CPU Benchmarking

The controversy surrounding BOT highlights a growing challenge in CPU benchmarking: how to fairly evaluate processors that can dynamically optimize for specific workloads. While BOT delivers real performance improvements in supported applications, those gains may not translate to the diverse range of software most users run daily.

Geekbench's stance is that benchmarks should reflect real-world performance across varied workloads, not optimized results for a limited set of applications. This philosophy has led them to exclude BOT-enabled results entirely rather than trying to account for the optimization.

Other Improvements in Geekbench 6.7

Beyond the BOT detection feature, Geekbench 6.7 includes several other updates:

• Improved SoC identification on Android devices, now reporting actual chip models along with their architecture
• Display of RISC-V processor names instead of just ISA strings
• Enhanced stability for Arm-based Linux systems running multi-threaded workloads

These changes continue Geekbench's tradition of refining its cross-platform benchmarking capabilities while maintaining consistency across different hardware architectures.

The Future of CPU Performance Measurement

The introduction of BOT and Geekbench's response to it raises questions about the future of CPU benchmarking. As processor manufacturers develop more sophisticated optimization techniques, benchmark developers may face increasing pressure to either adapt their tests or exclude certain optimization methods entirely.

For now, users of Intel's latest processors will need to decide whether they value the potential performance gains from BOT in supported applications or prefer to have their benchmark scores included in broader performance comparisons. With Geekbench 6.7, that choice has essentially been made for them.

The debate over BOT and its impact on benchmarking is likely to continue as more users test the feature and as Intel potentially expands its support to additional applications. What's clear is that the relationship between hardware optimization and fair performance measurement remains a complex and evolving challenge in the tech industry.