When a developer upgraded from an M1 MacBook Air to an M5 MacBook Pro, they seized the opportunity to benchmark Python performance across versions and hardware—revealing critical insights for CPU-bound workloads. The tests focused on Knave, a pure-Python serialization library handling a proprietary CSV-like format called "etab," which must operate in locked-down environments with zero dependencies. Generating and processing 1 million rows with varied data types, the benchmarks exposed two key trends:

Hardware Gains Outpace Language Optimizations
Python 3.7 to 3.11 showed steady performance improvements (attributed to core optimizations), but versions 3.12–3.14 plateaued. In contrast, hardware advancements proved dramatic: the M5 MacBook Pro executed tasks nearly twice as fast as the M1. Reference systems like older Xeon (AWS t3.medium) and Ryzen processors lagged significantly, underscoring Apple Silicon's evolution.

Bottlenecks and Optimization Frontiers
Profiling via cProfile (Snakeviz) and py-spy (Speedscope) pinpointed critical inefficiencies:
- Row/column splitting consumed ~40% of runtime due to pure-Python handling of edge cases in file parsing.
- Field serialization/deserialization faced overhead from null-handling logic and type conversions, where minor per-call costs compounded at scale.

"Function call and dereferencing overheads add up very significantly in CPU-bound code," notes the developer, highlighting that while Python's built-in csv module is faster, it couldn't handle Knave's corner cases without compatibility trade-offs.

Methodology and Future Promise
All tests used pyenv-built Pythons with --enable-optimizations --with-lto, and JIT enabled for 3.13/3.14. Scripts ensured reproducibility, with MyPyC compilation showing impressive gains—though compatibility issues with older Python versions limited testing. The anticipated 15% speedup in Python 3.15 could shift dynamics, especially for serialization workloads.

For developers, this underscores a strategic balance: leverage newer hardware for immediate gains, target PyPy/MyPyC for high-performance scenarios, and monitor Python’s JIT evolution. As the ecosystem matures, profiling remains essential to tackle hidden overheads in seemingly trivial operations.

Source: CrewTech.se