When Ansible 2.19 breaks on Ubuntu 18.04 due to Python version constraints, manual Python installation and clever workarounds can restore functionality while maintaining idempotency.
Ansible's rapid evolution has created an unexpected challenge for administrators managing older infrastructure. As the automation tool advances, its compatibility with legacy systems becomes increasingly strained, particularly when it comes to Python version requirements. This tension recently manifested for me when Ansible 2.19 ceased functioning on an Ubuntu 18.04 server that had been running smoothly with version 2.14.
The Python Version Dilemma
The root cause was straightforward yet frustrating: Ubuntu 18.04 ships with Python 3.6, while Ansible 2.19 dropped support for Python versions below 3.7. This server, despite receiving security updates through Canonical's Ubuntu Pro program until April 2028, would never receive a Python version bump. The error messages provided little guidance, simply failing with cryptic JSON deserialization errors that offered no hint about the underlying Python version incompatibility.
Manual Python Installation with pyenv
The solution began with installing Python 3.12 using pyenv, a Python version management tool. After installing the necessary build dependencies through apt-get, pyenv made it straightforward to compile and install the newer Python version alongside the system Python. The installation process involved cloning the pyenv repository and using its install command to fetch Python 3.12.12.
This approach preserved the system Python while providing access to a modern interpreter. The new Python installation lived at /home/admin/pyenv/versions/3.12.12/bin/python3, which needed to be specified in the Ansible inventory using the ansible_python_interpreter variable.
The Apt Module Challenge
With Python upgraded, Ansible could execute playbooks again, but package management remained broken. The python3-apt module, essential for Ansible's apt operations, appeared tightly coupled to the system Python version and couldn't be imported from the pyenv environment. This created a significant obstacle since package installation represents a core automation task.
Building Idempotent Workarounds
Rather than abandon the modern Ansible version, I developed a workaround that maintained idempotency while using raw commands for package operations. The approach involved two key steps: first, checking package installation status using dpkg-query, and second, conditionally installing packages only when necessary.
For the status check, the playbook executed dpkg-query -W -f='${Status}' for each target package, capturing the output to determine whether installation was needed. This command returns "install ok installed" for present packages and provides no output for missing ones.
Based on this information, the playbook conditionally ran apt-get update && apt-get install -qyy only for packages that weren't already installed. The conditional logic also ensured these steps only executed on older Debian/Ubuntu versions where the python3-apt module would be unavailable.
The Complete Solution
The final implementation involved modifying playbooks to skip standard package modules on older systems and instead use the raw command approach. The solution maintained Ansible's declarative nature while working around the Python module compatibility issue. For systems running Debian 11+ or Ubuntu 20.04+, the standard package modules continued to function normally.
This approach demonstrates how automation practitioners can extend the useful life of legacy infrastructure while still benefiting from modern tool versions. By understanding the underlying system constraints and building intelligent workarounds, it's possible to maintain compatibility without sacrificing the benefits of current software versions.
The broader lesson extends beyond this specific scenario: as automation tools evolve, administrators of mixed-infrastructure environments must develop strategies for bridging version gaps. Whether through containerization, version management tools like pyenv, or creative playbook design, maintaining operational continuity often requires thinking beyond standard module usage and embracing more fundamental system interactions.
For organizations managing diverse infrastructure with varying lifecycle policies, such techniques prove invaluable. They allow teams to standardize on modern automation practices while accommodating the reality that not all systems can or should be upgraded simultaneously. The key lies in understanding both the tools' requirements and the underlying system capabilities, then building bridges between them that preserve both functionality and maintainability.

Comments
Please log in or register to join the discussion