Green IT: How to Reduce the Impact of AI on the Environment

Green IT focuses on reducing IT's environmental footprint by rethinking how we build, deploy, and power IT systems. This includes everything from energy-efficient hardware and optimized code to data center design and software architecture decisions that minimize resource use.

AI is creating unprecedented environmental challenges without corresponding governance. Generative AI requires constant and significant computational power even for inference time, that is, usage time, when you prompt the system. This translates directly to energy consumption. Every time someone uses a chatbot, generates text or images, or triggers any generative AI feature, it consumes enormous amounts of energy. Constant inference is becoming a massive energy sink as AI features proliferate.

Due to the usage of AI, hardware churn is accelerating. GPU chips typically last only 2-3 years. AI is not free, but users are shielded from the costs. There's a complete lack of transparency; users don't see the environmental costs behind each query, so there's no natural restraint.

What's most concerning is how teams deploy AI features without questioning if they're truly necessary. We're scaling by default rather than by design, and even Europe's first regulatory frameworks, such as the EU AI Act, fall short on these sustainability impacts by only considering energy consumption, with no clear enforcement mechanisms.

According to the literature, reducing the impact of artificial intelligence on the environment will have to come from improvements in model compression, quantization, novel architectures, and hybrid strategies. Some examples include Retrieval-Augmented Generation, Small Language Models, and combining offline with online inference. But technology alone is not enough.

We have been designing digital systems for a stable world, a world that no longer exists. Today, the climate crisis is reshaping our assumptions about scale, cost, and resilience. It is time to rethink our assumptions. Sustainability is not just an opportunity to redefine technology. It is a constraint we must now build within, just like latency or scalability.

What's Changed Since Last Year

At QCon London 2025, I argued that the majority of AI's environmental impact reduction would come from model creation improvements (compression, quantization, and novel architectures) while inference-phase interventions (AI literacy, appropriate model selection, inference budgeting, compute constraints, and AI governance integrating FinOps and GreenOps) represented necessary but secondary work. This assessment has proven incomplete.

Emerging evidence indicates that for massively deployed consumer AI services, cumulative inference energy consumption can exceed one-time training costs within months of deployment. Technical efficiency gains from compression and quantization have materialized, typically delivering 2-4x improvements in inference efficiency. Available data suggests these improvements have coincided with exponential growth in inference volumes rather than absolute consumption reductions.

Publicly documented adoption of inference governance mechanisms remains rare. The pattern is consistent with rebound effect predictions: efficiency improvements enable expanded deployment and new use cases rather than reduced overall consumption.

The critical insight: technical optimizations deployed without complementary inference-phase governance mechanisms - budgeting, decision frameworks, appropriate model selection - risk accelerating rather than mitigating environmental impact through broader AI adoption.

Practical Steps Organizations Can Take

At Bpifrance, we are working to make the environmental costs of AI more visible and manageable. Through our Green IT working group, we have been exploring tools like Ecologits, LiteLLM, and Langfuse to help teams monitor, budget, and steer AI usage more effectively.

We are also focusing on building a cultural shift within the organization. With my PromptSage GPT project, I am building AI literacy, helping teams understand that thoughtful prompting is not just about better results; it is also a sustainability practice. Using AI more intelligently reduces unnecessary computation.

This is why I encourage product teams to take ownership of their usage impact and to embed environmental telemetry into the products.

Driving Successful Green IT Transformations

We already have many of the technical solutions. What we often lack is the shift in perspective. Start by asking different questions. Not just "Can we build it?" but "Should we?" Not just "How fast?" but "At what cost?" These questions open the door to more sustainable decisions.

To make this shift real, we need three key qualities: the curiosity to explore beyond our comfort zones, the honesty to acknowledge our real impact rather than externalizing costs, and the courage to ask uncomfortable questions of our providers, our vendors, and ourselves.

We have to unlearn to learn.

The Path Forward

The environmental impact of AI represents one of the most pressing challenges in modern computing. As AI features proliferate across applications and services, the cumulative energy consumption from inference operations threatens to dwarf the already substantial costs of model training.

Organizations must move beyond technical optimizations alone and embrace a holistic approach that combines efficient architectures with governance frameworks, transparency tools, and cultural change. The solutions exist - from model compression and quantization to retrieval-augmented generation and small language models - but their effectiveness depends on deployment strategies that prioritize necessity over novelty.

Sustainability is no longer optional; it's a fundamental constraint that must be built into our systems from the ground up, just like any other critical requirement.