AI Costs Are Rising – New Hardware Promises Relief, But It Won’t Reach Most Users Until 2027

Regulatory action → What it requires → Compliance timeline

Regulation: EU AI Act – Article 10(2) (Effective 1 January 2027)

• What it requires: Providers of high‑risk AI services must disclose per‑token pricing, demonstrate that cost structures do not create discriminatory barriers, and maintain an audit trail of token consumption for at least two years.
• Compliance timeline: Companies must publish a pricing‑transparency statement by 1 July 2026 and implement the audit‑log feature by the act’s effective date.

---

1. Why AI pricing is climbing

Generative‑AI models such as GPT‑5.5, Claude 3, and Gemini Flash 3.5 are being used for more token‑intensive workloads – agent‑based automation, code generation, and real‑time content creation. Those workloads consume orders of magnitude more tokens than traditional chat interactions. Because inference costs dominate the operating expense, vendors have begun to raise per‑token fees:

• OpenAI: $5 per million input tokens, $0.50 per million cached tokens, $30 per million output tokens.
• Google Gemini Flash 3.5: 3‑6× higher than the previous Flash‑Lite tier.

The price hikes reflect the gap between current GPU/accelerator efficiency and the token‑throughput required for these agent‑level applications.

2. New hardware that could lower the cost per token

Vendor	Upcoming product	Expected efficiency gain	Planned availability
Nvidia	H100‑X (next‑gen tensor core)	2.5× lower FLOPs per token	H2 2026
AMD	MI300‑X AI accelerator	2× lower power per token	H2 2026
Intel	Gaudi 3 custom ASIC	1.8× lower latency per token	H2 2026
Google	TPU‑v5e	3× lower cost per token in Cloud AI	H2 2026

All four announcements promise a measurable reduction in the cost per token metric that investors use to gauge AI profitability. However, each product still requires a ramp‑up period for silicon validation, firmware integration and data‑center deployment.

3. When will the savings be visible to end‑users?

The supply chain for advanced AI silicon is constrained by:

1. Wafer fab capacity – leading fabs are booked through 2027 for high‑performance nodes.
2. Custom software stack development – compilers, drivers and model‑optimization tools need months of testing.
3. Datacenter integration – large hyperscalers must retrofit existing racks, which adds logistical delay.

Because of these factors, most cloud providers project general‑availability of the new hardware in early to mid‑2027. Until then, pricing models will continue to reflect the higher cost of running existing GPUs at scale.

4. Compliance implications under the EU AI Act

The EU AI Act treats per‑token pricing as a consumer‑protection issue for high‑risk AI services. Companies must:

• Publish a transparent pricing matrix that separates input, cached and output token rates.
• Provide a cost‑impact assessment for users deploying agent‑based solutions, showing how token consumption translates into monetary cost.
• Maintain immutable logs of token usage for each user account, stored for a minimum of two years, to support audit requests.

Failure to meet these obligations after 1 January 2027 can result in fines up to €30 million or 6 % of global turnover, whichever is higher.

5. Practical steps for compliance officers

1. Audit current pricing disclosures – ensure they break out input, cached and output token rates.
2. Implement usage‑based billing APIs – align with the trend toward token‑metered pricing while keeping the data needed for the audit log.
3. Set up a token‑consumption data lake – capture raw token counts per request, enrich with user identifiers, and retain for the statutory period.
4. Plan for hardware transition – map existing workloads to the upcoming H100‑X or MI300‑X capabilities; schedule migration windows for Q1‑Q2 2027.
5. Communicate timeline to customers – provide a clear roadmap showing when lower‑cost tokens will become available, mitigating churn risk.

6. What this means for executives

• Margin pressure will persist until the new accelerators are in production. Expect token‑price elasticity to remain low through 2026.
• Cost‑per‑FTE calculations will become a standard KPI; finance teams should model token spend against projected headcount savings.
• Strategic investment in in‑house model training may offset external provider fees, but only if the organization can acquire the next‑gen hardware early.

7. Outlook

The AI‑hardware refresh promises a significant reduction in inference cost, but the lag between silicon announcement and customer‑visible savings creates a window where providers can continue to raise token prices without immediate competitive pressure. Compliance teams must prepare for the EU AI Act’s transparency requirements now, while product managers should align roadmaps with the expected 2027 hardware rollout.

---

Prepared by the compliance office, reflecting current regulatory expectations and technology timelines.