How to get multiple agents to play nice at scale - Stack Overflow

When it comes to AI agent orchestration, the challenges multiply exponentially as you scale. That's the central theme explored in a recent Stack Overflow podcast featuring Chase Roossin, group engineering manager, and Steven Kulesza, staff software engineer, from Intuit. They dive deep into what might be the hardest problem in engineering right now: getting multiple AI agents to work together in a complex system.

The Agent Coordination Problem

The fundamental challenge isn't just about building capable AI agents—it's about making them cooperate effectively. As Roossin and Kulesza explain, when you have multiple autonomous agents operating simultaneously, you're dealing with emergent behaviors that can be unpredictable and difficult to manage. This becomes particularly acute in enterprise environments where reliability and consistency are paramount.

The podcast explores several key approaches to this problem. One major strategy involves automated evaluations to make agent behaviors more predictable. By implementing rigorous testing frameworks, teams can identify edge cases and failure modes before they impact production systems. This isn't just about unit testing individual agents—it's about testing the interactions between agents and their collective behavior as a system.

Agent Swarms vs. Single Skilled Agents

A fascinating discussion point centers on the trade-offs between deploying multiple specialized agents versus relying on a single highly skilled agent. The "agent swarm" approach—where multiple agents each handle specific subtasks—offers flexibility and specialization but introduces coordination overhead. Conversely, a single agent approach simplifies architecture but may struggle with complex, multifaceted problems.

Intuit's experience suggests that the answer isn't binary. Their architecture evolved to use both approaches depending on the use case. For routine, well-defined tasks, specialized agents excel. For novel or ambiguous scenarios, a more general-purpose agent might be preferable. The key is building systems that can dynamically route requests to the appropriate agent or combination of agents.

Customer Behavior as Architecture Driver

Perhaps the most insightful aspect of the discussion is how customer behavior shaped Intuit's technical architecture. The team discovered that user interactions with their AI systems revealed unexpected patterns and requirements. This led to architectural decisions that prioritized flexibility and adaptability over rigid, predetermined workflows.

This customer-centric approach to system design is particularly relevant as AI agents become more prevalent in consumer-facing applications. The architecture must accommodate not just the technical requirements of the agents themselves, but also the unpredictable ways humans will interact with them.

Practical Takeaways for Engineering Teams

For engineering teams grappling with similar challenges, several practical insights emerge:

Automated evals are non-negotiable: Building reliable multi-agent systems requires comprehensive testing frameworks that can simulate complex interactions and edge cases.

Start simple, then scale: Begin with single-agent solutions for well-defined problems before attempting complex multi-agent orchestration.

Monitor emergent behaviors: Multi-agent systems can develop unexpected behaviors that only emerge at scale—continuous monitoring is essential.

Design for adaptability: Customer behavior will inevitably surprise you, so build systems that can evolve based on real-world usage patterns.

The Broader Context

This discussion from Intuit fits into a larger trend in software engineering where AI agents are moving from experimental prototypes to production systems. As these systems scale, the coordination challenges become central to their success or failure.

Intuit's experience is particularly valuable because they've been working on these problems in the context of financial software, where reliability and accuracy are critical. Their approaches to automated evals and adaptive architecture offer blueprints for other organizations facing similar challenges.

The podcast also highlights how Intuit has been sharing their learnings through other channels, including their work on LLM best practices and democratizing AI development. This commitment to knowledge sharing reflects the collaborative spirit of the engineering community.

Looking Forward

As AI agents become more sophisticated and prevalent, the challenges of coordination and orchestration will only grow more complex. The insights from Intuit suggest that success in this domain requires not just technical innovation but also organizational practices that support rapid iteration and learning from real-world usage.

For developers interested in working on these cutting-edge problems, Intuit is actively hiring. Their approach to building reliable, scalable AI systems offers a compelling example of how to tackle one of engineering's most challenging frontiers.

The conversation with Roossin and Kulesza ultimately reveals that getting multiple AI agents to "play nice" isn't just a technical problem—it's a systems problem that requires careful attention to architecture, testing, user behavior, and organizational learning. As more companies venture into this territory, these lessons will become increasingly valuable.