Building AI with AI: The Challenges of 'Slop Issues' in Open Source Development

Armin Ronacher, known for creating the Jinja templating engine and Flask web framework, has shared detailed reflections on the challenges of using Pi—an AI tool—to build Pi itself. In a recent blog post, Ronacher explores how AI assistants are changing the dynamics of open source development, creating new forms of 'slop' that burden maintainers and potentially undermine the collaborative nature of software projects.

The Changing Role of Issue Trackers

The most immediate change Ronacher observes is how AI agents are transforming the role of issue trackers in open source projects. When humans report issues, the descriptions are no longer just messages from users to maintainers—they've become inputs for prompts in AI sessions.

"We use them as inputs for prompts in Pi sessions," Ronacher explains. "It is something I might hand to my clanker and say: 'understand this, reproduce it, inspect the code, and propose a fix.' That means the shape of the issue matters in a new way."

This shift has created a new class of problematic issues that Ronacher calls 'slop issues'—reports that are 5% human and 95% AI-generated, often containing plausible but completely wrong diagnoses.

The Problem with AI-Generated Issues

The most frustrating failure mode, according to Ronacher, is when people submit issues that aren't in their own voice. Instead of reporting what they observed directly, they run their observations through an AI first, which rewords and often misrepresents the problem.

"Typically, it was prompted so badly that the conclusions produced are more often than not inaccurate but always full of confidence," Ronacher writes. "The result is complete guesswork on root causes, fake-minimal repros, suggested implementation strategies, analogies to adjacent but often the wrong code, and long lists of error classes that might or might not matter. That is worse than no diagnosis."

Even with custom instructions like "/is" that explicitly tell the AI not to trust analysis written in the issue, the problem persists because humans have already expanded the scope of their observations through AI processing.

Volume and Quality Challenges

Beyond the quality issues, there's also a significant volume problem. Ronacher analyzed Pi's GitHub tracker over 90 days and found that 3,145 external issues and pull requests were submitted by non-approved contributors, with 2,504 auto-closed. Only 17% of issues were reopened, and less than 10% of pull requests were merged.

"Many of the issues and PRs are complete slop and in some cases the humans did not even realize that they created them," Ronacher notes. "Sources of low-quality spam include OpenClaw instances, as well as some skills that people put into their context that seemingly encourage issue creation."

The Over-Engineering Problem

Ronacher also observes a pattern where AI assistants tend to over-engineer solutions, adding unnecessary complexity when the correct fix might be to make bad states impossible rather than handling them gracefully.

"At Pi's core is a rather well-designed session log with invariants that must be upheld," he explains. "The clanker's present-day behavior is to just assume that no such invariants exist, and instead to make the system work with all kinds of malformedness, blowing up the complexity in the process. Almost always, the correct fix is not to handle the bad state, but to make the bad state impossible."

This tendency to add local defenses against every potential problem rather than addressing root causes is creating bloat in AI-generated code, making systems harder to maintain over time.

Careful Parallelism for Issue Investigation

To manage the workflow, Ronacher and the Pi team have developed a system of careful parallelism for reproducing and investigating issues. They use custom slash commands like "/is" (for analyzing issues) and "/wr" (for wrapping up investigations) to create structured workflows.

"In practice this means it's possible to have several Pi windows open, each running /is against a different issue, and the UI keeps the investigations visually distinct while the agents do their independent reproduction and code reading," Ronacher describes. "Once the investigations are done, one can work through them sequentially."

The Future of Open Source in the AI Era

Perhaps most concerning is Ronacher's observation about how AI is changing the fundamental nature of open source collaboration. While AI has increased the amount of code and projects, it hasn't increased the number of people who need software or the number of maintainers who can review it.

"Open Source needs more collaboration, not more isolated work with a machine," Ronacher argues. "Human communication is hard, and it is tempting to avoid it when you can sit alone with your clanker. But isolation is not where Open Source derives its value. The value is in the community and the structure that lets projects outlive their original creators."

Ronacher prefers the term 'clanker' over 'agent' for these AI tools, emphasizing that agency lies with humans, not machines. "To me, clanker is a much preferable term for agent. Agency lies with humans, not with machines. Calling these things agents I still believe is a mistake, but alas."

As AI tools become more prevalent in software development, Ronacher's insights provide a valuable perspective on the challenges ahead—maintaining quality, managing volume, preserving human collaboration, and resisting the temptation to solve problems with local complexity rather than addressing root causes.

Read Armin Ronacher's full post on Building Pi With Pi.