The AI Slop Problem: How Open Source Projects Are Fighting Back Against Bot Contributions

When GitHub recently celebrated massive growth in their contribution metrics, few noticed what this achievement actually represented: a deluge of AI-generated content that threatens to drown out legitimate human contributions. For Archestra, a VC-backed startup building developer tools, this became an existential problem that forced them to implement controversial measures to preserve their open source community's integrity.

The first warning signs appeared when Archestra posted a $900 bounty issue for adding "MCP Apps" support to their platform. While legitimate contributors engaged meaningfully, the conversation quickly spiraled out of control as AI bots flooded the thread with nonsensical implementation plans and even aggressive comments. By the time the issue closed, it contained 253 comments—mostly noise that buried valuable input from real developers like @ethanwater, @developerfred, and @Geetk172.

"Our GitHub notifications became a wall of noise," explains Ildar Iskhakov, Archestra's CTO. "Real conversations from contributors actively working on bounties were getting buried, and we had to dedicate significant resources just to clean up the mess."

The problem escalated from isolated incidents to a full-blown epidemic. When adding x.ai provider support to Archestra, the team received 27 pull requests—most from AI-generated content that hadn't even been tested. "One of our team members had to spend half a day every week cleaning AI garbage out of the repo, removing untested PRs and closing hallucinated issues," Iskhakov notes. "When we forgot to do so, our repo quickly became a place completely unfriendly to legitimate contributors."

Attempted Solutions and Their Limitations

Archestra's initial response focused on identifying and filtering AI-generated content. They developed "London-Cat," a reputation system that calculated contributor scores based on merged PRs and other signals. While this helped distinguish between human and bot contributors, it didn't stop the spam.

Their next attempt, an "AI sheriff" bot, proved more problematic. While it successfully removed some AI-generated content, it also mistakenly flagged and closed legitimate PRs, creating new problems for the team. These reactive approaches failed to address the root cause: the fundamental mismatch between GitHub's open contribution model and the reality of AI-generated content flooding the platform.

The Nuclear Option: Contributor Onboarding

Faced with diminishing returns from their filtering efforts, Archestra made a difficult decision: they would implement a whitelist system to restrict who could create issues, open PRs, and leave comments. This "nuclear option" represents a significant departure from the traditional open source ethos, but the team concluded that quality matters more than quantity—especially when the quantity is artificially inflated by AI slop.

"We don't value metrics pumped by AI slop," Iskhakov states. "We want Archestra to be a great piece of software that everyone can contribute to, without it being swallowed by AI bots."

The implementation required clever GitHub hacking. The platform offers a "Limit to prior contributors" setting that restricts repository interaction to those who have previously committed to the main branch. However, this setting treats both AI bots and legitimate new contributors equally—locking both out.

Technical Workarounds for Legitimate Contributors

To solve this dilemma, Archestra developed a five-step onboarding process that converts legitimate contributors into "prior contributors" through a technical workaround:

1. Contributors complete an onboarding form on Archestra's website, which includes ethical AI guidelines and a CAPTCHA
2. A GitHub Action triggers on form submission
3. The action looks up the user's GitHub ID via the API
4. It adds the contributor's handle to an EXTERNAL_CONTRIBUTORS.md file
5. A commit is pushed to main, authored under the contributor's account using Git's --author flag

This approach leverages GitHub's definition of "prior contributor"—anyone whose GitHub account is the author of a commit on main. By creating a commit attributed to the external user (while maintaining Archestra's account as the committer), the system grants the contributor immediate access to the repository.

The technical implementation relies on GitHub's noreply email format ([email protected]). By combining the user's GitHub ID with this email format and using Git's --author flag, Archestra can attribute commits to external contributors while maintaining proper attribution.

Broader Implications for Open Source

Archestra's experience reflects a growing crisis in the open source ecosystem. As GitHub reports massive metric growth—much of it AI-generated—projects face increasing challenges maintaining quality and usability. The problem extends beyond mere inconvenience; AI-generated content introduces security risks, as demonstrated when attackers attempted to steer conversations in the LiteLLM repository using AI bots.

"Slop is not only demotivating contributors who want to spend their time doing good and have to break through the wall of noise instead, it's also bringing a substantial security risk," Iskhakov warns.

The tension between open collaboration and quality control highlights a fundamental challenge facing the open source community. While platforms like GitHub benefit from inflated metrics that suggest vibrant communities, the reality for many maintainers is a constant battle against low-quality or malicious AI-generated content.

The Path Forward

Archestra's solution represents one approach to this problem, but it's not without limitations. The whitelist system creates friction for legitimate contributors and contradicts the traditional open source ethos of accessibility. Additionally, the technical workarounds rely on GitHub's specific implementation details, which could change in the future.

As AI-generated content continues to proliferate, the open source community must develop sustainable solutions that balance accessibility with quality. This may involve:

• Platform-level changes to GitHub and similar services
• Improved AI detection and filtering mechanisms
• New community norms around AI-generated contributions
• Hybrid approaches that maintain openness while establishing quality gates

"Dear community, it's time to have a serious talk about the effect AI has on open source," Iskhakov concludes. "The current trajectory threatens the very foundations of collaborative development that has powered innovation for decades."

For projects facing similar challenges, Archestra has shared their approach publicly, though they acknowledge it's a stopgap measure rather than a complete solution. The real test will come as the community collectively addresses this challenge—finding ways to preserve the open, collaborative spirit of open source while adapting to the realities of AI-generated content.

As the lines between human and machine contributions continue to blur, one thing remains clear: the future of open source depends on our ability to maintain quality without sacrificing the accessibility that makes these projects valuable in the first place.