Fabraix Playground: Open-Source AI Agent Jailbreak Testing Platform

Fabraix Playground is an open-source platform designed to stress-test AI agent defenses through adversarial play. The project, hosted on GitHub at fabraix/playground, provides a live environment where the community can collectively probe AI systems to understand their failure modes and improve security.

The Trust Problem in AI Agents

AI agents are increasingly handling the mechanical, repetitive aspects of work—the tasks that consume time without requiring human creativity. This shift leaves humans to focus on what matters most: thinking, judgment, and creative leaps. However, this transformation hinges on one critical factor: trust.

As the project description notes, "None of it scales until people can hand real tasks to an agent and know it will do what it should — and nothing it shouldn't." This trust cannot be built in isolation by a single team behind closed doors. Instead, it must be earned collectively through open testing and shared learning.

How the Playground Works

The platform operates on a simple but powerful principle: every challenge deploys a live AI agent with real capabilities, not a toy scenario or mocked-up system. Each challenge includes:

• A specific persona for the AI agent
• A set of tools (web search, browsing, and more)
• Something the agent is instructed to protect
• A fully visible system prompt

Participants attempt to find ways past the agent's guardrails. When someone succeeds, the winning technique is documented and published for the entire community to learn from. This creates a continuous cycle: published knowledge forces better defenses, which invite harder challenges, which produce deeper understanding.

Community-Driven Security Research

The platform's structure ensures community involvement at every stage:

1. Anyone can propose a challenge by designing the scenario, agent configuration, and objective
2. The community votes on proposed challenges
3. The top-voted challenge moves to production with a ticking clock
4. The fastest successful jailbreak wins
5. The winning technique gets published in full detail

The project emphasizes that "Every technique we publish advances what the community collectively understands about how AI agents fail — and how to build ones that don't."

Technical Architecture

The Playground is built with modern web technologies:

• React frontend using TypeScript, Vite, and Tailwind CSS
• All challenge configurations and system prompts are versioned and open
• Guardrail evaluation runs server-side to prevent client-side tampering
• The agent runtime is being open-sourced separately

Developers can run the platform locally with npm install and npm run dev, connecting to the live API by default or pointing to a local backend using the VITE_API_URL environment variable.

Getting Involved

The project actively welcomes community participation:

• Propose new challenges to design the next scenarios the community tackles
• Suggest agent capabilities, including new tools, behaviors, or workflows
• Report bugs when something isn't working correctly
• Join the Discord community to discuss techniques and share approaches

About Fabraix

The Playground is developed by Fabraix, a company that builds runtime security for AI agents. The platform serves as both a stress-testing environment for their defenses and a way for the broader community to contribute to understanding AI security and failure modes.

As the project description states, "The more people probing these systems, the better the outcomes for everyone building with AI."

The open nature of this project represents a significant shift in how AI security research is conducted. Rather than keeping defensive techniques proprietary, Fabraix is betting that collective, adversarial testing will produce more robust AI agents faster than any closed development process could achieve.