GitHub's Copilot Applied Science team achieved remarkable productivity gains by treating AI agents as primary contributors, shipping 11 agents and four skills in three days through agent-first development practices.
)
The Evolution of AI-Assisted Development
When Tyler McGoffin found himself drowning in hundreds of thousands of lines of code from agent trajectories, he discovered something profound: the same repetitive analysis tasks that drove him to use GitHub Copilot could be automated entirely. What began as a personal productivity hack evolved into a team-wide methodology that fundamentally changed how software gets built at GitHub.
The Problem That Sparked Innovation
The catalyst was deceptively simple. As an AI researcher analyzing coding agent performance, McGoffin spent countless hours examining trajectories—detailed records of how agents attempted to solve tasks. Each benchmark run produced massive JSON files, and with dozens of runs to analyze daily, the manual workload became unsustainable.
"I used GitHub Copilot to surface patterns in the trajectories then investigated them myself—reducing the number of lines of code I had to read from hundreds of thousands to a few hundred," McGoffin explains. But the engineer in him saw an opportunity: "I want to automate that."
Building for Agent-First Development
The solution, dubbed "eval-agents," wasn't just about creating automation tools—it was about reimagining the entire development process. The guiding principle was clear: engineering and science teams work better together when agents are treated as first-class contributors.
The Three Pillars of Agent-Driven Development
1. Conversational Prompting Strategies
The most surprising insight was that effective AI collaboration mirrors human collaboration. "Guide its thinking, over-explain your assumptions, and leverage its research speed to plan before jumping into changes," McGoffin advises.
Instead of terse commands, successful prompts read like stream-of-consciousness planning sessions. For example, when adding regression tests, the prompt began with: "/plan I've recently observed Copilot happily updating tests to fit its new paradigms even though those tests shouldn't be updated. How can I create a reserved test space that Copilot can't touch or must reserve to protect against regressions?"
This conversational approach led to more thoughtful solutions than directive commands ever could.
2. Architectural Excellence as Priority
Perhaps the most counterintuitive finding was that traditional engineering best practices—refactoring, documentation, testing—became even more critical when working with AI agents. "Gone are the days where deprioritizing this work over new feature work was necessary," McGoffin notes.
The team spent significant time on what many engineers consider "non-essential" work: renaming variables for clarity, restructuring files for better navigation, writing comprehensive documentation, and adding test cases for edge conditions uncovered during development.
This investment paid dividends. With a well-maintained, agent-first codebase, Copilot could navigate patterns and understand context as effectively as any human engineer.
3. Blameless Iteration Culture
The shift from "trust but verify" to "blame process, not agents" represented a philosophical transformation. Just as effective human teams build systems to prevent mistakes rather than punish individuals, successful agent-driven development requires robust guardrails.
This means implementing strict typing to ensure interface compliance, comprehensive linters to enforce patterns, and extensive test suites that give confidence when making changes. "When Copilot has these tools available in its development loop, it can check its own work," McGoffin explains.
The Results: 11 Agents in 3 Days
The methodology proved itself immediately. Four scientists, new to the project, shipped 11 new agents, four new skills, and an entirely new concept called "eval-agent workflows" in under three days. The codebase changed by +28,858/-2,884 lines across 345 files.
This wasn't just about speed—it was about quality and sustainability. The team discovered that treating AI agents as junior engineers who need proper onboarding, clear context, and protective guardrails led to more maintainable, understandable code.
The New Development Loop
McGoffin outlines a systematic approach that combines planning, implementation, and continuous improvement:
- Plan with /plan - Use conversational prompts to outline features, ensuring testing and documentation are included
- Implement with /autopilot - Let Copilot handle the implementation
- Review iteratively - Use Copilot Code Review agent, addressing comments until satisfied
- Human review - Enforce patterns and principles
- Continuous maintenance - Regularly prompt for missing tests, code duplication, and documentation gaps
The Bigger Picture
What makes this story compelling isn't just the productivity gains—it's what it reveals about the future of software development. The skills that make someone a great engineer and teammate are the same skills that make them great at building with AI.
"The technology is new. The principles aren't," McGoffin emphasizes. Clean architecture, thorough documentation, meaningful tests, and thoughtful design remain fundamental, regardless of who (or what) is doing the coding.
Practical Takeaways
For teams looking to adopt agent-driven development:
- Invest in your codebase quality - The better your foundation, the more effective your AI collaborators become
- Embrace conversational prompting - Treat AI like a thoughtful colleague, not a command-line tool
- Build robust guardrails - Implement testing, linting, and type checking that help AI agents self-correct
- Maintain blameless culture - When mistakes happen, improve the process, not the blame
- Document everything - Clear documentation helps both humans and AI understand your codebase
The Future of Development
Agent-driven development represents more than a productivity hack—it's a fundamental shift in how we think about software creation. By treating AI agents as primary contributors and applying the same principles we use for human collaboration, teams can achieve remarkable results while maintaining code quality and team cohesion.
As McGoffin discovered, sometimes the path to your most interesting work involves automating away the tasks you thought were essential. The question isn't whether AI will change how we build software—it's whether we're ready to change how we work with it.
Ready to try it yourself? Download Copilot CLI, activate it in any repository, and use the planning prompt: "/plan Read and help me plan how I could best improve this repo for agent-first development"
Tags: AI agents, automation, GitHub Copilot, GitHub Copilot CLI
Comments
Please log in or register to join the discussion