Thoughts on AI-Assisted Development: Lessons from Building with LLM Agents

Eli Bendersky recently shared his experience using LLM agents to develop a new Go project called watgo, building on his previous success with restructuring a Python project with AI assistance. His reflections offer valuable insights for any developer exploring AI-assisted workflows.

The Development Process

For new projects requiring extensive design, Bendersky recommends starting with API sketches in Markdown files committed to the repository. This creates a reference point for both human and AI to work from. When implementing, he suggests asking the agent to write small, reviewable Changelists (CLs) in a logical order.

"Sometimes a single CL is a huge step forward, but requires lots of review, cleanup and refactoring to be viable," Bendersky notes. He's had instances where an agent produced several days of work in a single CL, which then required hours of cleanup and refactoring instructions. "Overall, it's still a productivity gain, just not as much as some pundits would like us to believe."

Keeping Humans in the Loop

Bendersky categorizes projects into two types when working with agents:

1. Low importance/prototype projects where deep code understanding isn't necessary
2. High importance projects requiring long-term maintenance

For the latter category, "vibe-coding" (submitting agent code without review) is ill-advised. The watgo project falls into this second category, where Bendersky insists on reviewing and guiding all code before submission.

"Even if the cost for writing code went down, maintaining a project is so much more than that," he explains. "It's triaging and fixing bugs, it's thinking through what needs to be done rather than how to do it, it's keeping the code healthy over time, and so on."

Practical Workflow

Bendersky recommends using a CLI agent running locally in your repository, with a separate VSCode window open for reviewing changes and making tweaks. Once satisfied with the changes, manually create commits. This approach makes it easier to maintain discipline during the review process.

"Keeping the CLs small is imperative," he emphasizes. "It's very tempting to sprint ahead submitting thousands of lines of code every day, but this temptation has to be avoided. Coding with an agent is like speed-reading; yes, you're making more progress, but comprehension suffers the faster you go."

Testing Strategy

A solid testing strategy remains crucial for AI-assisted development. "Agents produce - by far - the best results when they have a solid test suite to test their code against," Bendersky states.

For watgo, he adapted test suites from the WASM spec and wabt project. If your project lacks such tests, building one should be your first priority. However, he warns against self-reinforcing loops: "it's dangerous to trust agents for both the tests and the implementations tested against them."

Language Choice Considerations

Bendersky found Go to be particularly well-suited for agent-written projects. "Go is a fantastic language for agents to write, because it's designed to be very readable by humans," he explains.

The strengths of Go that make reviewing agent code more positive include:

• Infrequent language changes
• Fewer ways to accomplish the same task
• Rich standard library
• Opinionated choices already made
• Readability-focused design

"Since most of the time spent by humans when using agents is reading rather than writing code, these effects compound and produce a great experience," Bendersky observes.

Final Thoughts on Learning

For those new to a subject, Bendersky strongly recommends against using agents as a shortcut to learning. "To really learn something, you have to work through it from scratch, yourself, reading, designing, writing the code. Agents don't change this basic fact."

He cautions junior engineers to be particularly careful when relying on LLMs: "There's no replacement to hard-won experience and the sweat and tears of learning new, challenging topics. Learning is supposed to be hard; if it's too easy, you're probably not learning."

For senior engineers, however, agents can be valuable productivity tools when used judiciously—helping to increase productivity, avoid tedious tasks, and overcome procrastination.

Read more about Eli Bendersky's experience on his website.