Self-Improving Agent Documentation Through Log Analysis

Maintaining accurate project documentation for AI agents like Claude Code and Codex presents a significant challenge as projects evolve. The CLAUDE.md file (or AGENTS.md equivalent) serves as critical context for these agents, yet manually keeping it updated becomes increasingly burdensome as project complexity grows. This documentation drift creates friction during development, forcing users to repeatedly clarify context or re-explain project specifics across sessions.

A practical solution leverages the detailed session logs automatically generated during agent interactions. Both Claude Code and Codex store these logs locally:

• Claude Code: ~/.claude/projects
• Codex: ~/.codex/sessions

These JSONL files contain comprehensive session records including user prompts, agent responses, and execution details. While the schema differs between tools, both formats capture valuable interaction patterns that reveal documentation gaps.

The optimization process involves directing your agent to analyze these historical logs against your current CLAUDE.md file using prompts like:

"Review my Claude JSONL history files for this project and suggest improvements to CLAUDE.md. Identify instances of user frustration and recurring requests across sessions."

This approach transforms documentation maintenance from a chore into an efficient refinement process. In practice, agents successfully identify:

• Repeated requests for clarification about project structure
• Common misunderstandings about domain-specific terminology
• Frequently referenced files missing from context
• Outdated configuration instructions

However, parsing raw JSONL logs poses efficiency challenges. Agents often generate complex jq commands that struggle with larger log volumes. To solve this, I developed a purpose-built CLI tool that abstracts log parsing complexities:

The CLI enables rapid log analysis that previously took minutes to complete in seconds. Key capabilities include:

• Filtering sessions by date range
• Searching for specific error patterns
• Identifying repeated command sequences
• Aggregating user correction patterns

This efficiency opens possibilities for automated documentation refinement. A scheduled task could run weekly analysis, propose CLAUDE.md updates, and even implement approved changes - creating truly self-improving project documentation.

For teams adopting this approach:

1. Start with targeted log reviews after noticeable friction points
2. Gradually increase analysis scope as trust in the process builds
3. Curate initial suggestions before full automation

The technique demonstrates how leveraging existing telemetry unlocks significant workflow optimizations. As agent ecosystems mature, standardized approaches like AGENTS.md could further streamline this process across tools and teams.

The CLI tool referenced may be published on GitHub based on community interest. Implementation details vary between Claude and Codex due to their different log schemas.