AI in Security Reporting: Cisco's Experiment Reveals Benefits and Pitfalls in Incident Response Documentation

Cisco's recent experiment with using AI to generate security incident reports has revealed both significant time-saving potential and substantial quality control challenges that organizations must address before adopting similar approaches. The networking giant's Talos Incident Response team conducted a tabletop exercise to evaluate how well large language models (LLMs) could create accurate, comprehensive security incident reports, with results demonstrating that while AI can accelerate documentation processes, it introduces new risks that require careful management.

According to findings published in Cisco's blog post by Nate Pors, a senior incident commander in the Cisco Talos team, LLMs exhibit several fundamental limitations when applied to technical security reporting. These limitations stem from the underlying technology's nature as essentially "fancy autocomplete systems" that make educated guesses rather than providing definitive, reliable outputs.

The Cisco team identified four critical problem areas with LLM-generated security reports:

1. Inconsistent Data Usage: LLMs may use different data for each query, making it difficult to rely on them for repeatable, standardized research outcomes. In security contexts, this inconsistency could lead to different analyses of the same incident depending on when the report is generated.

2. Variable Conclusions: The same data can produce different conclusions when processed by an LLM. "In a data breach scenario, a model might suggest a full organization-wide password reset in one instance and a targeted reset in another," Pors noted. The AI then defaults to whichever recommendation it generates first, potentially leading to inappropriate security guidance.

3. Structural Inconsistency: Because LLMs generate content token-by-token, they can create documents with different structures and formatting on each run. "This unpredictability is problematic for professional environments where standardized layouts, such as consistent executive summaries or recommendation sections, are essential for quality control," the Talos team observed.

4. Data Omission: AI systems can discard critical information, potentially overlooking key details that security professionals would consider essential for proper incident documentation and response planning.

To address these challenges, Cisco developed several mitigation techniques that organizations considering AI-assisted reporting should implement:

1. Granular, Single-Task Instructions: Provide the LLM with specific, focused instructions targeting small portions of the report rather than attempting to generate complete documents at once. This approach "significantly reduces the risk of hallucination or cross-contamination between sections."

2. Source Specification: Explicitly instruct the AI which sources to use when generating content, ensuring it draws from appropriate and relevant information rather than potentially including unrelated or inaccurate data.

3. Style and Format Guidelines: Establish clear rules about the style and format of output to maintain consistency across reports, which is particularly important in security contexts where standardized documentation facilitates proper analysis and response planning.

4. Session Isolation: Start a new session and re-enter prompts for each new incident report to prevent cross-contamination of content from one report's source material to another, a problem Cisco observed when editing multiple sample reports within a single session.

Implementing these techniques, Cisco reported that the time required to draft an incident report based on a tabletop exercise fell by 50 percent. Importantly, "A blind test of the sample report in our quality assurance process showed no noticeable drop in overall writing quality." The peer reviewer, professional editor, and management reviewer all made complimentary comments about the report while unaware that it was AI-generated. The peer reviewer specifically noted that the incidence of typos and grammatical errors was far lower than in the average report.

However, significant challenges remain. Cisco found that AI grammar-checking prompts "hallucinated numerous grammar issues" while "failing to identify actual issues," with a success rate below 50 percent and inconsistent performance. "It is currently unsuitable for production use," Pors concluded.

The most critical recommendation from Cisco's experience is that security professionals must "take ownership of every word of the final report." During testing, the LLMs generated recommendations that were "duplicative, irrelevant, or not actionable." If used without manual oversight in production environments, this could result in poor-quality recommendations that might compromise security response efforts.

These concerns become particularly acute when considering that tabletop exercises represent simplified scenarios compared to actual security incidents involving analysis of log files from multiple systems. The complexity of real-world security incidents would likely exacerbate the challenges observed in Cisco's testing.

For organizations considering similar approaches to AI-assisted security reporting, Cisco's experience suggests a phased implementation strategy:

1. Begin with non-critical documentation to test and refine prompting techniques
2. Implement robust human review processes for all AI-generated content
3. Develop standardized templates and guidelines to maintain consistency
4. Establish clear protocols for source verification and content validation
5. Continuously evaluate AI performance and adjust approaches as the technology evolves

As security incident reporting requirements continue to evolve with increasingly complex threats and regulatory expectations, organizations must balance the efficiency benefits of AI against the need for accuracy, consistency, and reliability in their security documentation. Cisco's experiment provides valuable lessons for navigating this challenging landscape while maintaining the high standards required in security response operations.