Backup script ingested an accidental asterisk and deleted everything

A routine backup script failure that deleted an entire organization's data serves as a stark reminder of how simple coding errors can have catastrophic consequences for data protection. The incident, first experienced by a then-21-year-old programmer in 1981, illustrates fundamental error handling failures that remain relevant in today's regulatory environment.

The Incident: A Cascade of Failures

The incident began when a young programmer, whom we'll call Miller, developed a cleanup routine for a mainframe system that created virtual machines and disks for each user. At the end of each day, these VMs were shut down to free up resources for overnight jobs. Miller's routine was designed to remove the drives and back up their contents.

The core issue emerged from how the system handled temporary drive assignments. The mainframe assigned letters A-Z to available user drives, with Miller's routine designed to attach a temporary drive and capture its assigned letter. This approach functioned adequately until the system reached capacity.

"The routine attached to all users' drives and backed them up to a temporary drive," Miller explained. "But you never knew in advance what drive letter the system would assign to the temporary drive. So I wrote a routine to attach it and capture the letter."

The critical failure occurred when the employer provisioned another user account, exhausting all available drive letters. When the system attempted to assign a temporary drive, the call failed. Instead of returning a proper error code, the routine returned an asterisk (*) as a placeholder.

This error then cascaded through the system: the delete command incorporated the asterisk, interpreted it as a wildcard character, and proceeded to delete every file, all data, and all code across the system.

Legal Implications Under Modern Regulations

While this incident occurred in 1981, before modern data protection laws, similar errors today could trigger significant regulatory penalties under frameworks like the GDPR and CCPA.

Under GDPR, organizations can face fines up to €20 million or 4% of global annual turnover, whichever is higher, for serious data breaches. The incident described would likely qualify as a "personal data breach" under Article 4(12) of GDPR, as it involved the accidental or destruction of personal data.

Similarly, California's CCPA imposes obligations on businesses to implement reasonable security procedures and practices to protect personal information. A failure that results in complete data loss could be considered a violation of this requirement, potentially leading to regulatory action and private litigation.

The principle of accountability under GDPR (Article 5) requires organizations to demonstrate compliance with data protection principles. An error of this magnitude, particularly if it resulted from inadequate testing or oversight, could challenge an organization's ability to demonstrate such compliance.

Impact on Users and Organizations

The consequences of such data loss extend far beyond the immediate operational disruption. In Miller's case, it took him a full day to restore data while twenty other employees were unable to work, representing significant productivity loss.

For users, the implications can be severe:

• Loss of personal data with no backup
• Potential identity theft from exposed information
• Breach of confidentiality and trust
• Possible financial losses from disrupted services

For organizations, the impacts include:

• Regulatory fines and penalties
• Reputational damage
• Loss of customer trust
• Operational downtime
• Increased insurance premiums
• Legal liability costs

Preventive Measures and Best Practices

This incident highlights several critical lessons for modern organizations:

1. Robust Error Handling

Error handling must be a fundamental design consideration, not an afterthought. Systems should validate all inputs and reject potentially dangerous characters like wildcards in critical operations.

2. Code Review and Testing

Critical systems should undergo thorough peer review and comprehensive testing, including edge cases and failure scenarios. Miller's acknowledgment that "it was all on me" underscores the danger of insufficient oversight.

3. Principle of Least Privilege

Operations that modify data should have restricted permissions to limit potential damage from errors.

4. Regular Backups and Recovery Testing

Organizations should implement regular backups and regularly test recovery procedures to ensure data can be restored efficiently.

5. Compliance with Data Protection by Design

Modern frameworks like GDPR require data protection to be embedded into system design from the outset, not bolted on later.

6. Monitoring and Alerting

Systems should include monitoring to detect unusual behavior and alert administrators before catastrophic failures occur.

The Evolution of Data Protection

While the specific technology has evolved from mainframes to cloud systems, the underlying principles of data protection remain constant. The incident serves as a reminder that:

• Human error remains a significant factor in data breaches
• Simple coding mistakes can have disproportionate impacts
• Proper oversight and testing are non-negotiable for critical systems
• Data protection requires continuous vigilance, not one-time implementation

In today's regulatory environment, organizations cannot afford to treat such coding errors as mere technical glitches. They represent potential violations of legal obligations and failures in protecting fundamental user rights.

Miller's hard-learned lesson has stayed with him for over 40 years, serving as a cautionary tale for generations of programmers. As organizations increasingly rely on automated systems to handle sensitive data, the need for robust error handling and comprehensive testing has never been greater.

The incident ultimately raises a fundamental question: in an era of increasingly complex systems and stringent data protection requirements, can we afford to let any critical code go into production without proper scrutiny?