The Crowdstrike, Windows Outage 19th July 2024

Incident Summary:

On July 19, 2024, at 04:09 UTC, CrowdStrike deployed a sensor configuration update for Windows-based systems as part of its routine maintenance and operational enhancements within the Falcon platform. This update was intended to optimize performance and improve security monitoring capabilities. However, an unforeseen issue arose due to a logic error within the update, which inadvertently caused critical system failures. As a result, affected machines experienced unexpected crashes, leading to the appearance of the well-known blue screen of death (BSOD), disrupting normal operations for many users.

Upon identifying the problem, CrowdStrike's engineering and support teams immediately began investigating the root cause of the crashes. Through internal analysis and troubleshooting, they determined that the issue was directly linked to the configuration update. In response, the company worked rapidly to develop and implement a corrective measure to mitigate the impact on affected systems. By 05:27 UTC on the same day, CrowdStrike successfully resolved the issue by rolling back the faulty update and deploying a fix.

It is important to note that this incident was not caused by any external threat, cyberattack, or malicious interference. Rather, it was the result of an unintended software configuration error within the update process.

Impact:

Customers utilizing the Falcon sensor for Windows, specifically those running version 7.11 and above, who had active connections during the time window between 04:09 UTC and 05:27 UTC on July 19, 2024, may have experienced disruptions in the form of sudden system crashes. These crashes were directly triggered by the faulty sensor configuration update, leading to potential downtime and operational interruptions.

Users who were offline during this period or those running older versions of the Falcon sensor were unaffected by the issue. CrowdStrike has since taken steps to ensure that future updates undergo additional validation and testing to prevent similar occurrences.

Technical Details:

The faulty update targeted malicious named pipes used in cyberattacks but inadvertently triggered an operating system crash due to a logic error in Channel File 291, located in C:\Windows\System32\drivers\CrowdStrike\.

CrowdStrike has corrected the logic error in Channel File 291 and continues to protect against misuse of named pipes.

Remediation:

Further details on remediation steps are available on CrowdStrike's blog and Support Portal. Customers with specific support needs are encouraged to contact CrowdStrike directly.

Root Cause Analysis:

CrowdStrike is conducting a comprehensive root cause analysis to identify and address the underlying factors that led to this issue, aiming to enhance operational processes and prevent similar incidents in the future.

Note: Systems running Linux or macOS were not affected by this issue as they do not utilize Channel File 291.