Crowdstrike/Microsoft outage summary

On July 19th, 2024, multiple organizations across the globe were impacted by a defective content update from the CrowdStrike antivirus solution. The defective update only impacted Windows hosts. CrowdStrike has already mitigated this issue by reverting the changes made in the defective content update and deploying a fix. 

It is important to note that this issue is not linked to a cyberattack or a security incident.

Root Cause

The content update caused Windows hosts to experience a blue screen error related to CrowdStrike's Falcon Sensor. This caused the hosts to not be able to boot into the Windows operating system.

Workarounds

Syndis has compiled the following workarounds for those who are having trouble with booting up Windows hosts affected by this issue:

For individual hosts:

  • Reboot the host to give it an opportunity to download the reverted channel file. If the host crashes again, then:
  • Boot Windows into Safe Mode or the Windows Recovery Environment

Note: Putting the host on a wired network (as opposed to WiFi) and using Safe Mode with Networking can help remediation.

  • Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
  • Locate the file matching “C-00000291*.sys”, and delete it.
  • Boot the host normally.

Note: Bitlocker-encrypted hosts may require a recovery key.

For cloud environments and virtual machines:
Option 1:

  • Detach the operating system disk volume from the impacted virtual server
  • Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes
  • Attach/mount the volume to a new virtual server
  • Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
  • Locate the file matching “C-00000291*.sys”, and delete it.
  • Detach the volume from the new virtual server
  • Reattach the fixed volume to the impacted virtual server

Option 2:

  • Roll back to a snapshot before 18.07.2024 04:09 UTC.