close
close

Falcon Content Update Fix and Guide Hub

Page last updated 2024-07-22 0238 UTC

Updated 2024-07-21 2106 UTC

As indicated in our social media post on 7/21/2024 2106 UTC, CrowdStrike has been testing a new technique with customers to expedite remediation of affected systems. We are in the process of implementing an opt-in option for this technique. Customers are encouraged to monitor Tech Alerts for the latest updates as they become available. You will be notified when action is required.

We will continue to provide updates here as information becomes available and new fixes are deployed.

CrowdStrike is actively supporting customers affected by a defect in a recent content update for Windows hosts. Mac and Linux hosts were not affected. The issue has been identified and isolated, and a fix has been deployed. This was not a cyberattack.

Customers are encouraged to check the Support Portal for updates. We will continue to provide the latest information here and on our blog as it becomes available. We encourage organizations to verify that they are communicating with CrowdStrike representatives through official channels.

We assure our customers that CrowdStrike is operating normally and this issue does not impact our Falcon platform systems. If your systems are operating normally, installing the Falcon sensor will not impact your protection.

We recognize the severity of the situation and deeply apologize for any inconvenience and disruption. Our team is fully mobilized to ensure the safety and stability of CrowdStrike customers.

overview

Statement from our CEO

Sent on 19.07.2024 1930 UTC

Dear customers and partners,

I would like to sincerely apologize to all of you for the outage. CrowdStrike recognizes the severity and impact of the situation. We quickly identified the issue and deployed a resolution so we could focus on restoring customer systems with the highest priority.

The outage was caused by a defect in a Falcon content update for Windows hosts. Mac and Linux hosts are not affected. This was not a cyberattack.

We are working closely with affected customers and partners to ensure all systems are restored so you can deliver the services your customers rely on.

CrowdStrike is running normally and this issue does not impact our Falcon platform systems. Protection is not impacted when the Falcon Sensor is installed. Falcon Complete and Falcon OverWatch services are not disrupted.

We provide ongoing updates through our support portal at https://supportportal.crowdstrike.com/s/login/.

We have fully mobilized CrowdStrike to help you and your teams. If you have any questions or need additional assistance, please contact your CrowdStrike representative or technical support.

We know that adversaries and malicious actors will attempt to exploit these types of events. I encourage everyone to remain vigilant and ensure you are engaging with official CrowdStrike representatives. Our blog and technical support remain the official channels for the latest updates.

Nothing is more important to me than the trust our customers and partners place in CrowdStrike. As we resolve this incident, I am committed to providing you with complete transparency about how this happened and the steps we are taking to prevent this from happening again.

George Kurtz

Founder and CEO of CrowdStrike

Technical details

  • Technical details about the outage can be found here: Read blog Published 2024-07-19 0100 UTC
  • We assure our customers that CrowdStrike is operating normally and this issue does not impact our Falcon platform systems.If your systems are functioning normally, your protection will not be affected with the Falcon Sensor installed. Falcon Complete and OverWatch services will not be interrupted by this incident.
  • CrowdStrike has identified the trigger for this issue as the deployment of Windows Sensor-related content and we have reverted these changes. The content is a channel file located in the %WINDIR%\System32\drivers\CrowdStrike directory.
    • The channel file “C-00000291*.sys” with timestamp 2024-07-19 0527 UTC or later is the reverted (good) version.
    • The channel file “C-00000291*.sys” with the timestamp 2024-07-19 0409 UTC is the problematic version.
    • Note: It is normal for there to be multiple “C-00000291*.sys” files in the CrowdStrike directory – as long as one of the files in the folder have a timestamp of 05:27 UTC or later, which is the active content.
  • Symptoms include hosts experiencing a bugcheck/bluescreen error related to the Falcon Sensor.
  • Windows hosts with not affected, no action is required as the problematic channel file has been restored.

Unaffected hosts

  • Windows hosts brought online after July 19, 2024 0527 UTC are not affected.
  • Windows hosts installed and deployed after July 19, 2024 0527 UTC are not affected. Updated 2024-07-21 1435 UTC
  • This issue does not affect Mac or Linux based hosts

How do I identify affected hosts?

How do I identify affected hosts using the advanced event search query?
Updated 2024-07-22 0139 UTC

The queries used by the dashboards are listed in the corresponding dashboard KB articles below.

How do I identify affected hosts via the dashboard?
Updated 2024-07-22 0139 UTC

An updated detailed dashboard is available that shows the Windows hosts affected by the content update defect described in this Tech Alert. To identify Windows hosts affected by a content issue (v8.6), see Definitive Status Dashboards. Note that the queries used by the dashboards are listed at the end of the corresponding dashboard KB articles.

If hosts continue to crash and cannot stay online to receive the channel file update, you can use the following steps to resolve the issue.

How do I troubleshoot individual hosts?
Updated 2024-07-21 0932 UTC

  • Restart the host to give it a chance to download the restored channel file. We strongly recommend putting the host on a wired network (rather than WiFi) before rebooting, as Ethernet allows the host to connect to the Internet much faster.
  • If the host crashes again on reboot:
    • Option 1 – Manual
      • You can find the detailed steps in this Microsoft article.
        • Note: Hosts encrypted with Bitlocker may require a recovery key.
      • Option 2 – Automated via bootable USB stick

How can I recover Bitlocker keys?
Updated 2024-07-21 1810 UTC

How to restore resources in a cloud-based environment

Cloud environment Orientation aid

AWS

AWS Articles

Azure blue

Microsoft article

GCP

(PDF) or log in to view it in the Support Portal

Public Cloud/Virtual Environments

Option 1:

  • ​​​​​​​Detach the operating system disk volume from the affected virtual server.
  • Before proceeding, create a snapshot or backup of the disk volume to avoid unintentional changes.
  • Connect/mount the volume to a new virtual server
  • Navigate to the directory %WINDIR%\System32\drivers\CrowdStrike
  • Find the files named “C-00000291*.sys” and delete them
  • Disconnect the volume from the new virtual server
  • Reattach the fixed volume to the affected virtual server.

Option 2:

  • ​​​​​​​​Back to a snapshot before 2024-07-19 0409 UTC

Information about third parties
Updated 2024-07-20 2259 UTC

Additional resources