close
close

Torres calls for full Department of Homeland Security investigation into Crowdstrike outage

By SÍLE MOLONEY

SECURITY MEMBERS refill TD Bank ATMs on the Grand Concourse in Fordham Heights on Friday, May 5, 2023.
Photo by Sile Moloney

Congressman Ritchie Torres (NY-15) has called on the Department of Homeland Security to investigate the recent Crowdstrike outage, which he said had mixed consequences.

In a letter to HSI on Friday, July 19, the congressman wrote, in part: “Crowdstrike, an internationally recognized U.S. cybersecurity software company, was hit by a massive global ‘outage’ that prevented people from withdrawing money from ATMs, grounded flights, and disrupted healthcare.”

The congressman, who represents a large portion of the Bronx stretching from the Northwest to the South Bronx, continued: “According to Crowdstrike’s president and CEO, the exact cause of the outage ‘…is not a security or cyberattack. The issue has been identified, isolated and fixed,’ but is due to a software update.”

Torres added, “Even though the cause of the outage has been determined, the sheer number of systems down raises concerns that the cyber vulnerabilities of our critical infrastructure systems are coming to a halt due to a software update. At a time when cyberattacks are increasing in both their scope and sophistication, modernizing our local, state and federal cybersecurity systems is paramount. And ensuring they not only accept the software update, but also function afterward is the bare minimum.”

The letter continues: “I am therefore writing to request that the Department of Homeland Security (DHS), in collaboration with the Critical and Infrastructure Security Agency (CISA) and the Cyber ​​Safety Review Board (CSRB), conduct a joint investigation into this software update flaw and its impact on the cyber vulnerabilities of all systems, such as NOTAM and ATC, that underpin air traffic.”

The conclusion was: “Because CISA provides operational leadership for federal cybersecurity and DHS, along with other federal agencies – such as DOT, HHS, and GSA – is the Sector Risk Management Agency (SRMA) responsible for their respective sectors, it is incumbent on DHS to coordinate with these federal agencies to determine what actions can be taken and what investments can be made to protect our critical infrastructure systems from threat actors who seek to do us harm.

On Friday, following the outage, George Kurtz, president and CEO of CrowdStrike, former CEO of Foundstone, former CTO of McAfee and author of “Hacking Exposed,” announced that the company was actively working with customers affected by “a defect in a single content update for Windows hosts.”

The statement continued: “Mac and Linux hosts are not affected. This is not a security incident or cyberattack. The issue has been identified, isolated, and a fix deployed. We direct customers to the support portal for the latest updates and will continue to provide full and continuous updates on our website. We also recommend that organizations ensure they communicate with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers.”

In another update from Kurtz on Friday, he said: “There was no security or cyber incident today. Our customers remain fully protected. We recognize the seriousness of the situation and deeply apologize for any inconvenience and disruption. We are working with all affected customers to ensure systems are back up and running so they can provide the services their customers count on.”

He promised to release more updates as they become available, adding, among other things: “As previously mentioned, the issue has been identified and a fix has been deployed. There was an issue with a Falcon content update for Windows hosts.”

Kurtz later continued, “CrowdStrike continues to work closely with affected customers and partners to ensure all systems are restored. I am sharing the letter I sent to CrowdStrike customers and partners. Once this incident is resolved, I am committed to providing full transparency about how this occurred and the steps we are taking to prevent this from happening again. We are working on a technical update and root cause analysis, which we will also share with everyone.”

In addition, the following technical update was provided:

What happened?

On July 19, 2024 at 04:09 UTC, CrowdStrike released a sensor configuration update for Windows systems as part of its ongoing operations. Sensor configuration updates are an ongoing part of the Falcon platform’s protection mechanisms. This configuration update triggered a logic error that resulted in a system crash and a blue screen of death (BSOD) on affected systems.

The sensor configuration update that caused the system crash was fixed on Friday, July 19, 2024, 05:27 UTC.

This issue is neither the result of nor related to a cyberattack.

Effects

Customers using the Falcon sensor for Windows version 7.11 and later who were online between Friday, July 19, 2024, 04:09 UTC and Friday, July 19, 2024, 05:27 UTC may be affected.

Systems running the Falcon sensor for Windows 7.11 and later that downloaded the updated configuration between 04:09 UTC and 05:27 UTC may experience a system crash.

Introduction to the configuration file

The configuration files mentioned above are called “channel files” and are part of the behavior-based protection mechanisms used by the Falcon sensor. Updates to the channel files are a normal part of sensor operation and occur multiple times per day in response to new tactics, techniques, and procedures discovered by CrowdStrike. This is not a new process; the architecture has been in place since the inception of Falcon.

Technical details

On Windows systems, the channel files are located in the following directory:

C:\Windows\System32\drivers\CrowdStrike\

and have a file name that begins with “.C-”. Each channel file is assigned a number as a unique identifier. The channel file in question in this case is 291 and has a file name that begins with “.C-00000291-“ and ends with a .sys Although channel files end with the SYS extension, they are not kernel drivers.

Channel file 291 controls how Falcon evaluates named pipes1 Running on Windows systems. Named pipes are used for normal, inter-process or inter-system communication in Windows.

The update, which occurred at 04:09 UTC, targeted newly discovered malicious named pipes used by popular C2 frameworks in cyberattacks. The configuration update triggered a logic error that caused the operating system to crash.

Channel file 291

CrowdStrike has fixed the logic error by updating the content in channel file 291. No further changes will be made to channel file 291 beyond the updated logic. Falcon will continue to check and protect against named pipe abuse.

This has nothing to do with the null bytes contained in channel file 291 or any other channel file.

Renovation

For the latest recommendations and information on how to resolve these issues, please visit our blog or support portal.

We understand that some customers may have specific support needs and ask that they contact us directly.

Systems that are not currently affected will continue to function as expected, continue to provide protection, and there is no risk of this event occurring again in the future.

Systems running Linux or macOS do not use Channel File 291 and were not affected.

LETTER TO the Department of Homeland Security from Congressman Ritchie Torres regarding the July 19, 2024 Crowdstrike outage.
Source: Congressman Ritchie Torres (NY-15)

Root cause analysis

We understand how this issue occurred and are conducting a thorough root cause analysis to determine how this logic error occurred. This effort is ongoing. We are committed to identifying any fundamental or workflow improvements we can make to improve our process. We will update our root cause analysis findings as the investigation progresses.

1 https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipes

Torres’ letter can be read in full above.