Microsoft Outage
- Thread starter glamphotographer
- Start date
Sounds like they did all the usual testing and qos, but someone made a small (but fatal) change to a file, didn't re-test and then was included in the package that was pushed to all 24,000+ companies that run the CrowdStrike Falcon product across their collective (perhaps millions in total?) computers.
The fix for server farms was fairly simple thankfully but tedious and had to be done manually (at first) one virtual server at a time. I helped a client out as extra hands and got to see the process first hand.
Bare metal servers and all the desktops/laptops out there? That's the biggest headache of them all. Dealing with staff in remote offices, walking them through going into safe mode, granting them elevated access to delete the corrupt file, without any screen sharing options - at best using video calls to see their screen as they go - total nightmare.
Statement from CrowdStrike CEO and instructions on how to fix the issue.
www.crowdstrike.com
There also is the additional complication of anyone or any company running Bitlocker on their servers/workstations.
Falcon Content Update Remediation and Guidance Hub | CrowdStrike
Access consolidated remediation and guidance resources for the CrowdStrike Falcon content update affecting Windows hosts.
www.crowdstrike.com
There also is the additional complication of anyone or any company running Bitlocker on their servers/workstations.
Outage affected 8.5 million Windows devices world wide, according to Microsoft 
Our PCs onsite were fixed quickly by our IT crew. Less tgan 5 minutes. But we have many people working remotely. Conference calls through video chat on their cell phones to walk them through it. The other option was the constant restarting of their PC hoping it would eventually start up.
