On the global IT outage

(reposting from my facebook)

My thoughts on the global IT outage. I'll try and keep this layman-friendly.

For decades now, we've been adding IT to our key infrastructure, everything from railways to hospitals to emergency call lines, all the things that make life run properly. This isn't a bad thing - IT makes it possible to manage these large systems, to get information updates and act on them. Most of the time, planes fly on time and fire engines get to the right house because of the IT coordinating them. It's there for good reason.

But all that IT is a weakness. In April this year, hackers broke into a computer managing the water supply for Tipton, Indiana with the intention of poisoning 5000 people. This was far from the first such event - in fact it keeps happening. Supply chain attacks, social engineering, the list goes on. We live on shifting sand.

Today's failure shows that we don't even need attackers to break our infrastructure; we can do it with a careless update. CrowdStrike exists because Windows isn't secure on its own, but security software like that has to have the power to administer - or break - every computer it's installed on.

It's easy to blame the administrators of any given installation for not following proper security practice, but that's missing the bigger picture. Getting IT security right is really hard, the people who actually understand it are few (and expensive), there's basically no government guidance on how you're supposed to do it, and it can all be undone by one idiot who thinks the rules don't apply to them. Even if you do everything right, there are still countless ways security could be broken.

And it isn't even that there's a better alternative. Linux may be more secure than Windows in some ways, but it has plenty of its own issues. Same with Macs, and any other operating system you care to name. And when everything from the CPU to the web browser is vulnerable... there just isn't a right answer.

There's a Hollywood-fueled illusion of a "proper IT system" with perfect security, one where everything is done right, where the admins are on top of everything, where professionals did the job right, and that's utterly impossible to hack. Whenever we see a headline about a company gets hacked, we criticise them for not doing it right. And that's bullshit. Because a secure system like that does not exist.

Not yet.

Here's the thing. It IS physically possible to build an IT system that's actually secure. The tools and technology to create it do exist. But it will take a superhuman effort of generations for us to actually do it. Every step of the chain will need to be hardened: the CPU, firmware, hypervisor, kernel, drivers, virtual machines, containers, browsers, routers... it's a long list. And if you miss out any one of them, then the whole thing is weaker.

But it's not in any one company's interest to spend the time and money needed to do any of that. Which is why we haven't started. It can't be a private effort. Governments need to make it happen.

The security agencies of the world - the NSA, GCHQ, and their peers - could have spent their massive budgets over the last 30 years cooperating to create a platform for secure IT. This platform could be made available to anyone running key infrastructure - airlines, power plants, oil refineries, water systems, railways, ports, tax portals, emergency call lines, hospitals, police, all of them. They could have held accredited courses for the IT administrators of all these different industries on how to use it right. They could have secured the civilised world against broken infrastructure, regardless of whether the threat is from hackers or mistakes.

Instead those agencies chose to weaken our collective security by building (and inevitably leaking) a lot of hacking tools because they wanted to intercept everyone's private messages and track dangrous people. They were more interested in digital attack than digital defence.

The West has the most to lose by failing to secure our infrastructure. If the effort starts now, within our lifetimes problems like this could be a thing of the past. But it needs governments to have a vision that lasts beyond the next election cycle, beyond the next headline, and beyond their partisan interest.