“Fun one for later”: preparing for the incident you hope won’t happen

At around midnight, one of our developers was reading a blog. They forwarded it to our lead developer with a message:

“Fun one for later”

It was our first warning of the PHP security issue that would occupy much of the web industry on 21 October 2026. By the afternoon, organisations around the world were taking websites offline while they worked to protect them.

For Ten4, “fun” meant assessing the risk, making difficult decisions about our clients’ websites, preparing updates and keeping people informed while the situation developed.

Nobody anticipated this particular vulnerability. But we had spent time thinking about how we’d respond to something like it. We had an incident response framework, defined responsibilities and a team that had practised working through scenarios and real issues in the past.

That preparation made a considerable difference.

A plan gives people somewhere to start

Incident response planning is easy to put off. There’s always something more immediate to do, and spending time discussing a hypothetical emergency can feel a little indulgent when there’s actual work waiting.

Until the hypothetical becomes your Wednesday.

At that point, some very ordinary questions become urgent. Who’s coordinating the response? Who’s assessing the technical information? Who can make a decision that affects every client? Who’s contacting those clients? How do the people doing those jobs keep one another informed?

Working through these questions beforehand saves time when you need it most. It also saves attention. People have more capacity to deal with the problem in front of them because they aren’t simultaneously working out how to organise themselves.

For us, the framework provided a familiar starting point. We could move into the response with an understanding of our responsibilities and how our work was going to fit together.

There was still plenty to figure out. A plan can’t tell you everything that will happen, but it does give you a way to deal with the unexpected.

Every incident improves the plan

We ran a few war games several years ago to test how we’d respond under pressure. They helped shape our approach, but the framework has continued to develop through experience.

From the first version, our plan has included a review of what worked, what caused confusion and what could be improved. We use those lessons to update the framework, so every incident helps us prepare a little better for the next.

This time, we learned that a little more depth in our CRM, including additional email addresses to CC on urgent updates, would have saved us scrabbling for phone numbers when our main contacts weren’t available.

Some decisions are still uncomfortable

One of the hardest parts of this incident was deciding when to take websites offline.

Our clients rely on their websites to sell tickets, receive donations, share information and deliver services. Taking one offline has consequences. We understand that, and it isn’t a decision we take lightly.

Keeping a potentially vulnerable website online has consequences too. At the time, the National Cyber Security Centre advised organisations to consider taking systems offline where immediate remediation wasn’t feasible.

These are decisions you have to make with the information available at the time. You assess what you know, acknowledge what you don’t, and weigh the risks of taking over a hundred websites offline against the risks of leaving them running.

Technical judgement, an understanding of the client’s business and the authority to act all need to come together. People need room to challenge an assumption, and clarity about who will make the decision.

Our framework doesn’t make those decisions comfortable, but it does give us a way to reach them, explain them and act on them.

Pulling together takes coordination

Someone needs to follow the developing guidance. Someone needs to prepare and apply updates. Clients need explanations — technical enough to be meaningful to IT teams, human enough to be meaningful to the CEO.

Those activities depend on one another. A client update needs to reflect the technical position. The technical team needs to understand the impact on clients. Everyone needs enough shared context to do their job without interrupting everyone else constantly.

Our framework sets out how to keep client communications, the technical response and other key discussions in dedicated Slack threads. People can follow each topic in order, without conversations overlapping. It seems obvious, but a little communication hygiene goes a long way, especially when someone joins midway through the response and needs to catch up.

There’s a bigger community behind all of this

Once we knew how serious this particular issue was, we made sure to contact former colleagues to check that their agencies knew about it. On a day like that, the useful information is worth passing on.

With an even wider perspective, it made me stop and think about who actually maintains the technologies we depend on so heavily. There’s a much-shared XKCD cartoon showing an enormous stack of digital infrastructure resting on “a project some random person in Nebraska has been thanklessly maintaining since 2003”. It’s funny until you find yourself wondering which bit you’re standing on.