U, V or W for Incident Recovery

Posted on October 04, 2009

U, V or W

The Economist recently ran a piece describing 3 possible recovery shapes for the global economy.

… But the more interesting question is what shape the recovery will take. The debate centres around three scenarios: “V”, “U” and “W”. A V-shaped recovery would be vigorous, as pent-up demand is unleashed. A U-shaped one would be feebler and flatter. And in a W-shape, growth would return for a few quarters, only to peter out once more.

What struck me was the resemblance these shapes have to the response and recovery phases of real world computer security incidents. The shape says a great deal about the relative maturity of an organisations incident response capabilities.

V

The organisation has a well defined plan for rapidly responding to incidents.

Incident responders are clear on their mission, are empowered, equipped and well practised.

Key operational assets are identified and have well defined recovery procedures that are regularly tested by the IT team.

The defenders maximise their home advantage through building visibility in to help them quickly identify intruder activity and determine the extent of a possible breach.

Emergency contact procedures and numbers are regularly proven.

The network can be reconfigured dynamically to enable only “mission critical” internal and partner traffic during a severity 1 security incident (without bringing down the business).

IT teams are able to prove elements of system, application and data integrity against known, trusted baselines. Where this is not possible, recovery from trusted media is not only possible, but practised.

Discoveries about intruder methods and preferences from system and network forensics directly feed into build standards and monitoring systems. Vulnerability assessment checks are quickly updated and key assets rapidly scanned and remediated as appropriate to close open doors known to be preferred by intruders.

Sharing of threat intelligence with peers in the same vertical is the norm rather than the exception. Secure communication methods are already in place and the limits of such communication clearly defined in advance.

Post-incident procedures include root cause analysis driven by an experienced facilitator who either directly leads the change activity or hands off to someone with sufficient leadership to do so.

This organisation can continue core operations during an incident, can quickly recover and is able to bolster defence and detection rapidly to block or reduce the impact of repeat attacks from known groups.

U

Aspires to be a V organisation but fails to hire sufficiently experienced individuals to rapidly respond in the right direction.

Lacks management conviction and understanding about the true nature of today’s threat.

Fails to properly identify ancillary devices relied upon by key assets thus under-performs when it comes to recovery time.

Intelligence from forensics is partial and not fully acted upon. Intruders persistence mechanisms are discovered piecemeal and co-ordination between defender groups is only partially effective.

Some information sharing with peer organisations happens but its effectiveness is muted by managements teething concerns around trust and competition.

This organisation recovers from the attack but the process is painful, tiring and more expensive than it needs to be. The impact of the breach is bigger and probably not fully quantifiable.

The CISO survives by the skin of his teeth. His best people will leave if they don’t sense real appetite for change. They won’t want to go through that experience again.

W

This type of organisation hires in the IR talent on demand though outside contractors.
These hired hands are true experts – experienced, sharp and responsive.

The problem is they can’t possibly understand the business context. Nor can they wrap their arms around a large enterprise network in a highly compressed time-frame. They are handicapped by the lack of rapport they have with the various interest groups across the organisation, relying more on the power vested in them by senior management.

They do bring impressive tools and techniques to bear and impress some of “the locals”.

They work 24×7 to contain the intruders but are hampered by a lack of nimbleness in other teams they rely on to make changes to infrastructure. They foresee this (they are not new to this game) and circumvent where they can.

They stop the bleeding.

The patient – in the form of senior management – “feels better” again. Then management start weighing up the costs of having them around.

The consulting firm is kept on retainer but the star players are moved on to the next client site to fight the next fire.

But the intruders come back: better tooled up, more resourced and significantly better informed about the targets internal network.

The organisations management suddenly finds itself back on the incident response roller coaster heading downhill fast.

The consulting all-stars will be brought back on-site as soon as they can be freed up and will eventually succeed in regaining control. But the recovery costs are at least double, the breach costs are likely more than double (the intruders had time to optimise their data extrusion methods) and the organisation still doesn’t have a sustainable strategy.

But, they may stand the best chance of bypassing the U shape completely as they’ve now been burnt twice and its cost them considerably. They also know what the right talent looks like. They may even poach from the consulting company…

Which leaves just one question: how many V style organisations do you know?