Blog
Monthly Research
& Market Commentary


Black Swans over Hounslow

Value Centric Leadership / 09 Jun 2017 / By Glen Robinson, Bill Murray

After thinking hard about antifragility recently, we at LEF were interested to observe the recent BA incident, where computer system outages grounded hundreds of flights.  BA is facing a large compensation bill, with some estimates putting it at up to £150m.  The problem affected not only operations but customer-facing parts of the business; they weren't able to communicate with customers in their usual ways. 

Normally, when a data centre suffers power loss the UPS – Uninterruptible Power Supply – kicks in and supplies power.  Apparently, the UPS itself created a power surge when it started the recovery, damaging data centre hardware.  Ironically, a backup system made things worse.  The data centre was fragile.

Systems and processes – fragile by design

Even when data centres recover, systems may not operate properly.  For example: 

  • Specialists who can recover the company’s complex systems have left – for example, when the IT operation was outsourced.
  • The original system is being restored at the same time as a back-up system is being started.
  • The systems are unusable because the data has become unsynchronized.

Glen Robinson says that in the 21st century, disaster recovery is a difficult subject for the board to deal with. Everyone knows it’s needed, but are unwilling to invest what’s needed to ensure a seamless experience for their customers.

The British Airways IT failure is a function of the complex systems and processes which airlines need to use.

And.  Complexity.  Creates.  Fragility.

IT Fragility also comes from other sources: 

  • Cost cutting – the common mistake is that cost cutting costs nothing; it can ‘engineer fragility’ in to business processes and IT systems.  Little slack, no contingency.  Airlines probably invest more in IT than many other organizations apart from banks, so IT is a target for cost reduction.
  • Automation – routine human tasks in the IT organization are being replaced by intelligent software.  Unfortunately, this software is often still learning, and may not cope with surprises.
  • Cyber risk – new systems integrating to older systems creates new attack surfaces and attack vectors for bad actors to create mayhem; ask Wonga, Three mobile, Sports Direct and Tesco Bank for starters.

Note that the lesson from incidents such as this one is not that we should avoid cutting cost, automating or innovating.  The learning is that we should make sure that we are cognizant of the implications for the fragility of our business, and factor in measures to increase antifragility as we pursue these forms of value creation.

Remove fragility – build antifragility

Bill Murray and Dave Aron explain the stages to antifragility.  A fragile system quickly deteriorates.  A robust system holds out longer but ultimately fails.  A resilient system suffers but does not fail, and ultimately returns to normal. An antifragile system adjusts and ultimately comes back stronger.

Companies can stop creating fragile systems and processes, start building robustness and then build resilience: 

  • Stop engineering fragility in to systems – for example, replacing specialists with generalists.
  • Create contingency – for example, move to SAAS/IAAS with smart scale and pricing options, arrange alternative suppliers who can hot swap in, put systems specialists on retainers.
  • Create redundancy – for example, a multi-cloud strategy which helps to avoid failure.
  • Add architectural resilience – for example, move to ‘design for failure’ approaches.
  • Distribute more – for example, Ryanair has not two but three backup systems for IT, in Dublin, Poland and Madrid.

Glen Robinson says “If you’re thinking of continuing with a more traditional DR strategy, then I’d seriously recommend you think twice.  Doing it on the cheap really isn’t an option, you’re still carrying a huge amount of risk. Consider what you’re fully loaded DR costs would look like were you to do it properly, then look again at the alternatives.  I think you’ll find them more valuable than you imagined”.

There will be a price for introducing these disciplines, call it an option price – the price paid for avoiding catastrophes and having options in disasters.  But it is a small price to pay compared to losses in the tens/hundreds of millions from sustained global systems outages.

Become a master of disaster

Companies can go further and restructure to take advantage of Volatile, Uncertain, Complex, Ambiguous (VUCA) environments.  Leading digital firms like Netflix are becoming ‘antifragile’ – they become stronger under stress. Becoming antifragile is not just about increasing robustness or resilience in the face of expected shocks; it is about systematically stressing the organization, and learning and restructuring to take advantage of unexpected shocks. Antifragile companies continually challenge critical systems and processes so that they learn to heal, by employing internal ‘masters of disaster’ or ‘chaos monkeys’.  For example, an antifragile competitor of BA could have quickly deployed extra capacity in systems, processes and people to absorb and retain stranded BA passengers.  Antifragile companies, like Toyota, view this type of investment as ‘Intelligent Risk Taking’.

The way we think about disaster recovery isn’t serving businesses and government organizations well enough in the 21st century.  There are smarter, cloud centric approaches.  And look again at your business model, processes and systems through the lens of antifragility.  Start by stopping.  Stop building fragility into your business.

JOIN THE CONVERSATION

*{{ error }}
*{{ error }}
*{{ error }}
captcha
*{{ error }}
*{{ error }}
*{{ error }}

CATEGORIES

21st Century
Adaptive Execution
Assets/Capabilities
Identity/Strategy
Proactive, Haptic Sensing
Reimagining the Portfolio
Value Centric Leadership

ALSO IN THIS CATEGORY

Have You Got the Digital Brownfield Blues?
14 Nov 2017 / By Dave Aron
WT* is a Platform?
25 Oct 2017 / By Dave Aron, Bill Murray
WT* is a Digital Transformation?
10 Oct 2017 / By Dave Aron
IoT is Turning the World into a Soft Machine
25 Aug 2017 / By Dave Aron
So Much is Becoming Possible with ‘Matrix’ Technologies – How Will Your Organization Take Advantage?
23 Aug 2017 / By Glen Robinson