Key takeaways

The paper discusses the apparent paradox of “plan and conform” (aka centralised control) and “plan and revise” (aka guided adaptability) perspectives in safety management. As the paper discusses, this paradox is only apparent as both are needed in conjunction to navigate an ever faster changing world full of brittleness.

Detailed discussion

The paper starts of with an introduction to failure as a result of brittle systems:

“Failure is due to brittle systems, not erratic components, subsystems, or human beings.” (Woods, 2023, p. 2)

Which sets the theme for failure (and success for that matter) being an emergent property of a system that can’t be explained by looking at the individual components alone. This is a very common view on failure in complex systems (see e.g. “Engineering a safer world”) and very important as well later on when discussing the limits of the “plan and conform” perspective.

Brittleness as a core emergent property of systems is subsequently defined as:

“Descriptively, brittleness is how rapidly a system’s performance declines when it nears and reaches its boundary.” (Woods, 2023, p. 3)

Following from the definition of brittleness the central challenge of operating highly complex systems with emergent behaviour is then stated as:

“Because competence envelopes are bounded, a core question for all systems is—how does the system perform when events push it near or beyond the edge of its envelope?” (Woods, 2023, p. 3)

And this is very interesting I think because it’s not just the core question for all systems from an engineering and design standpoint. But also the core question for safety management. With the general task of safety management being to keep the system within its operating boundaries (setting aside the problem of knowing what those boundaries are) it is also core to safety management.

Next up the apparent paradox “Command-Adapt” is described in a contrasting description of upper-echelons view of the work and units-of-action view of the work (essentially the blunt-end/sharp-end contrasted view on work). And with that view of the work also comes a view on incidents and failures. For the blunt end:

“Incidents and failures generally are diagnosed as failures of operational personnel to work-to-rule/role/plan which then leads to new pressures to conform. This is the systems architecture that underlies an emphasis on rule compliance in safety management.” (Woods, 2023, p. 4)

And for the sharp end as the other side of the apparent paradox:

“The central theme of the guided adaptability perspective is “plan and revise”—being poised to adapt. This perspective recognizes that disrupting events will challenge plans-in-progress, requiring adaptations, reprioritization, and reconfiguration in order to meet key goals given the effects of disturbances and changes.” (Woods, 2023, p. 4)

And with these two perspectives (“follow the plan as it describes the safe boundaries of system operations” vs “things aren’t gonna go as planned and we need to change things around”) defined, the apparent paradox that you have to choose one or the other for how safety management is done is outlined. And then immediately uncovered as a wrong contrasting of approaches:

“Empirical studies, experience, and science all reveal that the paradox is only apparent: “good” systems embedded in this universe need to plan and revise—to do both. And the necessity of both is evident in the need to manage the risk of brittleness while coping with the side effects of growth and change” (Woods, 2023, p. 5)

And to me this is really the core of the paper. That this strict splitting up of two perspectives on safety management that we often see (plan-and-follow vs adapt-and-improvise, blunt-end vs sharp-end, Safety I vs. Safety II, etc) is largely the wrong question. Because we need to understand that both have their place and there needs to be an understanding of the trade-offs of when one is being employed rather than the other. And the problems arise when this isn’t recognised:

“The paradox dissolves, in part, when one realizes guided adaptability depends in part on plans. The difficulty arises when organizations over-rely on plans [7]. Over-reliance undermines adaptive capacity when beyond-plan challenges arise. Beyond-plan challenges occur regularly for complex systems. The catch is: pressure to comply focuses only on the first and degrades the second.” (Woods, 2023, p. 5)

In order to underline the fact that plans eventually fall short 2 classical assumptions about plans are discussed:

1. Plans can completely specify actions
2. Rationalisations about why findings about shortcomings of plans only apply to other areas and not one’s own

From the belief that the first assumption is true, it is usually derived why work-to-rule should be a guiding principle of safety management:

“If plans can fully specify actions, or nearly so, then work-to-rule/role/ plan is sufficient for productive and safe systems.” (Woods, 2023, p. 5)

This is the very common and alluring perspective that work is much more algorithmic rather than heuristic. This assumption that it’s possible to fully specify work also underlies for example the frequent over-eager and over-optimistic assumption of how much work can be (easily) automated (e.g. by a script or so-called “AI”).

Related to that assumption (and illusion) of control then rationalisations are produced of why one is in a special case and the ample findings of short comings of plans don’t apply.

“The usual response from organizations to these classic findings is simple: my world is stable and not like space operations, military operations, and emergency or critical care medicine. In my world variability can be blocked or suppressed, minimizing the need for adaptation since work-to-plan/role/rule will reliably produce desired outcomes.” (Woods, 2023, p. 7)

This rationalisation according to the paper is based on several erroneous assumptions:

• Surprises occur rarely
• It’s easy to know when a plan needs to be modified
• It’s quick to put modified plans in action
• Interdependencies are easy to limit and be analysed and modelled a-priori
• Effects of surprise can be easily compartmentalised and contained away from interdependencies

Some of these might be true for a moment but aren’t overall true throughout operation and especially lifecycle (design, growth, adaptations) of a system. And while these assumptions have been shown to be true and rediscovered over and over through research as well as experience they still serve as a kind of feedback loop to statement 1 and the call for compliance.

In order then to reconcile the two apparent-paradoxical perspectives is a reconceptualisation of plans through the lens of adaptability through 4 parts:

1. Plans are resources for action
2. Plans are necessary to recognise anomalies
3. Plans (and Automata) are competent but brittle
4. People (with the right help) provide the extra adaptive capacity to mitigate brittleness

This is to recognise that plans are useful as a starting off point and a resource to grab from when action is needed, but not as a strict specification. And one very important point is that improvisation and adaptability at the sharp-end requires the pre-requisite step of detecting anomalies and deviations. This is a lot harder without a baseline of “normal” which plans can provide. In terms of automation it needs to be recognised that automation (which includes various forms of AI in their respective hype cycles as well) is competent but brittle. And that there is a persistent believe that it just needs a new push of the technology to overcome this.

“Studies looking at joint systems of people and AI or operators and advanced automation revealed the fundamental brittleness of automata regardless of the underlying technology [13].” (Woods, 2023, p. 8)

And getting to the people part (which is the source of adaptive capacity) it’s very important to recognise that challenges and near-misses happen much more often than expected. And that created control systems like automation are subject to the same pressures as the systems they are supposed to control. And thus the same risk for brittleness:

“All systems are developed and operate with finite resources and live in a changing environment. As a result, plans, procedures, automation, agents, and roles are inherently limited and unable to completely cover the complexity of activities, events, and demands.” (Woods, 2023, p. 9)

And so the paper concludes on a way to perform work and safety management which is dubbed “Plan and revise: Guided Adaptability”. Which still means that there should be plans for the work to be done that are intended to be followed until it doesn’t make sense any more. The complement then is to learn from how they don’t make sense anymore and include that in the revision of plans for the future based on the best source there is for adaptations: humans.

“The irony is you can only monitor how well plans fit the world by understanding how people have to adapt to fill the gaps and holes that inevitably arise as variability in the world exceeds the capability of plans and the competencies built into any system [12].” (Woods, 2023, p. 11)

And to make it clear that adaptation itself is also subject to adaptation and not some perfect state of behaviour, the following quote towards the end of the paper is extremely apt in my opinion:

“You will have to establish the continuous feedback/learning loop in order to adapt how you adapt.” (Woods, 2023, p. 13)

Personal thoughts

I really liked the paper and the way it made the trade-offs of both perspectives on safety management very clear. It’s way too often the case in my opinion that a silver bullet solution is sought and once something is assumed as such, all the rest gets discarded. When in reality trade-offs and “best of both worlds” approaches to real world problems are much more likely to yield much better results. The different levels of views on the work in terms of high level planning and low level implementation also reminded me a lot of the waterfall vs agile and scheduled release vs continuous deployment discussions that are happening in technology where often one is seen as the superior approach over the other. But in reality even agile processes need longer term planning to fit into the bigger picture. And even continuous deployment means you are able to deploy whenever not that you have to. And sometimes planning a deploy to match the larger circumstances (outside of another teams test, maybe not on a Friday 😬, or even it can wait till the next morning) makes much more sense rather than deploying something as soon as you got the code review approved. As the famous mature engineering proverb goes

It’s trade-offs all the way down