Resolving the Command–Adapt Paradox: Guided Adaptability to Cope with Complexity

Key takeaways

The paper discusses the apparent paradox of “plan and conform” (aka centralised control) and “plan and revise” (aka guided adaptability) perspectives in safety management. As the paper discusses, this paradox is only apparent as both are needed in conjunction to navigate an ever faster changing world full of brittleness.

Detailed discussion

The paper starts of with an introduction to failure as a result of brittle systems:

“Failure is due to brittle systems, not erratic components, subsystems, or human beings.” (Woods, 2023, p. 2)

Which sets the theme for failure (and success for that matter) being an emergent property of a system that can’t be explained by looking at the individual components alone. This is a very common view on failure in complex systems (see e.g. “Engineering a safer world”) and very important as well later on when discussing the limits of the “plan and conform” perspective.

Brittleness as a core emergent property of systems is subsequently defined as:

“Descriptively, brittleness is how rapidly a system’s performance declines when it nears and reaches its boundary.” (Woods, 2023, p. 3)

Following from the definition of brittleness the central challenge of operating highly complex systems with emergent behaviour is then stated as:

“Because competence envelopes are bounded, a core question for all systems is—how does the system perform when events push it near or beyond the edge of its envelope?” (Woods, 2023, p. 3)

And this is very interesting I think because it’s not just the core question for all systems from an engineering and design standpoint. But also the core question for safety management. With the general task of safety management being to keep the system within its operating boundaries (setting aside the problem of knowing what those boundaries are) it is also core to safety management.

Next up the apparent paradox “Command-Adapt” is described in a contrasting description of upper-echelons view of the work and units-of-action view of the work (essentially the blunt-end/sharp-end contrasted view on work). And with that view of the work also comes a view on incidents and failures. For the blunt end:

“Incidents and failures generally are diagnosed as failures of operational personnel to work-to-rule/role/plan which then leads to new pressures to conform. This is the systems architecture that underlies an emphasis on rule compliance in safety management.” (Woods, 2023, p. 4)

And for the sharp end as the other side of the apparent paradox:

“The central theme of the guided adaptability perspective is “plan and revise”—being poised to adapt. This perspective recognizes that disrupting events will challenge plans-in-progress, requiring adaptations, reprioritization, and reconfiguration in order to meet key goals given the effects of disturbances and changes.” (Woods, 2023, p. 4)

And with these two perspectives (“follow the plan as it describes the safe boundaries of system operations” vs “things aren’t gonna go as planned and we need to change things around”) defined, the apparent paradox that you have to choose one or the other for how safety management is done is outlined. And then immediately uncovered as a wrong contrasting of approaches:

“Empirical studies, experience, and science all reveal that the paradox is only apparent: “good” systems embedded in this universe need to plan and revise—to do both. And the necessity of both is evident in the need to manage the risk of brittleness while coping with the side effects of growth and change” (Woods, 2023, p. 5)

And to me this is really the core of the paper. That this strict splitting up of two perspectives on safety management that we often see (plan-and-follow vs adapt-and-improvise, blunt-end vs sharp-end, Safety I vs. Safety II, etc) is largely the wrong question. Because we need to understand that both have their place and there needs to be an understanding of the trade-offs of when one is being employed rather than the other. And the problems arise when this isn’t recognised:

“The paradox dissolves, in part, when one realizes guided adaptability depends in part on plans. The difficulty arises when organizations over-rely on plans [7]. Over-reliance undermines adaptive capacity when beyond-plan challenges arise. Beyond-plan challenges occur regularly for complex systems. The catch is: pressure to comply focuses only on the first and degrades the second.” (Woods, 2023, p. 5)

In order to underline the fact that plans eventually fall short 2 classical assumptions about plans are discussed:

1. Plans can completely specify actions
2. Rationalisations about why findings about shortcomings of plans only apply to other areas and not one’s own

From the belief that the first assumption is true, it is usually derived why work-to-rule should be a guiding principle of safety management:

“If plans can fully specify actions, or nearly so, then work-to-rule/role/ plan is sufficient for productive and safe systems.” (Woods, 2023, p. 5)

This is the very common and alluring perspective that work is much more algorithmic rather than heuristic. This assumption that it’s possible to fully specify work also underlies for example the frequent over-eager and over-optimistic assumption of how much work can be (easily) automated (e.g. by a script or so-called “AI”).

Related to that assumption (and illusion) of control then rationalisations are produced of why one is in a special case and the ample findings of short comings of plans don’t apply.

“The usual response from organizations to these classic findings is simple: my world is stable and not like space operations, military operations, and emergency or critical care medicine. In my world variability can be blocked or suppressed, minimizing the need for adaptation since work-to-plan/role/rule will reliably produce desired outcomes.” (Woods, 2023, p. 7)

This rationalisation according to the paper is based on several erroneous assumptions:

• Surprises occur rarely
• It’s easy to know when a plan needs to be modified
• It’s quick to put modified plans in action
• Interdependencies are easy to limit and be analysed and modelled a-priori
• Effects of surprise can be easily compartmentalised and contained away from interdependencies

Some of these might be true for a moment but aren’t overall true throughout operation and especially lifecycle (design, growth, adaptations) of a system. And while these assumptions have been shown to be true and rediscovered over and over through research as well as experience they still serve as a kind of feedback loop to statement 1 and the call for compliance.

In order then to reconcile the two apparent-paradoxical perspectives is a reconceptualisation of plans through the lens of adaptability through 4 parts:

1. Plans are resources for action
2. Plans are necessary to recognise anomalies
3. Plans (and Automata) are competent but brittle
4. People (with the right help) provide the extra adaptive capacity to mitigate brittleness

This is to recognise that plans are useful as a starting off point and a resource to grab from when action is needed, but not as a strict specification. And one very important point is that improvisation and adaptability at the sharp-end requires the pre-requisite step of detecting anomalies and deviations. This is a lot harder without a baseline of “normal” which plans can provide. In terms of automation it needs to be recognised that automation (which includes various forms of AI in their respective hype cycles as well) is competent but brittle. And that there is a persistent believe that it just needs a new push of the technology to overcome this.

“Studies looking at joint systems of people and AI or operators and advanced automation revealed the fundamental brittleness of automata regardless of the underlying technology [13].” (Woods, 2023, p. 8)

And getting to the people part (which is the source of adaptive capacity) it’s very important to recognise that challenges and near-misses happen much more often than expected. And that created control systems like automation are subject to the same pressures as the systems they are supposed to control. And thus the same risk for brittleness:

“All systems are developed and operate with finite resources and live in a changing environment. As a result, plans, procedures, automation, agents, and roles are inherently limited and unable to completely cover the complexity of activities, events, and demands.” (Woods, 2023, p. 9)

And so the paper concludes on a way to perform work and safety management which is dubbed “Plan and revise: Guided Adaptability”. Which still means that there should be plans for the work to be done that are intended to be followed until it doesn’t make sense any more. The complement then is to learn from how they don’t make sense anymore and include that in the revision of plans for the future based on the best source there is for adaptations: humans.

“The irony is you can only monitor how well plans fit the world by understanding how people have to adapt to fill the gaps and holes that inevitably arise as variability in the world exceeds the capability of plans and the competencies built into any system [12].” (Woods, 2023, p. 11)

And to make it clear that adaptation itself is also subject to adaptation and not some perfect state of behaviour, the following quote towards the end of the paper is extremely apt in my opinion:

“You will have to establish the continuous feedback/learning loop in order to adapt how you adapt.” (Woods, 2023, p. 13)

Personal thoughts

I really liked the paper and the way it made the trade-offs of both perspectives on safety management very clear. It’s way too often the case in my opinion that a silver bullet solution is sought and once something is assumed as such, all the rest gets discarded. When in reality trade-offs and “best of both worlds” approaches to real world problems are much more likely to yield much better results. The different levels of views on the work in terms of high level planning and low level implementation also reminded me a lot of the waterfall vs agile and scheduled release vs continuous deployment discussions that are happening in technology where often one is seen as the superior approach over the other. But in reality even agile processes need longer term planning to fit into the bigger picture. And even continuous deployment means you are able to deploy whenever not that you have to. And sometimes planning a deploy to match the larger circumstances (outside of another teams test, maybe not on a Friday 😬, or even it can wait till the next morning) makes much more sense rather than deploying something as soon as you got the code review approved. As the famous mature engineering proverb goes

It’s trade-offs all the way down

Notes

Abstract

“The central theme of the centralized control perspective is “plan and conform”. The central theme of the guided adaptability perspective is “plan and revise”—” (Woods, 2023, p. 2)

“The paradox dissolves, in part, when one realizes guided adaptability is a capability that builds on plans. The difficulty arises when organizations over-rely on plans. Over-reliance undermines adaptive capacity when beyond-plan challenges arise. Beyond-plan challenges occur regularly for complex systems.” (Woods, 2023, p. 2)

8.1 Introduction: Failure is due to Brittle Systems

“Failure is due to brittle systems, not erratic components, subsystems, or human beings.” (Woods, 2023, p. 2)

“Descriptively, brittleness is how rapidly a system’s performance declines when it nears and reaches its boundary.” (Woods, 2023, p. 3)

• Is a system with tight boundaries also considered brittle?

“Because competence envelopes are bounded, a core question for all systems is—how does the system perform when events push it near or beyond the edge of its envelope?” (Woods, 2023, p. 3)

“With the right forms of adaptive capacity, systems have capabilities to anticipate bottlenecks ahead, to synchronize activities across roles and layers for mutual assistance as stress grows, and possess the readiness-to-respond to reconfigure and reprioritize activities to fit the challenges [5].” (Woods, 2023, p. 3)

8.2 The Command-Adapt Paradox

“Incidents and failures generally are diagnosed as failures of operational personnel to work-to-rule/role/plan which then leads to new pressures to conform. This is the systems architecture that underlies an emphasis on rule compliance in safety management.” (Woods, 2023, p. 4)

“The concern is how to keep pace with changing situations to mitigate the risk of brittle collapse.” (Woods, 2023, p. 4)

“From this perspective, safety staff support sharp end roles by putting in place organizational features that allow mutual assistance, or reciprocity, as situations deteriorate in the face of challenges [7].” (Woods, 2023, p. 4)

“The central theme of the guided adaptability perspective is “plan and revise”—being poised to adapt. This perspective recognizes that disrupting events will challenge plans-in-progress, requiring adaptations, reprioritization, and reconfiguration in order to meet key goals given the effects of disturbances and changes.” (Woods, 2023, p. 4)

“Empirical studies, experience, and science all reveal that the paradox is only apparent: “good” systems embedded in this universe need to plan and revise—to do both. And the necessity of both is evident in the need to manage the risk of brittleness while coping with the side effects of growth and change” (Woods, 2023, p. 5)

• This is I think a very crucial part calling out the fact that trade offs and not ultimates are the way to go

“The paradox dissolves, in part, when one realizes guided adaptability depends in part on plans. The difficulty arises when organizations over-rely on plans [7]. Over-reliance undermines adaptive capacity when beyond-plan challenges arise. Beyond-plan challenges occur regularly for complex systems. The catch is: pressure to comply focuses only on the first and degrades the second.” (Woods, 2023, p. 5)

8.3. Classic Findings on the Limits of Plans, Procedures, Automata

8.3.1. Can Plans Completely Specify Actions?

“If plans can fully specify actions, or nearly so, then work-to-rule/role/ plan is sufficient for productive and safe systems.” (Woods, 2023, p. 5)

• (Woods, 2023, p. 5) This is the automation fallacy

“Keeping pace with events invokes skills, forms of cognition, and coordinated activity over multiple roles that cannot be specified in procedures.” (Woods, 2023, p. 5)

• (Woods, 2023, p. 5) Algorithmic vs. Heuristic

“(a) plans will miss the potential for bottlenecks, overload, and oversubscription of key assets and contingency backups (this is the risk of saturation) and (b) plans will always tend to lag change in the real world. And modifying plans will lag the changes already underway” (Woods, 2023, p. 6)

“Hidden interdependencies are a potent source of saturation and lag as problems in one area push saturation to others, diagnostic work has to track effects at a distance from the originating disruption, and an expanding set of roles and players have to coordinate and synchronize their activities, often across organizational boundaries, to resolve losses of valued services [10, 29, 31]” (Woods, 2023, p. 6)

8.3.2. Rationalizations

“When an incident occurs, the limits of some components have to be part of the story (a) given the trade-offs that were necessary since resources are limited and goals conflict and (b) given that the system and its environment continue to change.” (Woods, 2023, p. 7)

“Believes the effects of surprise can be compartmentalized, whereas actually, surprises compound and spread over the extensive interdependencies in all modern systems.” (Woods, 2023, p. 7)

“In the aftermath of incidents and breakdowns, the assumptions lead to increased pressure for compliance rather than learning the importance of guided adaptability” (Woods, 2023, p. 7)

8.4. Reconceptualization

1. Plans are Resources for Action

“The finding that plans only function as resources for action—not specifications is generally traced to [11].” (Woods, 2023, p. 8)

“This is highlighted in definitions of skill: the ability to adapt behavior in changing circumstances to pursue goals despite trade-offs” (Woods, 2023, p. 8)

2. Plans are Necessary to Recognize Anomalies

“To see events and changes as unexpected requires a strong appreciation of what is typical, standard, or even “normally” abnormal” (Woods, 2023, p. 8)

“Seeing what doesn’t fit your model of what has been going on, or what should be going on, or what usually happens is a form of insight” (Woods, 2023, p. 8)

3. Plans (and Automata) are Competent but Brittle

“Studies looking at joint systems of people and AI or operators and advanced automation revealed the fundamental brittleness of automata regardless of the underlying technology [13].” (Woods, 2023, p. 8)

“the problem identified was the way the new capability was deployed produced competent but brittle systems.” (Woods, 2023, p. 8)

“Risk of brittleness is universal.” (Woods, 2023, p. 9)

4. People (with the right help) provide the extra adaptive capacity to mitigate brittleness

“(a) challenges occurred much more often than stakeholders realized, and (b) people in some roles were the critical source for resilient performance despite the stresses, risks, uncertainties, threat of overload, and bottlenecks” (Woods, 2023, p. 9)

“All systems are developed and operate with finite resources and live in a changing environment. As a result, plans, procedures, automation, agents, and roles are inherently limited and unable to completely cover the complexity of activities, events, and demands.” (Woods, 2023, p. 9)

“Without this capability for extensibility, brittle collapse would occur much more often than it is observed [6].” (Woods, 2023, p. 10)

“Adaptation is not about always changing the plan, model, or previous approaches but about the potential to modify plans to continue to fit changing situations.” (Woods, 2023, p. 10)

8.4.1. Plan and Revise: Guided Adaptability

“The new science shows that this assumption is guaranteed to be wrong in the future, regardless of how well the plan has guided performance in the past. The timing on this guarantee is linked to the pace of change within and around the organization and how those changes expand the tangle of interdependencies it exists within.” (Woods, 2023, p. 10)

“The irony is you can only monitor how well plans fit the world by understanding how people have to adapt to fill the gaps and holes that inevitably arise as variability in the world exceeds the capability of plans and the competencies built into any system [12].” (Woods, 2023, p. 11)

“Monitoring how people adapt to make the system work does not constitute approval that these adaptations are the “best” given the trade-offs faced in different situations. What is “best” is itself a dynamic judgment that can and should change as challenges vary— reprioritization rebalances goals in the trade space to fit the situation.” (Woods, 2023, p. 11)

“Driving gap-bridging adaptations underground also makes it harder to recognize how plans do not fit the changing patterns of variability in the world.” (Woods, 2023, p. 11)

“Recognizing what adaptations are going on allows one to see the resources—physical, cognitive, collaborative, and others—that people draw on to produce resilient performances in the face of challenges small and large.” (Woods, 2023, p. 11)

“(a) about challenges that recur in general even though the specifics vary in individual events and (b) about the ways people work and coordinate to handle challenges.” (Woods, 2023, p. 12)

“If safety is about “repair after something goes wrong”, no organization can keep up with the pace of change, growth, and scale of modern systems and activities [32].” (Woods, 2023, p. 12)

“However effective your organization has become, however you have developed and deployed new capabilities to grow, whatever your record of past improvement in reliability/ productivity/efficiency, and whatever the promises of new capabilities to-be-deployed, the world, in the near future, will produce challenges that go beyond the competencies embodied and require adaptive capacity to stretch.” (Woods, 2023, p. 12)

“You will have to establish the continuous feedback/learning loop in order to adapt how you adapt.” (Woods, 2023, p. 13)