XSCALE: An Agile Enterprise Ecosystem

- - posted in autonomous, continuous, ecosystem, excellence, lean, scale-symmetry, v0.8, xscale

XSCALE is a continuously delivering, continuously self-organizing, continuously adaptive Agile enterprise ecosystem that turns all the knobs to ten.

Why XSCALE?

There have been several efforts to provide enterprises with a way to scale Agile. The most popular are SAFe, LeSS, Agile Path and the Spotify Tribal model. Each has its strengths and weaknesses.

XSCALE leverages the strengths:

  • Standardizing team practices using Scrum-XP like SAFe
  • Transforming the enterprise iteratively like Agile Path
  • Refactoring component silos into feature teams like LeSS
  • Integrating tribal structures, CD and DevOps like Spotify

XSCALE introduces some new patterns to bind these practices together,:

  • Continuous adaptation using the Seven Samurai pattern language.
  • Continuous product planning using a method based on Lean Startup, Lean UX and BPP
  • Continuous self-organization using Iroquois councils, a combination of Open Space, BDD and Kanban.

Like any framework, XSCALE is a collection of pattern languages that have been shown to reinforce each others’ strengths. It aims to afford enterprises on all scales an iterative path to state-of-the-art Agile capabilities while minimizing risk of disruption to existing value streams.

XSCALE Principles

XSCALE conforms with the A:TNG values, which are themselves a strict superset of those in the Agile Manifesto. XSCALE also offers six principles that form a superset of the ones in the Manifesto.

eXcellence
Beck’s Maxim: Turn all the knobs to ten.
Scale-symmetry
Work breadth-first to control combinations.
Continuous
Measure and minimize cycle time and batch size.
Autonomous
Let consensus determine decisions and accountabilities.
Lean
Maximize net ROI.
Ecosystem
Adapt the whole to its parts.

Descriptions of the principles follow, then a selection of practice patterns that embody them.

eXcellence

The term “extreme” in the original XP unfortunately evoked the idea of recklessness, which caused considerable confusion in the fledgling days of Agile. The manifesto authors recognize XP’s intent with their 9th principle, “Continuous attention to technical excellence and good design enhances agility”. XSCALE calls this continuous attention “eXcellence”, and takes Beck’s definition:

  • If code reviews are good, we’ll review code all the time [Pair Working].
  • If testing is good, everyone will test all the time, even the customers [BDD]
  • If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality [The DRY principle - Don’t Repeat Yourself]
  • If integration testing is important, we’ll integrate and test several times a day [Continuous Integration]
  • If short iterations are good, we’ll make the iterations really short [INVEST Stories]

And so on - it wobbled a bit from edition to edition but Beck summed up the intent with the maxim,

Turn all the knobs to ten.

XSCALE regards these XP engineering practices, CD and Feature Teams as the fundamentals of technical excellence. These patterns interlock to assure technical debt is paid before it earns interest that impacts ROI, and thereby keeps the delivery process stable, predictable and flexible.

Scale-symmetry

Scale is about combinations of requirements, of dependencies, of defects, and of conversations. The number of combinations of a set grows much more rapidly than the number of its members; a set of just 10 things has over 3.6 million distinct combinations.

So how do golfers do it? Golfers are untroubled by combinations of billions of grass blades. They make simple distinctions about the grass: “the green”, “the rough”, and so on. Once the ball is placed they make more distinctions about distances, slopes and clubs - a map in their heads that says where the ball lies relative to the hole. They never think about the combinations of individual grass blades.

Agile actively minimizes combinations of concerns when it

  • _DRY_s code into distinct modules to group common behaviours
  • automates testing to keep ambient defect count low, thereby minimizing the multiplication of time required to sort out combinations of defects
  • combines people into small teams with distinct backlogs to limit the need for cross-team coordination
  • The Spotify patterns which factor cross-cutting concerns to distinct chapters and guilds to minimize paticipants in the conversations required to resolve them.

In general the principle is:

Work breadth-first to control combinations

In the above animation of a “Koch Curve”, a line segment is mapped to 4 smaller line segments to look like a tent. Then each of those segments is transformed the same way, breadth-first, and so on until it looks like broccoli. If a hill-climbing algorithm is used on this Koch curve, it will always get stuck in a local maximum. Analysing the curve breadth-first, however, always finds the global maximum.

XSCALE applies this to the challenge of determining which combinations of business alternatives have most value. Agile works like climbing a hill, continously integrating little stories into the evolving whole. So it can climb a molehill of value when there’s an Everest right next door. Small projects look to a good product owner to find their Everest, but when you’ve got portfolios of programs of projects, misalignment is easy and maximizing value is combinatorially hard.

XSCALE uses Behavioral Product Planning patterns to control this combinatorial space. BPP systematically defines breadth-first maps of system behaviours, then sorts these into easily optimized piles. It iterates to determine the features of maximum ROI and assure consistent decisions about business alternatives at all scales. This way analysis can be conducted iteratively, quickly and efficiently, quickly ruling out large swathes of low ROI “business space” without wasting analysis time on their in-depth exploration.

Continuous

SAFe promotes Punctuated Cadence over Continuous Delivery, employing a quarterly release train integration pattern. This is a great starting point for conservative organizations and enables them to keep their component silos, but at a price:

  • Increased cost of quality due to deferring regression and acceptance testing to “hardening sprints”.
  • Wasted time in meetings to figure out how to satisfy combinatoric inter-team dependency cycles.
  • Lost ROI from keeping finished features on ice for a quarter

By employing continuous rather than big-bang transformation, XSCALE augments SAFe with the abilty to iteratively break component silos into feature teams. The benefit is to avoid the above penalties and enable feature teams to use the GitFlow pattern language for continuous delivery. In general the principle is simply:

Measure and minimise cycle time and batch size

XSCALE applies the same principle in its product planning, stream coordination, and transformation patterns. Use the smallest practical.

What’s “practical”? Every process has a minimum responsible cadence. The best way to discover it is to start small and experiment. Try variations. Make your sprints a week long for a few weeks and see what breaks. If a week works well, try running two day sprints and see what happens. Or dispense with sprints and run review, retro and planning every time a feature is ready for release per Scrumban. The key is to work actively to experiment, learn and optimize.

Autonomous

Many of the benefits of Agile derive from peer collaboration and team consensus games like planning poker and retrospectives, and it wouldn’t be reasonable to call a process Agile at any scale if it doesn’t empower consensus through games. So the XSCALE principle is:

Let consensus determine decisions and accountabilities.

In theory, individual accountability should be applied when the business cost of delaying a decision is greater than the social cost of failing to maintain consensus. In practice it’s hard to figure these costs out. Historically the happiest medium seems to have been the Iroquois Confederacy, which used consensus at sub-Dunbar scales to control a hierarchy of peer councils.

Where consensus protocols were violated or timing called for a rapid decision, “war chiefs” were accountable. The combination of councils and limited-accountability chiefs proved stable at a multi-national scale over a period of centuries. XSCALE combines this approach with the Spotify tribal model to coordinate autonomous teams and streams.

Lean

Lean is often defined as “Minimizing Waste”. Unfortunately that definition only refers to one of the three Lean pathologies, muda. Lean also works to minimize overburden (muri) and irregularity (mura).

The waste maxim also admits an ambiguity. Lean Startup, Lean UX and LSD use Lean in its general meaning of process efficiency, but Lean Six Sigma and Lean Accounting interpret it to mean cost minimization, right first time, and hierarchical standardization. These forms of Lean philosophically oppose Agile’s focus on maximizing ROI, fail fast, and adaptive collaboration. To prevent confusion XSCALE defines the Lean principle simply as:

Maximize net ROI

We say net ROI to account for Cost of Delay (COD). The Lean principle is embodied in several specific practice patterns:

These Lean practices interlock to render a simple stream funding cycle without artificial project budgets and baselines.

Ecosystem

A biological ecosystem differs from a zoo in that it continuously adapts to changes in the populations and behaviours of its constituent organisms. The same might be said of an individual organism and its constituent cells.

The biological principles are generation and death. Generation continuously produces experimental variations in behaviour, and death removes variations that don’t efficiently adapt to the whole. Similarly we think of Lean pivoting and set-based design serving the function of generation, and Agile refactoring serving that of death, the principle for both is:

Adapt the whole to its parts

XP embodies this principle in its practice of Merciless Refactoring and Lean Startup in its Get Paid or Get Pivoting. XSCALE extends it to the rollout of structure and practices to squads in streams, and to streams in the enterprise. The ecosystem principle turns the Taylorist axiom of all-wise managers and serf-like workers on its head, leveraging the Agile function of management as servant-leaders to autonomous teams of peer workers.

In order to pivot and refactor the enterprise as a whole, XSCALE relies on a hierarchy of councils:

  • chapters as groupings of people with common capabilities across squads
  • stream councils steered by chapter representatives
  • an enterprise council steered by stream council representatives.

At each level there are also some individual accountabilities defined by these teams. Each squad in a stream has distinct roles for agile coach and product owner. Each stream has a product manager and a stream manager. And the enterprise has its executive officers.

The accountabilities of the individual roles are defined by treaties with their respective councils. Likewise the accountabilities of the enterprise as a whole, and each of its streams, and each of their chapters, are respectively defined by treaties in story-normal form. Councils each use this statement of their accountability to scope their planning using the Breadth-First Roadmap method described below.

XSCALE Patterns

XSCALE is “opinionated” in the same sense as Ruby on Rails, offering a set of concrete practices as a state-of-the-art starting point for your journey. As your streams and teams gain experience with these practices, your chapters and councils will naturally vary and augment them. Starting by building an uncompromised agile capability, you can grow and adapt it continuously without ever compromising it.

Enterprise Practices

XSCALE adopts the Beyond Budgeting premise that each value stream must either earn its keep or convince other, self-funded streams that it is affords thems sufficient value for them to invest in it to their mutual benefit.

Making such trade-offs has traditionally been the responsibility of senior executives. We expect execs will still guide and inform the choices made by the Enterprise Council, but the intent in spreading responsibility for making appropriation decisions to the stream leads is to assure that the enterprise pivots collaboratively to maximize net ROI for the whole.

BPP

While we describe the patterns in this section as “Enterprise Practices”, this doesn’t mean they’re only performed in the stratosphere. Following the principle of Scale-Symmetry these same patterns are used by streams and squads to optimize their respective ROI and reconcile same with available funding. As a pattern language based on and generating BDD we refer to these practices collectively as Behavioral Product Planning, and describe them in their own post here.

Continuous Adaptation

As per the Seven Samurai pattern language, XSCALE starts by constructing a “Minimum Viable Agile Capability” - a “spike” squad within a single stream that forms a model for all those to come. The spike squad prototypes and proves that its tools, relationships and processes integrate as an efficient end to end SDLC by delivering a stream of small concrete features with real business value.

In other words this is the familiar Agile steel thread pattern applied to organizing an Agile capability. It is fundamentally important to avoid pain at scale that the spike squad delivers whole end to end features, not merely components, so that all the technical and organizational fundamentals are reliably integrated.

To continuously integrate the spike squad with the pre-existing organization, Seven Samurai assembles the first Stream Council from the lead doers - not deciders - from each of the organization’s pre-existing component silos. As delivery squads multiply, these leads naturally function as chapter representatives on the Stream Council.

Meanwhile progressive senior managers are enroled into the initial Enterprise Council to provide a one-stop reporting and support shop for coaches. As XSCALE is adopted by more streams, their managers will take part in their respective councils and thereby may become stream reps on the Enterprise Council. As described below this leverages the Scale-Symmetric principle to assure consistent, iterative rollout of XSCALE across the enterprise.

Chapters and Councils

As explained in the Ecosystem section above, Streams and Squads have their specific purposes distinguished by treaties framed in story-normal form. Councils and Chapters adapt and expand these respective treaties by means of Breadth-First Roadmaps that detail how their Streams and Squads will organize to satisfy the terms of the treaties that define them.

We refer to this process as Continous Adaptation because it’s not a simple matter of executing a pre-defined plan. The BPP patterns are applied to estimate and prioritize the respective roadmap using themes drawn from the Seven Samurai patterns to reflect respective accountabilities and responsibilities at each level:

  • Chapters are accountable for the Adoption and Acceleration of practice patterns by their squads, and responsible for providing Awareness and Architecture sufficient to support them.
  • Stream Councils are accountable for overall Awareness and Architecture, and responsible for Stream Flow Assessment of their current state and BPP Analysis of their BFR.
  • The Enterprise Council is accountable for Assessment and Analysis across all streams, and responsible for cross-stream Alignment and, in conjunction with the CFO’s office, Appropriation of funding per stream.

Each council keeps an Adaptation Kanban to track the progress of features drawn from a backlog it generates by using BPP on its BFR, and cumulative flow diagrams - not to be confused with the Stream Flow diagrams - represent progress as the whole adapts over time.

Meeting Schedules

As XSCALE rolls out it becomes advantageous to synchronize the various council meetings so that they don’t cause calendar-stress. XSCALE leverages the squads’weekly cycle to achieve this:

  • All of a stream’s chapters meet simultaneously every Friday lunch so that squad activities aren’t interrupted. Chapters may also meet at other times coordinating via their Stream Council.

  • At the Friday lunch meeting a Chapter selects its representatives for the Stream Council, which is conducted immediately afterward while squads are conducting their weekly reviews. To avoid waste, time and location for reviews, retros and council meetings should be pre-determined and synchronized.

  • Squad retrospectives are held immediately following the Stream Council meeting so that any treaties proposed by Council might be immediately and unanimously ratified.

  • Guild meetings are held asynchronously as convenient to the Guild members.

  • Once a month each Stream holds a half-day full Council meeting for all its members in accordance with the Iroquois pattern. If ratification of a treaty was not achieved unanimously in squad retrospectives, the treaty is shelved until concerns can be ironed out at this monthly meeting. The Stream Manager may still require squads to temporarily adopt a treaty until the monthly Council can sort it out.

  • The Enterprise Council, consisting of Stream and Product Managers from all Streams, along with all relevant senior executives, also meets once a month to review Stream Flow diagrams and update the enterprise’s business canvas and related financial plans. The Enterprise Council may also meet asynchronously whenever one of its members calls for this.

Stream Practices

Stream Funding

[more to come …]

Stream Structure

By a Stream we mean a Spotifystructured tribe that self-organizes using a BFR and Feature backlog, a Stream Flow diagram, an Adaptation Kanban and a Council made up of Chapter representatives. We call this a stream rather than a tribe to emphasize its focus on optimizing a Lean value stream.

Because the Spotify model supports a semi-lattice rather than hierarchic enterprise structure, it is more open, more flexible and more responsive to change than traditional corporate structures. As Chris Alexander says, “As the relationships between functions change, so the systems which need to overlap in order to receive these relationships must also change”.

This is clearly reflected in the Spotify experience where “We can spin up a new squad to take advantage of an opportunity or handle an issue without worry about changing reporting structures. If a squad completes its goals and has no reason to exist anymore, we can dissolve it without punishing a manager”.

XSCALE augments the Spotify Model with a consensus decision-making pattern derived from the historical Iroquois Confederacy. This pattern coordinates decision-making across squads, streams and the enterprise as as whole is described in the Continuous Adaptation section below.

Stream Flow Diagram

Every Stream is intended to generate a return. If the Stream serves a purely internal function then its return may simply be defined as equal to its funding. XSCALE represents the return and the funding for a Stream in terms of Feature Points on a Stream Flow diagram, a special kind of cumulative flow diagram.

Note that where we refer to story points here we mean Monetized Story Points derived by the formula given in the Feature Points section above. The layers of the Stream Flow diagram represent, from bottom to top:

  • deployed features
  • integrated but undeployed stories
  • implemented but unintegrated stories
  • estimated but unimplemented stories
  • features with stories covered by BDD scenarios
  • BPP best fit of features to funding
  • squad/stream funding limit over time
  • stream revenue over time

Stream Flow represents all of these on one consistent diagram per squad. Because of normalization to feature points these diagrams are trivially rolled up per stream, and then to the enterprise as a whole. This diagam is extremely enlightening because it is directly amenable to traditional Lean analysis of cycle time, latency, WIP, etc. This provides visual and actionable identification of wastes, bottlenecks, irregularities and opportunities, as well as their relative magnitudes.

Team Practices

DevOps + CD + GitFlow

Teams are the most concrete embodiment of XSCALE principles:

  • Per the eXcellence principle, BDD per feature and per story is fully implemented and all CI and SIT tests pass before any feature is deployed.
  • Per the Scale-Symmetry principle, features (defined at this level as groups of stories that when deployed together will deliver some concrete business value) are each assigned to a distinct squad.
  • Per the Continous principle, sprints are 1 week long. This means all the sprint rituals have minimum practical batch size and cycle time.
  • Per the Autonomous principle, squads use the GitFlow pattern so they don’t need to wait for another squad in order to integrate, test and deploy.
  • Per the Lean principle a feature is deployed by its squad immediately as soon as it passes CI and SIT.
  • Per the Ecosystem principle, Ops forms a chapter that provides consistent virtualized test environments and production-like System Integration Testing (SIT) environments, and assures the behaviour of these CI and SIT environments are equivalent under test. This assures squads collectively own their code base andconsistently adapt to one another as it evolves.

Lean UX + BDD + XP

  • Streams use the Lean UX Product Team pattern where product owners from multiple squads along with, designers, analysts, SMEs and architects work together to break features down into groups of stories to supply the delivery squads. To comform with the Spotify structure this Product Team takes the form of a Chapter. Whenever the Product Chapter is not meeting, its members spend time collaborating with delivery squads.

  • Stories are produced in Gherkin format factored to maintain INVEST properties. Squads get together with members of the Product Team every Wednesday afternoon to estimate its latest batches of stories.

  • A story with INVEST defects is returned without an estiamte to the Product Team for clarification. Otherwise the throughput of the Product Team is graphed on the Stream Flow diagram according to the number of estimated feature points per week.

  • Squads hold reviews and retrospectives every Friday and task breakout sessions every Monday, and every day starts with a standup meeting.

  • SAFe targets existing enterprises with a high-discipline, top-down, big-change-up-front, punctuated cadence Agile model that combines Kanban, quarterly release trains, XP and Scrum. SAFe is a good starting point for conservative organizations as it’s designed to enable agility while keeping component silos intact. But that means it shys away from feature teams, continuous delivery and scale-symmetry.

  • LeSS and Agile Path are scale-symmetric alternatives based on traditional Scrum. They offer low-discipline high-collaboration, continuous cadence models that employ Scrum of Scrums to deal with integration and cross-cutting concerns. XP practices are optional but feature teams are mandatory. Like SAFe, LeSS uses big-change-up-front but Agile Path introduces a method of continuous transformation. Specific practices for product planning, test automation and so on are left for teams to determine ad hoc.

  • The Spotify model dispenses with Scrum of Scrums in favour of a pattern of Tribes and Guilds, strongly emphasizing bottom-up autonomy. It gives little prescription on team practices, calling its smallest teams squads to avoid dictating even the Scrum rules. It rightly expects a structure of top-flight autonomous teams will be able to decide for itself how to evolve Agile practices. But it still commits to Continuous Delivery and DevOps.

Comments