Moving from Authority to Responsibility

Lean Thinking, which underlies Agile methods like Scrum and XP, has as one of its central pillars “respect for people”. Agile reflects this in terms of ‘whole team’ accountability, collaboration and self-organisation. All these factors lead to agile seeing team members from a more ‘humanistic’ point of view – they are more than just resources that can be swapped in and out of projects – the sort of ‘bean-counting’  mindset giving rise to the ‘mythical man-month’.  Agile takes teams and individuals much more seriously, calling for long-life, cross-functional teams that are allowed the time, latitude and autonomy to gel into a high-performing whole. This change is well summed up by the phrase ‘Move from Authority to Responsibility’. Where organisational structure is based on individual authority rather than joint responsibility, it leads to fragmentation, isolation of roles, hand-offs, friction and ultimately poor organisational performance.

Its great to have really simple and memorable phrases to guide our day to day decisions and I think this is one of those: Build Responsibility, not Authority.

How agile are you?

One of the big culprits in failed agile teams is the tendency to cherry pick those practices that seem to ‘fit with the way you work’, ‘with the way we do things around here’. Agile explicitly calls for methods to be customised depending on context. But often this can be misconstrued as selecting those bits that are compatible with how you work now – thereby leading to no fundamental change in the way you work. Examples are iterations as long as the release cycle, calling the project manager a ScrumMaster without a change in role and considering a feature or story ‘done’  when it has been coded and passed to QA. This leads to a Cargo Cult adoption where the team adopts the language and some ceremonies of agile, without understanding the fundamentals of how it works. No wonder the benefits are elusive…

When assessing how well teams have adopted agile methods like scrum, the approach is usually compliance based – an evaluation of how closely the team follows the defined method – whether customised or not. There are two fundamental difficulties with this:

1) The way in which agile practices are implemented in a team has a great bearing on how they support or constrain agility – for example, a daily stand-up meeting that spends 45 mins getting status updates from everyone is really not going to help a self-organising team co-ordinate their actions for the day. Even a stand-up of 10mins where the three standard questions are posed can be ineffective if the team doesn’t engage and feel ownership of it. Therefore, assessment by compliance evaluates, well…, compliance – not agility which is probably what you want to know.

2) Since each project & team implements their development method differently (a scientific fact from extensive research), and since that implementation evolves over time, using compliance as the basis for assessment hinders inter-team comparisons – akin to comparing apples and oranges. A lot of the value in assessing a team is so you can benchmark and compare to other teams as a way to identify possible paths to improvement. Without the ability to effectively compare, the assessment just isn’t all that valuable.

3) Most methods already used by teams have some really good aspects. Moving to agile should preserve these (unless replacing them with something even better). If attempts to be compliant with some textbook method causes these to be lost, then we’re really ‘throwing out the baby with the bathwater’.

To overcome these, my colleagues and I have been developing an assessment that looks at agility from first principles – regardless of what method is being used by the team. Of course agility is a complex concept with many facets such as creativity, responsiveness, simplicity and quality. By focusing on how any given method contributes to these facets, we can assess how it contributes or detracts from agility as a whole. We can also compare very different methods, like scrum, XP or indeed waterfall. And we can make recommendations which preserve whats valuable in what you do today while tackling those areas promising improvement.

In another post I’ll discuss an alternate assessment technique I use – rather than assessing agility, this one identifies barriers to adoption and helps map out an adoption strategy tailored to a team, project and organisation.

Agile Product Management

Agile is reaching a stage of maturity in some companies where attention is turning to how it can be scaled up from projects to products, portfolios and the whole enterprise.

In classic agile, where the focus is on one small, co-located team, product management is carried out by the ‘product owner’ role. In scrum, this role is responsible for the value being delivered to the customer, for the ROI. The tools they use include the product backlog, release planning, iteration planning and iteration review.  The product owner should have an intense interest in ensuring a common ‘Definition of Done’ is agreed across the team or multiple teams,  and work with the scrummaster to ensure it is always met. By ensuring each iteration only delivers ‘Done’ features gives transparency into progress as well as enabling various product release options at the end of each iteration (for example if a user story has been developed but not integrated and tested, it cannot be included in a release and so constrains the product owners options).

Each agile project must have one, and only one, product owner. Although this role can be supported by various domain experts (eg marketing, customer operations) or ‘Area Product Owners’ for large products, final decision making power must reside with one person.  With ownership of the product backlog, the product owner must work with teams to ‘groom’ the high level user stories, breaking them into small, even sized stories that can be implemented independantly and deliver ‘minimum marketable value’. They also work with the team in selecting the stories that go on each iteration backlog and in serving as or facilitating access to the ‘on-site customer’ for story elaboration. The product owner is also responsible for defining acceptance criteria and reviewing and accepting stories at the end of each iteration.

Many see agile as an implementation of lean principles in software development, and so talk of enterprise scale often refers more to lean than to agile. Here I discuss some of the emerging themes in organisational level agility:

  • Cadence: Agile introduces the idea of timeboxed iterations which are repeated over and over, each including planning, executing, delivering value and reflection and improvement. Methods like scrum recommend that these iterations are of constant duration, eg 2 weeks. This introduces a ‘cadence’ into the project – a smooth rythm which helps establish a continuous flow of value and facilitates ongoing coordination between project stakeholders.  This basic rythm of iteration development is also supplemented by sub-rythms, such as daily stand-ups, or hourly builds, and by meta-rythms, such as production releases every couple of  months. Just as these cadences provide a placeholder for synchronisation and communication between individuals in a team, they can also serve the same purpose for multiple team programs, multiple program portfolios and multi-portfolio organisations.
  • Options: Agile uses the idea of a prioritised backlog to ensure only the features that are most important at a particular time are worked on. Lean uses the ideas of delaying committment to the ‘last responsible moment’ and set based design. These mechanisms have similar aims – don’t constrain future options by what your doing today unless you really have to – that is, keep your options open.  This is especially powerful at the product, program or program management level – imagine if the program could be re-directed, or even terminated, at any point according to how market, technology or other contexts evolve.  By only investing resources in what can deliver value in the short-term, the program retains control of potential future expenditure. This is a compelling reason to move from projects to increments as a system development metaphor, as described in a previous blog.
  • Pull: Lean philosophy introduces the idea of pull, with the ‘kanban’ or ‘just-in-time’ material control mechanisms the most common examples.  Where traditional planning approaches schedule the pushing of materials, parts, customer requirements or software features to where the plan says they will be used next, pull systems use signalling built into the process to co-ordinate production and consumption along the value chain.  In manufacturing, a part isn’t produced until its needed by the next operation on the line. In software development, requirements aren’t elaborated and fixed until just before they are developed. In test driven development, code isn’t developed until a test fails. Effective pull relies on the even flow established by a cadence. And pull and associated in process signaling are core to self-organising teams and value chains – it eliminates the need for an ‘out-of-band’ signalling system represented by a ‘management layer’ assigning tasks, providing visibility through status reports, etc. Pull and the self-organisation it enables are key to scaling up agile development and are therefore essential to product, program and portfolio management that uses multiple teams.
  • Unified Build: In large enterprise development, many teams may be involved. Detailed coordination of multiple technical teams has always been a difficult and costly management task, with significant overhead in terms of meetings, documentation, negotiation, etc. With agile approaches, inter team coordination moves from the planning level to the code level – continuous integration and automated test are essential coordination mechanisms both within and between teams. Where products are being developed by multiple teams, continuous integration must build the entire end product, not just a subset of it.

Lean philosophy has been proven to scale effectively over decades in the worlds largest companies, the most famous example being Toyota. In the 80s’ and 90s’ western companies adopted many of these techniques in their manufacturing operations. Now there are moves to leverage the same underlying philosophies in development, with agile being a well progressed example in software development. Higher level organisational roles such as product management continue to evolve to the new philosophy and specific approaches will emerge over the coming years.

Agile vs. Lean

I often come across the question “Whats the difference between agile and lean, and how are they similar”. Big question!
There is an ongoing debate on the relationship of lean and agile methods and philosophies. Lean arose from Japanese production and is based on the two ‘pillars’ of continuous improvement and respect for people. It aims for elimination of waste, continuous, even flow and constant learning and striving for perfection (read innovation here). Agile arose from software development and focuses on emergent design, incremental delivery and continuous and disciplined planning. While lean has been highly developed for repeatable production tasks according to a preconceived design, it has also proved highly effective in development tasks where it aims to ‘outlearn the competition’. Agile has been used in development to create new designs and, as the name suggests, is specifically about change rather than repeatability.
These quite different lineages have been used to justify the stance that lean and agile are quite different approaches, and that even though Lean is now being applied to software development, they are mutually exclusive:
Kent Beck – “The values, principles, and practices of the two approaches are different, even though complementary.”
A contrary view is, however, emerging. Mary Poppendeick reflects the view that agile methods are an implementation of the lean philosophy in software development. Her seminal book “Lean Software Development: An Agile Toolkit” which first defined the lean approach to software development, reflects this in its title. This view is described well by Dean Leffingwell:
“I’ve been struck again and again by common principles – not just like principles or complementary principles – but virtually identical, fundamental, and immutable principles that provide the cornerstone values and philosophies of both lean and agile approaches. I’ve become more and more convinced that agile is a software instance of lean– purpose built for the unique challenges, intangibles and thought-based work of developing software as fast as efficiently as possible – but as lean as lean can be in that context.”
As this debate develops I hope to write a series of blogs exploring the issues and the debate, and do some ‘future-gazing’ as to how the emergence of lean will influence the evolution of agile development.