Extending the Scrum and Sprint Metaphors

Scrum uses the metaphor of a rugby team to convey the ‘directed autonomy’ of a ‘whole team’ working together to get the ball across the try line. Each member of the team has a role such as out-half, fullback or forward, but once the ball is in play, any member does whatever they can to progress towards the objective, regardless of their speciality. This is contrasted to a relay race team, where each member runs part of the race on their own, passing the baton at pre-determined points. This echos a waterfall process, with distinct roles and responsibilities and hand-offs between task specialities.
Another metaphor used in scrum is that of a sprint. During a sprint, distractions and interuptions to the team are minimised – the ScrumMaster has a specific responsibility to ensure they can focus purely on acheiving the preset sprint goals. This minimises context switching and multi-tasking which have been shown to be very wasteful, and increases the predictability of what will be delivered at the end of the sprint. But no-one can sprint all the time – if they try, then they are really just jogging. I like the sprint metaphor – there is one immediate, overwhelming objective and all distractions and other concerns are ignored until the sprint ends.  And during a sprint, ‘moving’ the goalposts is not allowed – no changes to the sprint goal or user stories and tasks is allowed (if this is necessary, then the sprint is cancelled). Just like a play in rugby, all focus is achieving one objective until the whistle blows.

But the ‘classic’ scrum method (Schwaber &  Beedle),  recommends each sprint is followed directly by another – sprints include ‘bookend’ events at the start and end, such as iteration planning, reviews, retrospectives, etc. Classic scrum also calls for just a little bit of distraction sprinkled throughout the sprint such as spending 5-10% of the time on ‘grooming’ the product backlog – allowing the next sprint backlog be prepared in advance of the subsequent sprint and therefore no need to dilly-dally before rushing into development again. Activities such as these require the focus of the team be diverted from the immediate sprint goal to focus on future stories and tasks.

I have a number concerns with this model.

  1. Distractions, multi-tasking and the consequent context switching are very disruptive to concentration and sap energy from a team. Try doing some real, focused work like writing a report or some complex code with constant interruptions such as calls and meetings about different topics – its very difficult to progress rapidly and steadily with the primary task. But people in the real world have to do ‘other’ stuff in the average workday: office communication sessions, performance appraisel with their boss, arrange or attend a presentation to another department, participate in the mentoring program, file travel expenses, attend the mandatory office security training session…..the list is looong. Sometimes a team will count on only 6 hours out of 8 being available for ‘real work’ because of these distractions. But how can you sprint efficiently if your not running 25% of the time? If you have to slow down, attend to something else, and then build up sprint speed again it is very wasteful.
  2. Agile purports to be an empirical control process – rather than planning the future based on experience of the past it instead uses an ‘inspect and adapt’ paradigm to continually refine the short-term plan based on present reality. In agile methods, this takes the form of getting feedback from stakeholders (customers, users) at the end of each iteration, and, based on that, building a plan for the next sprint. In many real world cases, the next sprint has already been substantially planned by the time the previous one is finished – team members have been grooming the backlog, breaking down stories and tasks, estimating, etc. Although this model impinges on the pure inspect-adapt model, many would see it as pragmatic – a sensible compromise to optimise utilisation of the team. However, I have seen this quickly lead to local optimisation (great for getting more work done, but not necessarily delivering more business value), premature investment in story elaboration, and erosion of team ownership of stories and estimates (since often one or two team members will work on stories for the next sprint while the remained deliver the current tasks). Reverting to the rugby metaphor, its like some players disengaging and setting up the next set piece before the play has ended.
  3. I discussed the idea of cadence in a previous post – cadence across teams and projects is a great basis for synchronising inter team communications. For example, in a multi-team project if one team is sprinting while the other is between sprints, there is little opportunity for joint reflection. If there is time set aside between sprints to synchronise between teams, and if the sprints start and end at the same time, it gives an opportunity for this communication. It also gives an opportunity to go ‘off-project’ and do all that ‘other stuff’ not directly related to the project. For example, what if performance reviews, training, presentation of company quarterly results, etc could only be scheduled between sprints – this would involve a cadence at the company level operating across all functions.

Again, like the rugby metaphor, I think a break from the flurry of play to take stock, tie your boot laces and plan the next move is an essential element of working effectively.

Agile Product Management

Agile is reaching a stage of maturity in some companies where attention is turning to how it can be scaled up from projects to products, portfolios and the whole enterprise.

In classic agile, where the focus is on one small, co-located team, product management is carried out by the ‘product owner’ role. In scrum, this role is responsible for the value being delivered to the customer, for the ROI. The tools they use include the product backlog, release planning, iteration planning and iteration review.  The product owner should have an intense interest in ensuring a common ‘Definition of Done’ is agreed across the team or multiple teams,  and work with the scrummaster to ensure it is always met. By ensuring each iteration only delivers ‘Done’ features gives transparency into progress as well as enabling various product release options at the end of each iteration (for example if a user story has been developed but not integrated and tested, it cannot be included in a release and so constrains the product owners options).

Each agile project must have one, and only one, product owner. Although this role can be supported by various domain experts (eg marketing, customer operations) or ‘Area Product Owners’ for large products, final decision making power must reside with one person.  With ownership of the product backlog, the product owner must work with teams to ‘groom’ the high level user stories, breaking them into small, even sized stories that can be implemented independantly and deliver ‘minimum marketable value’. They also work with the team in selecting the stories that go on each iteration backlog and in serving as or facilitating access to the ‘on-site customer’ for story elaboration. The product owner is also responsible for defining acceptance criteria and reviewing and accepting stories at the end of each iteration.

Many see agile as an implementation of lean principles in software development, and so talk of enterprise scale often refers more to lean than to agile. Here I discuss some of the emerging themes in organisational level agility:

  • Cadence: Agile introduces the idea of timeboxed iterations which are repeated over and over, each including planning, executing, delivering value and reflection and improvement. Methods like scrum recommend that these iterations are of constant duration, eg 2 weeks. This introduces a ‘cadence’ into the project – a smooth rythm which helps establish a continuous flow of value and facilitates ongoing coordination between project stakeholders.  This basic rythm of iteration development is also supplemented by sub-rythms, such as daily stand-ups, or hourly builds, and by meta-rythms, such as production releases every couple of  months. Just as these cadences provide a placeholder for synchronisation and communication between individuals in a team, they can also serve the same purpose for multiple team programs, multiple program portfolios and multi-portfolio organisations.
  • Options: Agile uses the idea of a prioritised backlog to ensure only the features that are most important at a particular time are worked on. Lean uses the ideas of delaying committment to the ‘last responsible moment’ and set based design. These mechanisms have similar aims – don’t constrain future options by what your doing today unless you really have to – that is, keep your options open.  This is especially powerful at the product, program or program management level – imagine if the program could be re-directed, or even terminated, at any point according to how market, technology or other contexts evolve.  By only investing resources in what can deliver value in the short-term, the program retains control of potential future expenditure. This is a compelling reason to move from projects to increments as a system development metaphor, as described in a previous blog.
  • Pull: Lean philosophy introduces the idea of pull, with the ‘kanban’ or ‘just-in-time’ material control mechanisms the most common examples.  Where traditional planning approaches schedule the pushing of materials, parts, customer requirements or software features to where the plan says they will be used next, pull systems use signalling built into the process to co-ordinate production and consumption along the value chain.  In manufacturing, a part isn’t produced until its needed by the next operation on the line. In software development, requirements aren’t elaborated and fixed until just before they are developed. In test driven development, code isn’t developed until a test fails. Effective pull relies on the even flow established by a cadence. And pull and associated in process signaling are core to self-organising teams and value chains – it eliminates the need for an ‘out-of-band’ signalling system represented by a ‘management layer’ assigning tasks, providing visibility through status reports, etc. Pull and the self-organisation it enables are key to scaling up agile development and are therefore essential to product, program and portfolio management that uses multiple teams.
  • Unified Build: In large enterprise development, many teams may be involved. Detailed coordination of multiple technical teams has always been a difficult and costly management task, with significant overhead in terms of meetings, documentation, negotiation, etc. With agile approaches, inter team coordination moves from the planning level to the code level – continuous integration and automated test are essential coordination mechanisms both within and between teams. Where products are being developed by multiple teams, continuous integration must build the entire end product, not just a subset of it.

Lean philosophy has been proven to scale effectively over decades in the worlds largest companies, the most famous example being Toyota. In the 80s’ and 90s’ western companies adopted many of these techniques in their manufacturing operations. Now there are moves to leverage the same underlying philosophies in development, with agile being a well progressed example in software development. Higher level organisational roles such as product management continue to evolve to the new philosophy and specific approaches will emerge over the coming years.