In a series of posts I’m going to examine some of the claimed benefits of agile methods – are they justified? My first post looked at the cost of development with agile. Here we address the speed of development.
Agile methods are often portrayed as delivering software faster than traditional, planned methods. Here are some of the ways agile gets value to market faster:
- Note I use the word ‘value’ above rather than software – traditional methods don’t deliver any value to the customer until the entire product release is completed. But through incremental delivery, agile delivers the most valuable software features much earlier than this.
- Through timeboxing development, agile facilitates concurrent design, test and development allowing the entire end to end process be radically compressed.
- By breaking requirements into small ‘user stories’ it eliminates ‘batch dependancies’ where one feature can’t be completed or released until others in the requirement are also complete – the value ‘flows’.
- Delivering the highest priority items first often means features included ‘just-in-case’ are never developed – they are superseded by new higher priority items as the business context evolves
- Using smaller, evenly sized user stories and short iterations reduces variability and thereby queuing times in complex systems such as software development teams
- Methods such as Scrum & XP allow the team to work faster by explicitly eliminating distractions and interruptions – for example, in scrum the scrummaster is tasked with barring interruptions to the team and ensuring the sprint backlog is not changed in any way during the sprint. Other sources of distraction, such as being assigned to multiple projects at the same time, and changing team composition, are not encouraged.
- Finally, although there is an up-front cost in providing automated test, continuous integration and other automated processes, these save time over the duration of the project and allow the team focus on delivering value with greater speed.
“The raison d’etre of a firm is to continuously create knowledge” – so say Nonaka and Takeuchi, creators of the most dominant theory on how organisations create knowledge. This sentiment is echoed by Peter Drucker, probably the most famous and respected thinker on management: “knowledge is the only meaningful resource”. And yet, in current discussions on the building of a smart or knowledge economy in Ireland, the focus is on innovation. So what is the relationship between knowledge and innovation?
Schroeder and Van de Ven provide the following definition:
‘The process of innovation centers on the temporal sequence of activities that occur over time in developing and implementing new ideas from concept to concrete reality’.
There are two major components to innovation – coming up with ideas and designs for something new, and implementing them in a ‘concrete’ way. The first of these is implicitly tied to knowledge – it takes knowledge of problems and possible fragments of a solution to come up with ideas and designs, and those ideas and designs are themselves new knowledge.
Therefore, innovation at an organisational level requires knowledge creation at an organisational level. I have just completed a white paper outlining Nonaka and Takeuchi’s framework of knowledge creation – the SECI framework. This should be of interest to anyone promoting innovation and knoweledge creation in their organisations and is available here
Agile uses the idea of fixed timeboxes where the scope of work completed for an iteration is estimated based on a duration such as 2 weeks. Importantly, even if all the planned work for an iteration isn’t complete, what has been developed is built and delivered anyway – the iteration is timeboxed and the schedule does NOT slip. This ensures there will be some value delivered to the customer, even if it falls short of what was planned. But in lean thinking, the focus is on flow – the continuous flow of value rather than the incremental flow. In a discrete production context, this manifests as a striving to reach a batch size of 1 – rather than a ‘production run’ on a machine making 1000 widgets, it makes just 1 as it is pulled from the subsequent operation. This move to smaller batches and therefore towards continuous flow is possible because of new manufacturing technology – for example a machine that can be instantly reprogrammed can easily switch from making one type of widget to another. Similarly, in software development, we strive for ever smaller increments, ever shorter iterations. And, like production, this is enabled through technology such as automated testing and continuous integration. These technologies reduce the cost of testing, building and deploying and allow us deliver value in a way approaching the ideal – continuous flow – advocated by lean philosophy.
In a series of posts I’m going to critically review the common value propositions of agile – do the claims stand up to scrutiny?
Claim 1: Agile makes for cheaper software.
There are a number of justifications for this claim.
- No more kitchen sink – with waterfall a common pattern is the tendency of the customer to squeeze in all possible requirements at the beginning of the process since the ‘change control process’ will make it very difficult to add requirements afterwards, and any changes would give a justification for the development team to slip the schedule. So lots of features, poorly understood and unlikely to be needed, are included up front. This leads to more complexity in the design which now has to take into account more features, delays starting time of development as it takes longer to get signed off requirements, and puts features with low or even no current business value on an equal priority for delivery as critical functions (since all must be delivered in a big release). The software delivered in the end therefore has more functionality than in agile – it costs more to produce.
- Invest only in artifacts with business value: Agile values working software over documentation, but also values documentation with business value over that merely used to ‘run the project’. So time spent developing requirements specs, test plan documents, status reports, etc can be spent instead on great user guides, operations guides, regression test suites, etc.
- Lower Opportunity Costs: Because agile delivers value earlier, it reduces lost opportunity costs and enables the business address opportunities in the market more quickly.
- Lower Product LifeTime costs: The up front investment in continuous integration, automated test suites and refactored, well structured code has been shown to be rewarded several times over in reduced ongoing maintenance costs throughout the softwares life, and even extending that life beyond where ‘dirtier’ code becomes uneconomical to maintain.
- Delayed Investment: By returning business value from early in the development lifetime, an earlier return on the investment is realised, and further investment can be delayed till its really necessary.
- Global vs. Local Optimisation: A key lean principle is global over local optimisation. Large batches rather than continuous flow, specialisation rather than skills redundancy and other traditional tactics to increase resource utilisation aim to acheive optimisation of a particular task set such as development or testing. But it is the optimisation of the whole that is important and agile cuts costs by focusing on this global optimisation.
Are there other ways agile cuts costs? If you know more please share them…
Next blog I’ll look at how agile can speed time to value.
I often come across the question “Whats the difference between agile and lean, and how are they similar”. Big question!
There is an ongoing debate on the relationship of lean and agile methods and philosophies. Lean arose from Japanese production and is based on the two ‘pillars’ of continuous improvement and respect for people. It aims for elimination of waste, continuous, even flow and constant learning and striving for perfection (read innovation here). Agile arose from software development and focuses on emergent design, incremental delivery and continuous and disciplined planning. While lean has been highly developed for repeatable production tasks according to a preconceived design, it has also proved highly effective in development tasks where it aims to ‘outlearn the competition’. Agile has been used in development to create new designs and, as the name suggests, is specifically about change rather than repeatability.
These quite different lineages have been used to justify the stance that lean and agile are quite different approaches, and that even though Lean is now being applied to software development, they are mutually exclusive:
Kent Beck – “The values, principles, and practices of the two approaches are different, even though complementary.”
A contrary view is, however, emerging. Mary Poppendeick reflects the view that agile methods are an implementation of the lean philosophy in software development. Her seminal book “Lean Software Development: An Agile Toolkit” which first defined the lean approach to software development, reflects this in its title. This view is described well by Dean Leffingwell:
“I’ve been struck again and again by common principles – not just like principles or complementary principles – but virtually identical, fundamental, and immutable principles that provide the cornerstone values and philosophies of both lean and agile approaches. I’ve become more and more convinced that agile is a software instance of lean– purpose built for the unique challenges, intangibles and thought-based work of developing software as fast as efficiently as possible – but as lean as lean can be in that context.”
As this debate develops I hope to write a series of blogs exploring the issues and the debate, and do some ‘future-gazing’ as to how the emergence of lean will influence the evolution of agile development.
With several companies I’ve worked with, agile has been seen as a silver bullet to solve problems of late delivery, bad quality and high cost. But agile is more than a set of practices – its a system of values and principles that need commitment and resources to bring success. Senior management will often say “this team is agile” just because they read the book, have daily stand ups and use words like sprint, iteration and user story. But this has been likened to a cargo cult, where performing agile ‘ceremonies’ are somehow construed as bringing benefits of a true agile implementation. Perhaps this is another aspect of the ‘illusion of control’ invoked by detailed planning and scheduling of projects in waterfall methods. And like the damage such illusion did to waterfall, blind belief in a sprinkling of agile magic dust can prove to be a major setback for true agile adoption.
Agile practices such as test driven or test first development, continuous integration, refactoring and automated builds are generally invisible to the customer. Work on these doesn’t deliver any features or, it first seems, value. But these are essential to the longer-term quality of the product, the ability to frequently deliver working increments, etc. Activities like integration have to happen at some stage – taking this pain early in the project gives much better predictability for the remaining deliverables. Customers need to be educated up-front about the benefits of these practices – trying to ‘hide’ them, as I’ve seen teams do, only leads to mistrust.
Hoping to get to this talk on using social media to win business