The Agile Innovation Dilemma

One the seminal events for the development of the agile software development movement was the 1986 publication in HBR of “The New New Product Development Game” by Nonaka and Takeuchi. Describing lean production principles applied to new product development, the paper introduced the metaphor of a rugby team where a clear goal, overlapping skill sets and joint accountability allow teams dynamically adapt and self-organise to achieve their objectives despite unforeseen setbacks and challenges. From this, the term scrum was used by Sutherland and Schwaber in 1995 to describe an incremental, team based approach to software development. In this way, agile development and innovating new products share a common lineage.
Agile methods have long been advocated in supporting innovation. They explicitly call for self-reflection and improvement of the method through retrospection. Close customer contact and an understanding of the business problem to be solved can help the development team develop more innovative solutions than if they were coding to a static functional specification. Proponents have written of ‘hyper-productive’ scrum exhibiting ‘punctuated equilibrium’ leading to discontinuous or radical innovations. However, agile practicioners offer an alternate view – the intense focus on delivering small increments of customer centric features in the immediate future undermines their ability to seek out alternate solutions, to ‘think outside the box’. This is what I refer to as the “agile innovation dilemma”. So what has gone wrong?

Agile methods promise, first and foremost, agility – that is, flexibility in responding to changing requirements, technology and markets. But in promoting agile, more traditional values such as productivity, quality and time to market are often to the fore.  In efforts to realise these benefits, agile implementations often become exercises in micro-managing development teams, with local optimisations coming to the fore. These are reflected in many of the ‘patterns of failure’ seen in agile projects, and I’ll be describing some of these in future posts. Unfortunatly, these tend to mitigate against innovation, resulting in ‘pressure cooker’ development environments where all focus is on delivering the next increment of functionality, where the product owner is breathing down your neck, and where doing the simplest thing can take precedence over doing the ‘right’ thing.

When did you last hear of agile being introduced to enhance the innovation of a team?

Agile and SMEs

I’ve been reading some material on attempts to implement agile methods in small companies and a recurring theme is that such firms don’t have the resources to invest in training & coaching and yet can’t afford to make mistakes when adopting an innovation like agile. Unlike larger companies, small organisations can’t launch process re-engineering and improvement programs, can’t run pilot projects which could fail, or experiment and learn by their own mistakes.
It got me thinking about how small and large firms are agile in seemingly different ways. Christen Claytonson introduced the idea of The Innovators Dilemma in his book of the same name which, simply put, says that the strategy of listening to your customer and innovating your product to move up the value chain is doomed to failure. This approach was espoused for decades with the tagline “the customer is always right”. The belief was that by delivering what the customer asked for made sure you were delivering what they wanted.
But Christenson showed through several examples that the customer doesn’t always know what they want, or even what its possible to get. He uses the example of mini steel mills in the US which undercut the larger mills by cheaply churning out simple steel products, while the established firms ignored them and focused on the higher margin work – following their strategy of moving up the value chain. But the mini mills gradually ate into the market, becoming more sophisticated themselves and aiming for higher value work. By the time the large mills realised what was happening their high cost, high margin business was collapsing. They had experienced “disruptive innovation”.
A more recent example, and one closer to home, is the rise and rise of low cost airlines like Ryanair – I don’t think any customer asked the airline to charge them for peanuts, or fly them to an airport an hour outside town, or charge them for luggage. But by listening to their customers, established airlines had missed the opportunity for low cost, mass air travel.

So what has this got to do with agile & SMEs? For small companies, business agility has been a powerful weapon – the ability to sense and exploit emerging opportunities, leverage their small size, co-location and lack of bureaucracy to move quickly has given them a competitive advantage versus the scale and resources of larger firms. But recently big firms have begun to invest in agility. Agile and lean philosophies are helping large firms transform from high overhead, hierarchical command and control organisations to more loosely federated small units with increased autonomy and their own sense and adapt mechanisms. If larger firms succeed in developing business agility (and I believe they are)  it will dilute a major competitive weapon of the small firm.

SMEs need to find a way to counter this movement before they get beaten at their own game.

Knowledge Creation is at the Heart of Innovation

“The raison d’etre of a firm is to continuously create knowledge” – so say Nonaka and Takeuchi, creators of the most dominant theory on how organisations create knowledge. This sentiment is echoed by Peter Drucker, probably the most famous and respected thinker on management: “knowledge is the only meaningful resource”. And yet, in current discussions on the building of a smart or knowledge economy in Ireland, the focus is on innovation. So what is the relationship between knowledge and innovation?
Schroeder and Van de Ven provide the following definition:
The process of innovation centers on the temporal sequence of activities that occur over time in developing and implementing new ideas from concept to concrete reality’.
There are two major components to innovation – coming up with ideas and designs for something new, and implementing them in a ‘concrete’ way. The first of these is implicitly tied to knowledge – it takes knowledge of problems and possible fragments of a solution to come up with ideas and designs, and those ideas and designs are themselves new knowledge.
Therefore, innovation at an organisational level requires knowledge creation at an organisational level. I have just completed a white paper outlining Nonaka and Takeuchi’s framework of knowledge creation – the SECI framework. This should be of interest to anyone promoting innovation and knoweledge creation in their organisations and is available here