Agile Transformation 2015 and the Gartner's Hype Cycle

I urge you to read up on the Gartner Hype Cycle before proceeding.

Let's face it - Agile is on life support and everybody is in denial. Sure, it shook us out of Waterfall complacency. It also handed us a boatload of cool jargon and delivered some but mostly is continuing to painfully disappoint. 

This could end in two ways - either discard Agile and pick up the next hot methodology waiting in the wings (antifragility?) or fix Agile ruthlessly to make it work. I am rallying for the latter.

The Agile story for an organization starts with 'Agile Transformation' and truth be told, often never ends because of the chasm between ideal Agile and realized Agile. There is usually no effort to quantitatively measure the progress of the transformation and therefore, this wild-goose chase goes un-noticed.

Here is a simple idea for assessing the success of Agile using the Hype Cycle: 

1. Deconstruct Agile into its components and garner feedback from the teams on the efficacy of each component. 
2. Construct the Gartner's Hype Cycle based on this feedback and take a long hard look at where each component stands, and why. 

Such analysis could be a great foundation for re-designing the process towards success.

Each organization needs to chart out the Agile Hype Cycle for themselves, using the feedback received from the early adoption teams or even the mature teams. 

What do you do next with this information? For one, confront the Hype and Trough items and assess how to get creative, and perhaps even deviate some, to push those items towards the plateau of productivity. This could potentially give Agile a new lease of life.

To demonstrate the Agile Hype Cycle that I recommend for each organization, I went ahead and created one with my understanding of how each aspect in Agile is faring in general:

Gartner's Hype Cycle for Agile 2015*

* Created by Deepa Karnad Dhurka based on empirical information

Agile Components	Status
Scrum meetings	Slope of enlightenment Teams value this quick daily sync up and have adapted it to their specific needs.
Sprints/Demos	Slope of enlightenment Scoping a sprint at a high level and setting goals for that short period is an organized approach to check-pointing progress. Demos are useful for sharing information in a heterogeneous team that works on different projects.
Sprint Planning and Burndown	Trough of disillusionment Extremely difficult to predict exact estimates. Can lead to bidding wars for tasks. Updating tasks with burndown is tedium. It constantly reminds the developer of process which is contrary to Agile’s objectives.
Backlog Grooming	Trough of disillusionment How much is enough? Is this work product mature for planning and budgeting long lead projects? Unclear.
Sprint Retrospective	Trough of disillusionment Influence is limited to the specific cross-functional. Northbound feedback is absent leading to growing chasm between the ‘coaching’ community and the developer's.
CI/CD	Slope of enlightenment CI/CD has matured, and is accepted as the de-facto release and deployment model.
Iterative Development	Peak of hype Designs for refactoring. Lets sloppy design through. Is expensive over the long term, especially with growing codebase. Less exciting often than developing something new.
Technical Debt	Peak of hype Designs in bugs to allow for expedient development. Lets sloppy code through.
Agile Roles	Trough of disillusionment Transient roles, each with a learning curve. Some roles overlap with traditional titles like Project Manager or Line Manager. Can lead to tension and confusion due to unclear boundaries of responsibilities. Most organizations aren’t great at mapping these to appraisal goals. Career growth suffers. Accountability is another serious concern with an abundance of roles.
Agile Transformation	Trough of disillusionment The ideal Agile implementation always seems beyond reach, leaving the organizations forever in transition.

Iterative software development is anything but a new concept. Fred Brooks’ The Mythical Man-Month, first published in 1975, prescribes it. It is beside the point that I wish I lived in a world where Person-Month was the norm; 40 years since this book, it still isn't.

Agile however has a lot of wrappers around the core concept of iterative production/deployment. Agile is a system (or dare I say it, a ‘process’) that although claims to be ‘loose’ and a ‘recommendation’, is followed more fastidiously than Waterfall. I don’t remember Waterfall dictating types of meetings and meeting cadence for a project – at least not in practice. Also, the set of Principles is akin to the Commandments, making it look and feel like a religion. I won’t go down that analogy any further but there is more than one example I can provide of the parallels.

I have been a Product Owner long enough that I have seen the liberation Agile brings and also the colossal costs (for individual and organization) of failure. The ‘retrospectives’ run at individual team level do very little to influence the enterprise-level Agile design and therefore the feedback loop is imperfect, if it exists at all.

Agile has a bunch of good things about it, including a genius name, but it is also extremely flawed. If an organization doesn’t look these flaws in the eye and address them in ways that are very unique to each company, everyone will forever remain in transformation. 

I’ve sat in too many ‘why Agile is awesome’ workshops (the script is nearly the same everywhere) and lived it enough to know that Agile is but ‘human’ with all its zits and warts!! It is time to admit the shortcomings of Agile and make an earnest attempt at fixing them. It is time for Agile Coaches to be honest about the failures and engage cross-functionals in creative solutions to fix the process company-wide. 

I hope for the developer's sake that these discussions start now.