Skip to main content

Case Studies: Software Systems Failure

Software nightmare stories are very common - but one thing I've learned by listening to these stories over the years is the technologist must be optimist at heart.  Why - because they deal constantly with tons of failure.  And out of those failures they create innovative disruptive new sectors of the world economy (sometimes, case in point the Apple Newton and then the iPod and iPhone).

Let's look at a few case studies.

Time has just published a look at the Obama Healthcare rescue team.  Code Red by Steven Brill

"What were the tech problems?  Where they beyond repair? Nothing I saw was beyond repair.  Yes, it was messed up.  Software wasn't built to talk to other software, stuff like that.  A lot of that,"  Abbott continues, "was because they had made the most basic mistake you can ever make.  The government is not used to shipping products to consumers.  You never open a service like this to everyone at once.  You open it in small concentric circles and expand" -- such as one state first, then a few more -- "so you can watch it, fix it and scale it."
What Abbott could not find, however, was leadership.  He says that to this day he cannot figure out who was supposed to have been in charge of the launch."

The Secret Startup That Saved the Worst Website in America  How a team of young people, living in a repurposed McMansion in Maryland, helped rebuild

Stock Options? Don’t Need ‘Em! I’m Coding For Uncle Sam! (Medium)
The people behind the new government agency that’s recruiting the nation’s best tech talent to reform its hideous computer systems.
"The experience shows how great the differences can be. In newly released figures, the government says that constructing the original enrollment system, known as the Federally Facilitated Marketplace operating system, cost $200 million and would have required $70 million a year to maintain. The new version of the site, revamped by USDS engineers from Google, Y Combinator startups and other commercial tech outposts, cost $4 million to produce, with annual maintenance costs also $4 million."
Or take a look at an Agile/Scrum successful rescue -- the FBI Sentinel case management system.
FBI's Sentinel Project: 5 Lessons Learned by InformationWeek's John Foley
Case Study of a Difficult Federal Government Scrum Project: FBI Sentinel by Michael James
DoD Goes Agile by Jeff Sutherland 
DOJ's Report on Sentinel Project by Inspector General - Dec. 2011
How the FBI Proves Agile Works for Gov. Agencies by CIO's Jason Bloomberg
"Wait, agile rescued a huge money-pit fiasco of a government project? You mean, iterative, skunkworks, put-the-customer-on-the-team, forget-the-plan agile? You betcha. Agile turned out to be the hero in the tights and cape, coming to save the day."

Effective Practices and Federal Challenges in Applying Agile Methods  GAO-12-681: Published: Jul 27, 2012
What can one make from failure? Well, author John Kotter of the airport best seller's shelf (Leading Change) created his 8 step model from investigating why companies consistently fail to institute the desired organizational changes that they assumed were mission critical. His conclusion, if they had just done eight things well then the organizational change would have succeeded.
So what can we learn from two of the US governments most recent software project failures?  I think it can be summarized in one phrase - the Big Bang model only works for universes (or God).  The rest of us better learn how to iterate, grow, and evolve systems.

Mary Poppendieck wrights about the ING transition to agile... The End of the Enterprise IT
"It was a bold move for the Netherlands bank. The leaders were giving up their traditional hierarchy, detailed planning and “input steering” (giving directions). Instead they would trust empowered teams, informal networks, and “output steering” (responding to feedback) to move the bank forward. The bank was not in trouble; it did not really need to go through such a dramatic change."
"Despite the challenges, ING Netherlands views its new organizational structure as a significant success with sizable benefits to the company. The strategy adopted by ING Netherlands – an organizational structure composed of small, integrated teams, along with an emphasis on simple customer journeys, automated processes, and highly skilled engineers – is expected to spread to other parts of ING Bank."
Did your company start a "Digital Transformation" in the last decade?  Many did - I couldn't ever tell what that meant... I've been doing digital stuff since the 1980s ... that transformation happened when we adopted the Personal Computer and put a PC on everyone's desk.  I remember teaching people how to use a mouse (it wasn't a natural concept).

Digital Dreams that never came True

HBR has a warning: "Why So Many High-Profile Digital Transformations Fail" - March, 2018 from companies such as:  GE, Burberry, Nike, Ford, P&G, and my fav LEGO.
"With innovative information technology, however, executives sometimes lose their rational decision approaches. Certainly it’s true that in times of radical technological change there’s a lot of figuring out to do.  Executives have to understand what new technologies can do, and understand their impact on markets, products/services, and distribution channels.  These decisions are inevitably influenced by hype from vendors and the media, expensive consultants offering “thought leadership” insights, many high profile experiments, and a few exciting success stories that keep people wanting more. A charismatic CIO or Chief Digital Officer may make it even harder to be level-headed in those heady times."

  See Also:
Before Scaling up, Consider...

Agile Failure Disrupting Organizational Performance - by Agilitrix

Agile Succeeds 3X more than Waterfall - CHAOS Report 2011 - MountainGoat Software
ObjectMentor success stories:
    Sabre takes extreme measures - ComputerWorld

"Led by IS&T’s vice president, John Charles, the ambitious reorganization began in February 2015 and aims to spur innovation through agile software development practices adopted from industry. Charles emphasizes that this is not a typical reorganization, but rather a complete transformation of MIT’s IT department."

The famous SalesForce all-in, one day transition
    Slideshare presentation   The Development Dilemma PDF

Scrum at Scale Keynote 2015 (Scrum Inc.) - has infographic from 2015 Chaos Report

A Case Study of SME Web Application Development Effectiveness via Agile Methods  by: Peter Clutterbuck, Terry Rowlands and Owen Seamons University of Queensland, Brisbane, Australia

Agile Delivery at British Telecom  by Ian Evans, British Telecom  (Methods and Tools)

Live from LA Success Tour: 5 Stories of Agile Success (RallyDev)

LeSS Case Studies (over a dozen, mostly European companies)

Microsoft's agile case study - would you believe it? by Steve Denning of Forbes

Whole Foods Market case study Home Grown Agile - An Agile journey with one of America’s most trusted brands.

In 2016 there is a preponderance of organizations stating they are doing Digital Transformation.  What could this term mean?  Are they changing from analog to digital systems -or- are they mutating all employee numbers from arabic to binary?

Southwest Airlines’ Digital Transformation Takes Off - FastCompany
Rolling out new software is hard in any company, but it’s even harder when your employees still send faxes.

David Anderson suggest Agile is Costing You Too Much - makes the case that there are few if any Agile Case studies and publishes 8 Kanban case studies.  I loved the Sabre story David tells of circa 2006 Sabre reboot of "agile", I lived through a 2011-2013 reboot at Sabre.  They are continually rebooting that initiative - pondering why?
Post a Comment

Most Popular on Agile Complexification Inverter

David's notes on "Drive"

- "The Surprising Truth about what Motivates Us" by Dan Pink.

Amazon book order
What I notice first and really like is the subtle implication in the shadow of the "i" in Drive is a person taking one step in a running motion.  This brings to mind the old saying - "there is no I in TEAM".  There is however a ME in TEAM, and there is an I in DRIVE.  And when one talks about motivating a team or an individual - it all starts with - what's in it for me.


Pink starts with an early experiment with monkeys on problem solving.  Seems the monkeys were much better problem solver's than the scientist thought they should be.  This 1949 experiment is explained as the early understanding of motivation.  At the time there were two main drivers of motivation:  biological & external influences.  Harry F. Harlow defines the third drive in a novel theory:  "The performance of the task provided intrinsic reward" (p 3).  This is Dan Pink's M…

Elements of an Effective Scrum Task Board

What are the individual elements that make a Scrum task board effective for the team and the leadership of the team?  There are a few basic elements that are quite obvious when you have seen a few good Scrum boards... but there are some other elements that appear to elude even the most servant of leaders of Scrum teams.

In general I'm referring to a physical Scrum board.  Although software applications will replicated may of the elements of a good Scrum board there will be affordances that are not easily replicated.  And software applications offer features not easily implemented in the physical domain also.

Scrum Info Radiator Checklist (PDF) Basic Elements
Board Framework - columns and rows laid out in bold colors (blue tape works well)
Attributes:  space for the total number of stickies that will need to belong in each cell of the matrix;  lines that are not easy eroded, but are also easy to replace;  see Orientation.

Columns (or Rows) - labeled
    To Do
    Work In P…

Exercise:: Definition of Ready & Done

Assuming you are on a Scrum/Agile software development team, then one of the first 'working agreements' you have created with your team is a 'Definition of Done' - right?

Oh - you don't have a definition of what aspects a user story that is done will exhibit. Well then, you need to create a list of attributes of a done story. One way to do this would be to Google 'definition of done' ... here let me do that for you: Then you could just use someone else's definition - there DONE!

But that would be cheating -- right? It is not the artifact - the list of done criteria, that is important for your team - it is the act of doing it for themselves, it is that shared understanding of having a debate over some of the gray areas that create a true working agreement. If some of the team believes that a story being done means that there can be no bugs found in the code - but some believe that there can be some minor issues - well, …

What belongs on the Task Board?

I wonder about these questions a lot - what types of task belong on the task board?  Does every task have to belong to a Story?  Are some tasks just too small?  Are some tasks too obvious?  Obviously some task are too larger, but when should it be decomposed?  How will we know a task is too large?

I answer these questions with a question.  What about a task board motivates us to get work done?  The answer is: T.A.S.K.S. to DONE!

Inherent in the acronym TASKS is the point of all tasks, to get to done.  That is the measure of if the task is the right size.  Does it motivate us to get the work done?  (see notes on Dan Pink's book: Drive - The surprising Truth about what motivates us) If we are forgetting to do some class of task then putting it on the board will help us remember.  If we think some small task is being done by someone else, then putting it on the board will validate that someone else is actually doing it.  If a task is obvious, then putting it on the board will take vi…

Team Performance Model - by Drexler and Sibbet

Many of you have all heard of the Tuckman model of team dynamics (Forming, Storming, Norming, Performing).  It was created in 1966 and has become the most popular model for describing team behavior.  Is it time to level up in your mental model of team dynamics?  Are you ready for a richer more functional model?

Introducing the Team Performance Model by Drexler and Sibbet

Orientation - Why am I here?
"Orientation is about understanding the purpose of a team and assessing what it will mean to be a member.  you need to understand the reason the team exist, what will be expected of you and how you will benefit from membership.  In a new team, these are individual concerns, because the group is only potentially a team.  that is why these concerns are illustrated as occurring in your imagination at an intuitive level.  As a team leader it is important to provide time and space for people to answer these internal questions themselves."

Keys to when Orientation challenges are resolve…