aibl Research: Advice from 1999

Last week we talked about why AI projects break, and I argued that the published research is easy to misread. Now let’s address the harder problem. Once everyone accepts the project has failed, somebody has to decide what happens next, and that decision is where organisations do themselves real damage.

Few pieces written about AI failure engage with the research, because most of it predates large language models by 25 years, so it doesn’t turn up early in a search and suffers from our digital recency bias when it does.

Mark Keil at Georgia State spent two decades on what he called escalation of commitment: the pattern where organisations keep funding a project long after the case for it has gone. The counterpart, de-escalation, is the process of breaking that cycle. The best entry point is Montealegre and Keil’s 2000 study of the Denver International Airport baggage system in MIS Quarterly, or their practitioner write-up for MIT Sloan Management Review, which draws on more than 40 cases of IT projects that ran away from their owners.

Their central finding is that de-escalation runs in four phases: recognising the problem, re-examining the original course of action, searching for alternatives, and implementing an exit. Organisations often skip the second phase. They go straight from “this is failing” to “what do we build instead”, and that’s a mistake.

Four things from the ancient literature that belong in any AI post-mortem

De-escalation means redirection at least as often as termination

Keil and Robey define it as reversing commitment to a failing course of action, either by killing the project or by significantly redirecting it.

If you frame the meeting with the binary of kill or continue, it forces everyone into camps and gives the original sponsor nothing they can accept without conceding they were wrong. Put the middle options on the table explicitly and early: narrow the scope to the single workflow where it demonstrably works, rebuild the data foundation and defer the model, or replace the model with the simpler deterministic system. Someone senior needs to champion that last one or nobody will take it seriously.

This middle road frames the argument around the work, not the people.

Kill criteria should trigger a review, not a cancellation

Cancelling a project signals that the original funding decision was wrong. Absent a process, the political cost of raising cancellation falls entirely on the individuals who will bear the cost, which is why projects sit at “amber” for six months while everyone waits for things to improve or someone else to speak up.

A pre-agreed, measurable condition that automatically convenes a formal review moves things from a person (politics) to a calendar (operations). Set the criteria at the start, when nobody is invested yet, and make the default at each review gate “pause and reassess” rather than automatic continuation.

Some people will be against it from the jump

RAND found that subject-matter experts sometimes offered passive resistance to AI projects because they believed the projects existed to replace their jobs.

Passive resistance won’t jump off the page, but it frequently decides the outcome anyway, because these are the people whose domain knowledge determines the training data and its definitions. That data structure will still be there for whatever you build next, which makes it an expensive problem to leave undiagnosed.

The traditional way to avoid this problem is to have two or more experts independently define the data and its parameters. That drives good work from everyone.

Protect whoever raised the alarm

Keil and Robey wrote a separate paper on this, “Blowing the Whistle on Troubled Software Projects”, in Communications of the ACM in 2001.

How your organisation treats the first person to say that a project is failing sets the bar for every project that follows. If that person is ignored, moved sideways, or even pushed out, the next failure will take longer to surface and be more painful. This kind of action is never going to be a policy, but an organic event, so it’s on senior leadership to notice and on executive leadership to make sure senior leadership isn’t the problem.

One thing to do differently

Before the post-mortem starts, ask everyone in the room to write down privately whether they think this was a basic failure, a complex failure, or an intelligent failure.

I’m cribbing these categories straight from Amy Edmondson’s Right Kind of Wrong. A basic failure is a known process done badly. A complex failure is several small things going wrong at once in a familiar setting. An intelligent failure is a well-designed bet in territory where nobody could have known the answer in advance. That last one deserves a completely different organisational response from the other two.

Take a distributor building a demand forecasting model.

The basic failure: the training data included a 2020 demand spike and nobody excluded it. The model internalises a surge that isn’t coming. Holding out anomalous periods is a known step that was skipped, and it’s a straightforward post-mortem and fix.

The complex failure: the model was sound. But a major supplier changed its lead times a month into the pilot, and the one planner who understood how the SKU hierarchy had been reorganised in 2023 left in March. Neither would have sunk it, but together they meant the forecast was never used in a state where it could have been judged.

The intelligent failure: you bet that promotional lift could be predicted from three years of historical data. But the promotions were never randomised, they always ran in periods the business had already picked as strong. The historical record cannot separate the promotion’s effect from the reason it was scheduled. It wasn’t obvious without building the thing and looking, and now you know something valuable about your own data, and how to approach the problem next time.

Most internal fights about a failed AI project are a disagreement about which of the three occurred, even if we don’t realise it. Getting the answers on paper before anyone speaks turns a blame game into a question that has an answer. It takes about five minutes, and it is the highest-return time in the whole process.

Sources

If you’d like to read further yourself, here are the sources I consulted. Most of them are behind academic paywalls, but you can get the abstracts for free.

Montealegre, Ramiro, and Mark Keil. “De-escalating Information Technology Projects: Lessons from the Denver International Airport.” MIS Quarterly 24, no. 3 (2000): 417–447. Primary academic source for the four-phase model.

Keil, Mark, and Ramiro Montealegre. “Cutting Your Losses: Extricating Your Organization When a Big Project Goes Awry.” Sloan Management Review 41, no. 3 (2000): 55–68. The practitioner translation, drawn from 40-plus escalation cases over eight years.

Keil, Mark, and Daniel Robey. “Turning Around Troubled Software Projects: An Exploratory Study of the De-escalation of Commitment to Failing Courses of Action.” Journal of Management Information Systems 15, no. 4 (1999): 63–87. Source of the termination-or-redirection definition.

Keil, Mark, and Daniel Robey. “Blowing the Whistle on Troubled Software Projects.” Communications of the ACM 44, no. 4 (2001): 87–93.

Edmondson, Amy C. Right Kind of Wrong: The Science of Failing Well. New York: Atria Books, 2023. The basic / complex / intelligent taxonomy. Her HBR interview, “It’s OK to Fail, but You Have to Do It Right” (July 2023), is the free short version.

Read but didn’t cite

Keil, Mark. “Pulling the Plug: Software Project Management and the Problem of Project Escalation.” MIS Quarterly 19 (December 1995): 421–447. The foundational paper.

Keil, Mark, Joan Mann, and Arun Rai. “Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models.” MIS Quarterly 24, no. 4 (December 2000): 631–664. Cite this if anyone argues escalation is anecdotal.

Smith, H. Jeff, Mark Keil, and Gordon Depledge. “Keeping Mum as the Project Goes Under.” Journal of Management Information Systems 18, no. 2 (2001): 189–227. Companion to the whistleblowing paper, and arguably more useful, since it models the individual decision to stay silent rather than the organisational response to speech.

Pan, Gary, Shan Pan, Michael Newman, and Donal Flynn. “De-escalating IT Projects: The DMM Model.” Communications of the ACM 52, no. 10 (2009). Source of the commitment-transformation sequence behind the recommendation on protecting the first person to speak.

Klein, Gary. “Performing a Project Premortem.” Harvard Business Review, September 2007. The right citation for a general readership.

Boston Consulting Group. From Potential to Profit: Closing the AI Impact Gap, 2025. The 10-20-70 principle.

Richard