Why aren’t we better at this?

In one of my conference talks, I talk a little about how we, as an industry, are pretty terrible at what we do. One of the slides I show is this summary of the 2020 Standish Chaos Report:

I’m currently reviewing a book manuscript where these same numbers are presented, and they continue to bother me. I won’t steal any of the thunder from the author, BUT an overall success rate of less than 50% for “agile” projects is pretty terrible.

We get a D

Is there any other industry / profession where a success rate of < 50% is acceptable?

No one goes into a project thinking it’s going to fail, BUT how often are projects started without clear outcomes in mind? How often is a project started under one “sponsor” and then inherited by another who doesn’t have the same vision? That’s gotta have an impact, right?

Are the projects we’re taking on too large? Are we biting off more than we can chew? Are developers being asked to do too much? Does the “business” just have no clue about how to run projects?

Is it “all of the above?”

I won’t even get into the idea of “estimation” because we’re terrible at them.

Success or Failure

I’ve been a part of successful projects, and I’ve been a part of failed projects. I’ve been involved in death marches, and I’ve been involved in projects where everyone absolutely had their shit together, features were implemented and shipped, tests were written, and people went home at a reasonable time each day. I’ve been involved in waterfall projects and agile projects. I’ve also been involved in “digital transformations” and “SAFe implementations.”

The best, most productive teams I’ve worked on, the ones with the most success, were self-organized teams. They followed the process that worked best for THAT team. The software we shipped was high quality with few defects. Those teams didn’t have frequent fire drills due to production issues. Those teams had solid business stakeholders who were willing to collaborate and compromise about timelines and features, but who were also very clear on their vision and the outcomes they expected.

Some of the worst, least productive teams I’ve worked on, the ones with dysfunction, failure, and frequent fire drills…yeah, those followed processes pushed down from “on high” and the developers had very little say in even their day-to-day work. The vision from stakeholders was muddy, but the demands were clear; work faster.

I know it’s possible that the good teams produce garbage and that the garbage teams product good software, but it’s not probable. There’s something to be said for a tight, well-oiled team that has gelled and that has good leadership.

There’s something to be said for project sponsors and technical management that have a clue, people with integrity willing to push back on insane deadlines, impossible timelines, and 2+ year backlogs. I think that’s where I see the biggest gap. I believe most developers want to solve problems and work on successful projects, but sometimes (most?) the direction they are given from leadership asks them to do too much with too little, whether that’s too little time, too few people, too little training, requirements that are vague, or a myriad of other issues.

Final Thoughts

I don’t have any answers, and I’ve been doing this for a long time. I, along with many technical people, do the best we can with what we have. I know I push back on things when I think it’ll help, but most times, the decisions come from way above my pay grade, so I’m left trying to figure out how to fit 20 weeks of work into 12.

There are days I feel like Detective Murtaugh from the Lethal Weapon movies

I’m too old for this shit

Do you have any thoughts about why success rates for software projects are so low? Do you have ideas for how we can make things better?

---