Poorly Defined and Changing Objectives and Requirements

If we look at the data, we can see that the top 4 reasons can be summarized as:

1. The Objectives and Requirements changed over the life of the project
2. The Objectives and Requirements were inaccurate or incomplete in the first place

Number 7 on the PMI list directly relates to the inability to manage those changing objectives and requirements.

Lack of Risk Management

The fifth most common cause of Project failure is lack of risk assessment/management.  To be honest in all my years of doing large IT projects, I would be hard pressed to come up with a single case where someone ever asked the question “What happens if this project fails?” at the start of a project.  In fact, it is much more likely that there will be an “OMG we are screwed!” moment after the writing is already on the wall and the ship is rapidly heading for the bottom of the ocean.  This is when I often get a call from clients who are hoping and praying that I can come in and save them.  It’s usually too late at this point in my experience, but if they are prepared to knuckle down and make changes then usually something can be salvaged. Unfortunately, they are often not able to make the necessary changes, even when they are sinking.

Inadequate Estimation and Resource Management

Five of the remaining items can be classified as relating to issues with Estimation and Resource Management.

There is no doubt that estimation is hard for most projects unless they are fairly small and/or you have done the same or similar project many times before with very little deviation (which is true in some industries).  The data and my personal experience indicate that the larger and more complex the project is, the less likely your estimates are to be accurate.

In fact, estimates should include a confidence level, which implies a range for those estimates.  Unfortunately, people rarely specify or quantify the confidence level or the range for those estimates.  Even if they did, executives usually forget that the estimate came with a confidence level and range and only remember one number.  I have never seen someone budget $50 million plus or minus $50 million, which might be as good as an estimate can get when initially proposing a project.

Over time estimates should improve and the range should tighten, but too often they don’t.

Going hand in hand with estimation is resource management.  When was the last time you estimated a project and set aside a budget, based on not only an estimate but a detailed resource plan as well.  I can tell you that in my experience doing large projects in the Financial Services sector, I don’t think I have ever seen it.  The assumption is that we will somehow find the resources as needed.  That usually doesn’t work too well.

Moreover, when you are into the project and you estimate the line items, you should also be assigning resources to them and making sure you have enough resources to get them done and that you aren’t just assigning everything to a few people.  In my experience, there is always a few people that shoulder a much larger share of the work than the average project member.  While you should cherish these people, it is not a good way to ensure project success and usually guarantees that you will not deliver on time or budget.   

Lack of Executive Support

This one I see a lot and it always surprises me when I see it, because if this is in play, then even a perfectly executed project is still a failure.

So what does lack of Executive Support mean?  Well to be blunt it means that someone decided to initiate a project that the business doesn’t care about and doesn’t think they will realize any actual value from.

It’s really simple, if your project isn’t directly solving a problem that the business has, or isn’t opening up a new opportunity for the business, that they agree with and value highly enough to authorize and properly support the project, then you just shouldn’t be doing it.

Organizational Resistance

This one is not from the PMI’s list, but is something I have seen in my personal experience time and time again.

Imagine the case where the business wants to implement a project that will dramatically streamline operations and improve efficiency within the company.  Now consider that with that efficiency will also come head count reductions.

What do you think is going to happen?

In my experience the person who currently runs the part of the organization that is going to be most affected by that project is usually going to be incented to make sure that project doesn’t succeed.  Moreover, that same person is most likely the leading expert in the company on that area of operations and the main expert whom they need to rely on in order for the project to succeed.  You know that project that is going to make the expert (and probably their whole department) redundant and out of a job.

I have seen this exact case many times and what happens is the person who is most affected, and their team tend to delay, stall and put up road blocks, not overtly, but passively.  They will then simply wait out the project, knowing that without their cooperation it will run over budget, over time and will likely be cancelled when the executive sponsor inevitably loses their mandate for the initiative.

IT Project Specific Causes

Moving over to the IT world where the team at Proceve have spent most of their time, we find a few other common causes of failure that are unique to IT projects.  Many of these also apply with minor variations to other types of projects as well, but I will leave it up to you to map these issues to your specific domain.

Skills Mismatch

Often I see companies, specially when they are trying to modernize some software, assign one of their legacy employees to run the project and make the key decisions.  While on the surface this might not seem too bad, too often that person has no experience in the new technologies or best practice for implementing the new system with the latest technology.

I can’t tell you how many times I see financial services firms assigning a person who grew up maintaining their legacy batch oriented Cobol systems on a mainframe and make them the key decision maker designing and implementing a real-time high performance, service oriented architecture for a $500+ million dollar project.  Typically the person ends up making the new system exactly like the old system just with a new programming language.  It never works correctly and never delivers the value that the business hoped for.

This is a pure case of the person being in charge not having the correct skill set for the job.

This often then trickles down to the individuals executing the plan.  Often the people working on the project don’t have the domain knowledge to execute effectively or they are learning on the job (which is inefficient).

Another example I experienced recently was a large project that outsourced the QA function for a complex software build.  The firm doing the QA was staffed with hundreds of people who had absolutely no knowledge of the business domain, the nature of the software being developed or even an understanding of the basic requirements they were supposed to be testing.  The result was that they were not able to do the job and the BA’s had to take up the slack and do the testing before it even went to QA because there was no faith in QA’s ability to test.

Improper Technology Selection and Architecture

Often going hand in hand with the skills issue, I see projects where the technology stack that was selected and the architecture that was designed are not able to meet the business requirements.

A simple example of this is that often people ignore the non-functional requirements when performing this selection, so they end up with an implementation that cannot perform well enough at scale or under load no matter how much hardware they throw at it, or the operation cost, due to those same inefficiencies, ends up being prohibitively high.

Probably more common, and directly related to the biggest cause of failure listed above (the constant change in requirements) is an architecture that is not sufficiently flexible and thus can’t accommodate those requirements changes.  In my experience, this is most often manifested in the data and API designs.  If the architecture is too rigid, then it is often impossible to make the changes needed to meet the new requirements or it becomes very expensive (time and money wise) to make those changes.

Conclusion

So while I focused on the data that the PMI published, I must say the same causes of failure have been found by pretty much everybody who has ever studied these issues.  Moreover, these findings are consistent with my personal experiences from over 4 decades and about $30 Billion dollars worth of projects. 

In coming articles, we will look at some of these reasons for failure in more depth and I will try to convey some specific observations about root causes and possible ways to mitigate them, based on my experience.