The home page of our website includes a heading that reads “There is more to a successful project than finishing on-time and on-budget.” This resonates with many IT leaders, project managers, and project team members, but what does that really mean? I’ve managed a lot of new technology implementations, and noticed that there is a tendency to measure project success soon after the implementation is over. While this might be an appropriate time to evaluate the implementation and/or planning efforts, typically it’s too soon to be able to gauge the success of the project as a whole. Since their focused involvement often ends with the implementation, it’s easy to see how project managers (PMs) and vendors can be the worst culprits of this premature determination of success. Here’s the normal progression I’ve observed: After the flurry of a go-live, end users’ calls eventually slow to a trickle, and support transitions back to the Help Desk. The vendor transitions support from their sales and implementation teams to standard support. The PM evaluates how the implementation compared to the plan, calculates the variance, completes the close-out report and moved on to the next project. Sound familiar? If so, you too might be wondering: If the project is live and the users have stopped calling in for help, why is it too soon to call the game a win? Especially if you’re able to monitor usage and can tell the new technology is in use? To answer this, I want to borrow an analogy I heard in a TED Talk by Rachel Botsman (the talk’s subject is unrelated but totally interesting – highly recommended!). Beginning with the statistic that the average power drill is used for only 12 to 13 minutes over its lifetime, Botsman questions whether it makes sense for individuals to buy their own power drills and other tools, instead of perhaps sharing or renting. She reminds the audience that although it’s easy to focus on getting a drill, “what you need is the hole, not the drill.” Let’s apply this to our earlier question about why it’s premature to evaluate the overall success of a new technology right after implementation. There are always exceptions, but most of the time companies are willing to invest in implementing new technologies for the same reasons individuals buy power drills: both are tools to help them accomplish something else; the technology and drill are means to other ends – not the ends themselves. If the goal truly is just to launch a new technology (i.e., own a power drill), then the success criteria of finishing on-time and on-budget may be enough. But if your goal is something that the new technology will help you do such as increase efficiency, improve service delivery, or lower operating costs (i.e., put a hole in the wall), then measuring project success immediately after implementation is akin to evaluating how well your new drill looks in the box on the way home from the hardware store. So how do you measure a project’s success? Every project has some elements of success and failure, but to truly evaluate the overall success of a project, you need to assess the project’s value to the organization. In other words: Did you get the hole that you needed? The business stakeholders are the only ones who can really answer this question, but you can (and should!) help them do this by asking some key questions before the project ever starts: 1) For each stakeholder group, what are the business goals they hope this project will help them achieve? 2) How do these align with the overall goals of the organization? (If they don’t, then it’s time to question why you are doing the project.) 3) For each of the business goals, what are the measurable objectives or outcomes? For example, reduce expenses by X%, improve response times by Y%, reduce errors by Z%, etc. 4) By when do these goals and objectives need to be achieved? It may be three months, six months, or a year after implementation before you can evaluate whether the goals were achieved or not. Be sure to have a process in place to evaluate the results after the project has officially closed. 5) What steps will you take to ensure end user adoption of the new technology and/or processes the project will introduce? (Our post, “What the IRS Can Teach Us: 5 Steps to End User Adoption”, is a great resource if you are looking for tips on achieving end user adoption.) If you’re the PM, be sure to include the answers to these questions as success criteria in the project charter. Pay special attention to #4 – especially if the results will be evaluated after you close out the project. Put simply, IT leaders cannot know if a new technology was worth the investment without evaluating how well it accomplishes the goals, and that often can’t be captured in the project’s final status report.
Measuring the Success of New Technology Implementations. And Power Drills.
