Critical Success Criteria

How Do You Know When You're Done?

Cover Image

Writing software takes time and this is at immediate odds with every single project manager. As soon as there is a project there is someone who wants it done now. One of the most important skills in sustainable development is determining what to build.

It was 2011, I was working at a large company and the Group Product Manager (GPM) had just committed to the VP of another team that we would finish our project in three months….and we had previously scoped the project to take six. I circled my pod and explained that since we had a limited amount of time we had to pick apart the specifications, identify the top-priority (P0) features, and work on those first. It was when I reviewed the checkin requests later that week that my jaw dropped.

The development team had gone into full-on panic mode. All the progress we had been making towards Agile development melted away as people fell back to what they were comfortable with. Huge multi-page specs were written and developers settled down to waterfall the heck out of this project.

One of the developers on the team had chosen as her very first task to implement the footer links on the bottom of the page. Not an entirely unexpected starting task as the web page needs a place to link to legal text, support, etc. What shocked me was the footers were implemented dynamically: An XML file defined the footer links and was used to generate the page content. So instead of modifying an HTML page we could modify an XML file. This was a lot of effort for something that was not in the critical path of the user experience.

Most software has a purpose. When a user sits down to accomplish a task there is a clear point at which the user considers their effort a success. At an ATM, the user is attempting to withdraw money and once they have bills in their hand they are done. Everything else is a distraction; if the user had their druthers they would walk up to a machine and it would magically spit out money1.

The site we were building allowed customers to submit applications to a catalog. The customer’s task was complete when they had submitted their application for approval. Not a single step in the submission workflow involved the footer links. The meta success-criteria for our team was an internal beta that showed the submission process end to end—no one external to the company was going to see these footer links!

Any distraction that delays the customer from reaching the critical success criteria is probably inconsequential and very likely an annoyance to the user, making it a very good candidate for postponing as long as possible. On the same project I sat in two-hour design-review meetings with a couple of dozen people talking about how the hint bubbles were going to appear and disappear next to the forms. A man-week of effort lost arguing about pixel placement.

In some cases the distractions are necessary or beneficial. For ATMs, you are required to enter your PIN; this distraction is for security purposes and one people are willing to tolerate to keep their money safe. I remember when some ATMs started returning your card before giving you the cash. This added “distraction” actually acknowledged the propensity of a person to focus solely on the success criterion (receiving money)—people were leaving the ATM without their card. By swapping the order of operations the software could ensure a greater overall success of the transaction. As it turns out, there are multiple success criteria for ATM withdrawals:

  • Get money
  • Keep money secure
  • Leave with card

What features are in the critical path of your users? What features are getting in the way of your users completing their goal? Stop working on the latter and spend your time on the former, you’ll be surprised how much more time you have. A user can use an ugly application that works; a beautiful application with no functionality is simply a piece of art which—unless you are a museum—isn’t going to make you a whole lot of money. And if you get pushback from a product manager just ask the magic question:

“Are you willing to delay the release for this feature?”

For more on this look into methodologies like Lean Software Development and ideas like Minimum Viable Product (MVP). They are very good at deferring all but the most critical tasks.

1 For simplicity sake I am referring to a single use case. Of course there are many more reasons why people use an ATM.