Early lessons from Novopay

Several inquiries into Novopay, of varying independence underway, are currently underway and it’s going to be interesting to see these outcomes. But what are some of the lessons we can already learn from Novopay, especially in the context of the proposed $1.5 billion dollar IRD spendup? Here are the first two. [UPDATE: Review says Novopay can be fixed]

It’s important that we resist the urge to rush to judgment on the Novopay situation; it’s likely many more details are yet to emerge before the picture is completely clear. While the documents released recently by the government certainly paint a fairly damning picture, that’s only half the story.

However even at this early stage, some things are clear. For starters, it’s fair to say that the project launch has been an abject failure, certainly in the eyes of the users. Even if it’s fully salvaged now and regardless of where blame eventually sits, the fact is the launch will be used as a case study in future IT courses for many years to come.

There are also some IT project lessons we can learn for future projects such as the upcoming major IRD revamp, that perhaps could even help avoid some seriously massive problems in that and other large-scale taxpayer-funded projects in the future.

Lesson 1: Does the tender process lead to bad results?
Someone asked me the other day whether it was clear yet at what point the project first came unstuck and in our view, it was probably before it even started; likely before the tender had even been won when various companies were jostling over the project.

Why? We’re talking about a very complex and complicated project. Not only are there more than 100,000 people relying on the system, there is also massive complexity in how these teachers and support staff work with many schools doing things in different ways, part-time teachers working across multiple schools, big variations in pay and contract rates and much more. You could say “yes, but it’s just a payroll system…” but it really ain’t that easy in this case.

So what do you do if you're tendering on a hugely complex and complicated project like this one? Well you basically have two options.

You either spend a considerable sum - probably in the hundreds of thousands or more - working through the requirements in absolute detail and properly spec the whole project out, all the while knowing that there's still a very high chance you'll lose the tender and thus have to write off that time investment. And, of course, you have to build the ever-growing cost of this work on unsuccessful tenders into the price of successful ones.

Or option 2, you basically “wing it”; you still spend a bunch of time, but only end up with a rough idea of how complex you think the project will be, cost that out, build some fat in for complexity you missed, put in a bid and cross your fingers.

I’m not saying this is what Talent2 did in this case at all, but it is clear the complexity was completely under-rated, and one does wonder if the incumbent lost the contract because they knew full well the complexity involved and quoted based on that – being undercut in the process. But I digress.

While a simplification, those are the two options in front of most companies bidding for complex projects in Government, and neither lead to a good result. From that perspective, it’s excellent that Minister Joyce has announced there’ll be a review into how future projects are tendered.

While price is always going to be important, treating it as a key upfront deciding factor in procurement demonstrably leads to lower quality outcomes. There are other ways of doing it such as “Qualifications-based Selection” or QBS model, which is a process that only considers price once a preferred supplier has been found based on other criteria. The US Federal Government has been using QBS for “hard” Engineering since 1972. It makes sense really – would you like to be the official who explained that the bridge fell down because they chose the “cheapest” provider? Not a good long-term career prospect.

But what are the implications for the upcoming IRD system? For one, we have to accept the fact that the larger and more complex a project is, the more chance it’ll fail and when it does, the bigger a mess it will make.

So when we’re talking about billion dollar projects, the chance of failure is huge and the cost fallout could be felt for many years to come.

The logical answer to any IT person with even a semblance of experience is simply to chop it up into a series of far smaller projects or components, focusing on how each component interrelates. While this approach might take a little more work up front, the risks are far lower than what appears to be the current “big bang” thinking of just farming the whole project out to some offshore company and hoping for the best.

Under the component approach, if some projects fail the fallout is contained and an appropriate plan b put in place. To put it another way, if one company is handling the lot, you can’t just kick them to the curb and get someone else to do it without massive cost. With smaller projects you can.

But more to the point, it doesn’t exclude capable and incredibly competent local providers from what will likely be the largest Govt ICT spend-up in a while. They’re paying for it through their taxes. Rather than sending the work offshore, the Government could back New Zealanders and use this project to truly transform our industry and position it to take on the world, in a way that dramatically reduces risk.

As I say, it’s an obvious approach to IT people. But will the Government see sense?

Lesson 2: Launching to 100% capacity on day 1
Say what? Here we go again. All IT and project people should understand why that was a bad idea, yet by all accounts that’s exactly what happened in this case. Again.

Rather than a staged release and picking, say, 100 schools the first run, putting them across ot the system then ironing out the teething issues in a managed and manageable way before taking on more, someone just flicked the switch and hoped for the best.

And predictably, boom! Some relatively minor teething issues occurred but because they were at maximum capacity on day one, guess what happened? The issues were at a scale they couldn’t manage and their support service was totally overwhelmed, mainly initially over missing payslips of all things. Manual processing was required, which takes time and causes audit issues and they really never caught up from there.

So what the reviews should be looking at is why and when the decision was made to push go on 100% capacity on day 1 – that was asking for it. Was it made as a result of political pressure to “go live at all costs”? Was it pressure from the incumbent? Or was it simply an “acceptable risk” from those who should have known better?

There was originally intended to be a staged release, or at least a pilot programme, of just the South Island. It was scrapped, presumably because the decision-makers had such complete faith in the system that it wasn’t needed…

Conclusion
So there certainly are going to be many lessons learnt from Novopay and those two are barely scratching the surface. However neither is new, and one does have to wonder why they hadn’t been learnt before. And if we already knew this stuff, will we just continue to make the same mistakes into the future?

Let’s just hope that if our government does throw a billion dollars into revamping our tax system kiwis get a chance to do it, and do it right, rather than being cut out of the project altogether. And let’s hope they don’t make the same mistakes all over again, at massive cost to New Zealand and kiwi taxpayers.

Paul Matthews is chief executive of the Institute of IT Professionals NZ.