The Berl report into the social costs of alcohol being used by the Law Commission is work that doesn’t look like it meets the “normal standards you would expect”, according to Deputy Secretary of the Treasury Dr Peter Bushnell.
Economists Eric Crampton and Matt Burgess have previously blogged their analysis of the report, as well as publishing a working paper for Canterbury University harshly criticising the methodology and conclusions of the report, to which Dr Bushnell has added his voice.
“I think the points they’re making are sound about adding the costs of production into the cost of it, and not counting any benefits. In a market if you’re selling something that people are prepared to pay for, then they’ve at least got that much benefit, otherwise they wouldn’t have bought the stuff. So if you exclude the benefits then you’re clearly only looking at one side of the story.”
“I can see the point being made in the article – it looks pretty shonky” said Dr Bushnell. “I think the fact that some work’s done that academic review says is pretty shonky is a problem by itself. But the question is what’s the best way to solve that?”
He agrees that an alcohol cost analysis Treasury did in 2002 is more consistent with Dr Crampton and Mr Burgess’s findings than the Berl report, which was jointly commissioned by the Ministry of Health and ACC.*
However, the mere fact Law Commission president Sir Geoffrey Palmer is seeking out economic advice is positive, “because in the past lawyers often assumed that economics had nothing to do with it.”
That said, the onus should be on the Law Commission to be rigorous Dr Bushnell said.
“Sir Geoffrey’s reputation is reduced [if] he’s putting weight on something that actually doesn’t stack up. So the Law Commission ought to ... build in processes that give adequate QA and so on.
“What we’re saying is it’s your reputation that’s at risk here. It doesn’t reflect well on the Law Commission if it ... backs [work], that doesn’t have a sound basis.”
Dr Crampton goes further. He believes the Berl report belies a more general problem with economic consultancy reports, in that there needs to be somebody looking at the Requests For Proposals (RFPs) that a ministry sends out, and checking the results when they come in.
“We really need to have Treasury vetting RFPs going out from the Ministries checking that they’re spec’d up to actually measure what they need to measure, and Treasury vetting the reports coming back. Absent that, nobody does it, folks take reports at face value and policy is not formed on sound bases”, Dr Crampton said.
But there are two main reasons nobody does this: the primary one is a lack of resources.
“Treasury just doesn’t have the resources to look at every RFP that goes out and then check the results when they come back”, Dr Bushnell said.
The secondary one is the need to maintain the independence of bodies such as the Law Commission or Commerce Commission.
“There’s sort of a question as to what are the processes that respect that independence, which is what you set them up for, but also ensures that you’ve got enough rigor and quality in the advice”, Dr Bushnell said.
“But I don’t think we can expect to have a body set up that’s going to examine every economic claim made by any organisation in the public sector.”
Sir Geoffrey was overseas when contacted by NBR, and has declined to comment on the matter thus far.
*An earlier version of this story incorrectly attributed the commissioning of this report to the Law Commission rather than the Ministry of Health and ACC. NBR regrets the error.