Online property valuation models: How accurate are they?

Properazzi

Alistair Helm

Alistair Helm

As might have been anticipated, my recent article providing a guide to the current portfolio of providers of online property valuations models triggered the inevitable question – "just how accurate are they?"

So I thought I would do some desk research. However, before I unleash a barrage of criticism stating that there are heaps of examples where the automated valuation models (AVMs) are so wide of the mark to make them laughable, let me simply say this. There are more than 1.5 million AVMs or potential AVMs for New Zealand properties – there will always be outliers and extremes. I do not have the time or patience to review thousands of properties or even hundreds of properties. I chose to select just 12 properties.

The method I used was to track the latest published auction results as the auction year re-started after Christmas. I simply took the first 12 I saw, which comprised eight properties in Auckland and four in Tauranga. So again I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only. The sales are only for those two areas of the country and represented a very quiet period of the year.

With these 12 property sales results I went to each of the five providers:

I knew none of these providers had updated their valuations to take account of any of these actual 12 sales. Neither would the sale records have been picked up through local council sales or agent reporting, so there was no bias of an AVM being influenced by these recent sales.

Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM.

So here is the table of results. The colour code used is blue where the AVM equalled the sale price exactly, red signifies an AVM below the sale price with green where the AVM is above the sale price. Finally, grey indicates that the provider had no AVM for the property.

AVM_analysis_Feb_18.png

As you can see, the visual skew towards red indicates that, based on this sample set, most AVM’s were below sale price. (The original version of this article I used an average variance measure, after receiving valuable feedback I have now used the calculation of gross median error.)

All providers achieved a gross median error of less than 10%, with Realestate.co.nz achieving less than 5%, which is impressive. I would deduce that a factor in its accuracy, is it benefits from the very latest REINZ data each month of unconditional sales, while all other providers rely largely on settled sales, which come through at least a month to two months later.

Another perspective I was keen to examine in respect of the accuracy of AVMs was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed the range as a percentage of the midpoint price.

AVM_analysis_Feb_18.png

This analysis is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes, both just under 10%. This effectively means that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable given this is a computer-based estimation with no detailed knowledge of the specifics of the property.

Of interest in this analysis is the wide margin in the range from Trade Me Property at close on 30% with its tightest range being for a single property at just 19%. Similarly, Realestate.co.nz seem to apply a standard circa 21% to all AVM’s.

Alistair Helm is the former chief executive of Realestate.co.nz and former head of product at Trade Me Property. He has just restarted his independent Properazzi blog after a three-year break.


10 · Got a question about this story? Leave it in Comments & Questions below.

This article is tagged with the following keywords. Find out more about MyNBR Tags

Post Comment

10 Comments & Questions

Commenter icon key: Subscriber Verified

Which all goes to show you need a property inspection and knowledge of the area and the market together with the judgement of a good valuer

Reply
Share
  • 0
  • 0

I was surprised when Auckland Council only increased my valuation by 15% then disappointed when homes.co dropped the value of my house 18% a week later.

Reply
Share
  • 0
  • 0

Alistair, do you have any technical valuation or statistical experience?

It would be great to know this.

You yourself admit that your sampling is not representative so what's the point of your analysis, what are you trying to show?... or what added-value are you trying to provide?

Garbage in garbage out.
#nativeadvertising

https://en.wikipedia.org/wiki/False_premise

Reply
Share
  • 0
  • 0

Jacinder our PM bought a house valued at $1.7 million in Sandringham, Mount Albert an expensive location were $2 million is often achieved.

Labour supporters who expect a cheap half price home from enforced Labour policy will not be impressed with Jacinders extravagant purchase.

Reply
Share
  • 0
  • 0

Well they're not called Chardonnay socialists for nothing. They're pretty much only Labour in name these days.

Reply
Share
  • 0
  • 0

The Socialist ideology to which Jacinda subscribes only applies to everybody else. This is why its fine for her to earn $500,000 from the taxpayer for as long as she stays in the job.

Reply
Share
  • 0
  • 0

This is consistent with the analysis I did when house-hunting. Valuations were generally plus or minus $150k at the $1.2m range.

Reply
Share
  • 0
  • 0

Well done as it just goes to show the fallacy of Valuations per se as Valuers simply do just what the models do and come up what should be the "price" based on comparative analysis.

The actual price paid by a buyer depends so much on an amalgamation of a myriad of small factors that the only test is the sale price realised.Valuers should never put a fixed number on a property but rename their reports to "likely range of selling price" or if they cant do that should have a disclaimer that says"this Valuation is based on analysis of comparable sales however the market can demonstrate behaviour that does not obey historic sales trends"..if they were honest as the willing seller/willing buyer concept is a myth and that theory will never reflect realised prices.Valuers need to get with it otherwise they will become outmoded as an informed buyer will always know more than a Valuer will ever know.Maybe they should just comment and audit the realised sale price which they probably do anyway even if they dont tell the client. .

Reply
Share
  • 0
  • 0

Like any service, you get what you pay for.

Most valuers are rubber stampers, and because they are the majority dictate prices. In using registered valuers, the best advice I would give is ask them how much property they own. The best will have plenty, and it won't be many of them.

Ask yourself, would you prefer to you an advisor who has plenty of skin in the game, or one that effectively operates on commission?

Reply
Share
  • 0
  • 0

A valuation isn't worth anything. Get 3 different valuers to appraise a property and you'll get 3 different valuations. A valuer's "expertise" differs little from any real estate agent worth their salt -- the putative"valuation" is derived from an amalgam of subjective perception coupled with empirical evidence of past and present sales, that may -- or may not -- correlate with the subject property. To quote that head-clutching awful phrase: " At the end of the day", it's what a willing seller and a willing buyer agree upon, that determines a property's worth at any given point in time.
(edited)

Reply
Share
  • 0
  • 0

Post New comment or question

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.