Kent Lardner from Property Data Solutions give us his unique perspective into the world of Automated Valuation Models (AVM) and what techniques the mortgage industry can use to assess their viability.

Back in 2005 a special report to Parliament was presented by the NSW Ombudsman. Titled ‘IMPROVING THE QUALITY OF LAND VALUATIONS ISSUED BY THE VALUER GENERAL’ (Google it to find the report), it offers lenders currently investigating AVMs a detailed insight into methods for testing.

The report indicated “that only 31% of sales on average met the strict 5% margin of error standard across the sample districts and only 66% of sales across these 43 districts were within the 15% acceptable margin of error.” This is a good benchmark for your own AVM testing.

Whilst I will not go into great detail about the different testing methods the government uses, it is important to identify that more than one test exists. Of the 3 methods used; 1) Coefficient of Dispersion (COD), 2) Mean Value Price Ratio (MVP) and 3) Price Related Differential (PRD), the most commonly understood method by lenders is the MVP. Simply put the MVP is the ratio between the AVM result and sale price.

For anyone undertaking an AVM test project, it is well worth taking the time to understand all 3 methods. The object of using these different tests is to see how uniform the accuracy is across all markets as well as measuring the extent to which high and low valued properties are assessed.

What to look out for

Regardless of how ‘advanced’ a model is, a multi million-dollar investment in AVM technology will yield poor results with limited or poor quality data. Many of the models used in the US and UK have been based on highly detailed data sets, including variables such as views (yes/no), property style, age of dwelling, number of stories, number of fire places and so on. In Australia we have a much more restricted data set in most states, resulting in higher errors.

In simple terms, AVMs will be accurate where the data accounts for most of the variation in price. If the input data is limited to dwelling type, land size, bedroom and bathrooms and is used in relatively homogenous markets, it will return more accurate results.

Recently I have begun testing a ‘location score’ based on the position of every property within a local area. This score is calculated on a range of factors, including how busy the road is, proximity to rail lines and so on. Testing has shown this variable to be often more significant than even the bedroom count, bathroom or other quantitative data describing the dwelling itself. Included in an AVM, the specific location of a property will have a significant impact on the result. The important point here is what testers need to look for is what the model does not see.

Tips for testing an AVM

Split the values within a suburb into quartiles. Test the results of the AVM according to the bottom 25% of prices, the middle 50% and highest 25%. What you are looking for is accuracy across all price range(s) you are seeking to apply the AVM to. In practice, you may not want to use automated models for the highest 25% of the market in any location, regardless of price or loan amount. By flagging the upper quartile in your tests, you may find the AVM average error rate for Q1-Q3 improves.

Flag beachside suburbs. Even if views are captured in the data (yes/no), many beachside and harbourside suburbs will suffer lower accuracy rates. This occurs when the sample sets used to calculate the model include higher priced properties in which the view was not measured and captured in the data. You may find that removing beachside or harbourside locations altogether will improve the overall AVM error rate.

Look closely at townhouses and units in locations where these two property types are mixed up in the database. It is common to see townhouses flagged as a unit in the data. The result of this will be to reduce the townhouse price. In terms of risk to a lender, what you are most interested in is over-pricing. Therefore your primary focus in testing is to look at locations where units may be over-valued. This will be in location with a high level of town-house sales mixed in with unit sales.

Flag main roads in your test sample. If the data used by the AVM captures road type, then there will be little or no difference between properties on main roads and those on a quiet suburban street.

Review the valuation method used. If your data sample used in the test includes new loans rather than refinanced loans, AVM providers will have access to the last sale. One of the easiest methods to apply is the last sale price and index it to current market values. It is important to know the method applied as a lender, as the difference between a refinance based AVM and new loan may differ.

Look closely for unsuitable property types. If providing a data sample to an AVM provider, seek to remove properties that may fit the description Unacceptable Property Types as listed by API (Ref: “Residential
Desktop Advisory Note”.) Examples include new house and land packages, serviced apartments, restricted usage property etc. As these property types should not be eligible for your AVM use, they should not be included in the tests (or should at least flagged in the test sample). This may further improve the AVM results.

Property Data Solutions has recently launched an integrated AVM product with our base data subscription service. The PriceFinder (Beta) solution is designed as a checking tool rather than a premium based transactional AVM. As it does not charge transactional fees per AVM, we see it as an ideally positioned as a tool for risk and underwriting staff performing tests on third party AVMs and valuations.

If you need any assistance with AVM’s please contact me at kent@pdslive.com.au

Kent Lardner