Predictive Analytics in Real Estate

The real estate sector accounts for a significant slice of all the wealth on the planet, and of the power that accompanies that wealth. Though the global property market is calculated to be the third largest world industry after insurance provision and pension funds¹, the value of all existing land and property was estimated in 2016 at US$217 trillion, with residential property accounting for 75% of that². The figure has risen since, while the global pandemic and the move away from cities has only fueled the real estate sector further^3,4.

The numerous markets and sub-markets that grew up in tandem around this huge locus of money have been seeking certainty and consolidation through technology for centuries, and were vanguard adopters of new forecasting methodologies⁵, construction techniques⁶ and rent-enabling technologies^7,8.

However, the sector has recently become more conservative, as the pace of innovation has quickened⁹, with a smaller and more adventurous tier of organizations turning to AI software development to formulate predictive analytics solutions capable of addressing the new challenges of the market.

In this article we’ll take a look at current implementations of AI in real estate when it comes to predictive analytics and at some of the possibilities that new research might offer for the sector.

It took 3-4 years after the advent of GPU-accelerated machine learning (in 2010-11) for a slate of new AI-enabled companies and platforms to make an impact on the real estate market, while a number of older names in traditional real estate analytics have pivoted to also adopt predictive analytics:

• One Californian real estate company uses AI-powered predictive analytics to traverse the high volumes of data that issue from real estate agents, using advanced analytics to understand which leads concluded in a successful sale¹⁴.
• Another startup out of California uses image recognition to evaluate damage from smoke, water, and other attritional factors that may affect the retail or rental value of a property¹⁵.

• Launched in 2014, Opendoor is a digital platform for home purchasing and selling, and uses the merged architecture of diverse machine learning real estate valuation models to arrive at better estimates of what a property is worth¹⁶ (a technique known as ensembling¹⁷).

• Property platform Mashvisor uses machine learning-assisted predictive analytics to provide tools to help clients locate ideal investment properties, choose the best location for development, and evaluate optimal rental strategies¹⁸.
• Enodo Score uses AI-driven predictive analytics to evaluate whether upgrades and improvements to an apartment or house would be rewarded with commensurate rises in property values¹⁹.

• San Francisco-based Spacequant uses data from operating statements and rent rolls to develop machine learning analytics to facilitate loan origination and to provide real-time data on commercial real estate performance, among other services²⁰.
• Besides a wide portfolio of AI real estate services and a database of 110 million US properties, Zillow hosts a tech hub and research group dedicated to new applications of AI to the real estate market. The company operates its Zestimate valuation tool over Amazon Kinesis and Apache Spark running on Amazon EMR²¹.

Besides obvious factors such as space, age, local crime statistics, the state of the economy and school access, some very abstract data can come into play when estimating the current value of real estate.

These include the presence of trees, how frequently earthquakes occur, pandemics, proximity of coffee shops and grocery stores, and the qualities of online reviews for the area. Also, traditional automated valuation models can fall short in areas where houses are traded infrequently, such as Korea²⁵. In such cases an area's property value index may be calculated based on more recent data from similar or adjacent locations.

However, such 'translocation' of data can be problematic where the current prevailing conditions are different — for instance, after the advent of a global pandemic that has affected the traditional model of centralized cities²⁶.

According to McKinsey, nearly 60% of the predictive power of AI-driven real estate analytics comes from non-traditional variables:

Taking these new data sources into account involves some sacrifice, because it's impossible to maintain decades of statistical continuity when novel sources come along and disrupt the variables. Nonetheless, this bold approach yields deeper and non-obvious insights into factors affecting rental or sale valuation.

Variables for machine learning datasets related to real estate may benefit from a wider view of economics and world affairs. Since there are subtle but essential relationships between credit, gross domestic product (GDP) and real estate markets²⁷, a myopic overview of historical local property values is likely to miss international trends that should be considered when evaluating property prices.

For instance, the conditions that underpinned the financial crisis of 2008 (increased lending and high loan-to-value mortgage agreements) inevitably powered forecasts in the US for rising property values under traditional analytical models.

This is because demand was continuing to outstrip availability, barriers to entry were low, and the historical variables for the likelihood of a new property slump were too couched in historical context for the modeling parameters that were in use at the time²⁸. The subsequent crash revealed the shortcomings and limited focus both of the data and the forecasting methodologies.

Likewise, the effect that the pandemic has had on global property values has brought into focus the need for machine learning frameworks to account for historically absent factors that sit far outside the traditional data-gathering scope — but without defaulting to pessimistic predictions that fail to account for 'local resilience' in a real estate market.

If a machine learning system does not have access to factors that will influence a real estate analytics model, real-life people are likely to fill in the gap, confounding the model with unanticipated market behavior.

For instance, after the property crash of 2008, individual real estate markets in the UK reacted proactively and independently based on a number of imponderable factors, such as emerging news of the crisis from the US²⁹.

This meant that one neighborhood, or even one street, would suddenly have several houses for sale, lowering local property values in defiance of a more robust national market, and, inevitably, contributing to a wider spread of property devaluation.

A well-designed predictive analytics model needs the ability to identify and ring-fence sporadic local events of this nature, rather than treating them as coalmine canaries that augur wider market disruption (and thereby act out a self-fulfilling prophecy).

As we've seen, proximity to major coffee chain outlets is a new and interesting factor affecting real estate valuation; but the larger-scale operators themselves also use predictive analytics via machine learning to identify areas where they should open new stores:

A predictive analytics system with this remit has to consider a wide variety of factors³⁰:

• Demographic data by segment and area code
• Local property price index
• Store level and size appropriate for the locality
• Local culture
• Crime rate and type
• Established competitors
• Planning applications for competitors
• Parking availability
• Local transport network
• Local purchasing power
• Cost-effectiveness of area rents

Starbucks, which operates over 32,600 stores worldwide³¹, uses a commercial mapping and business intelligence service³² in order to ascertain the best locations for new outlets. The framework collates and interprets a high volume of statistical data for the location and even considers whether a new outlet would be compromised by the opening of a competing chain in the same area (this strictly mathematical rationale also explains why popular chains sometimes run outlets within eye-sight of each other³³).

The framework also helps in determining which locations should offer 'Starbucks evenings', occasions where alcohol is sold in store — another case where it’s necessary to evaluate local demographics, among other factors.

It seems likely that the continuing growth and adoption of predictive analytics in real estate will depend on several factors:

• An end to the uncertainty around AI regulation, ensuring that new and fruitful data architectures for predictive analytics do not depend on sources that are imperiled by pending government legislation.
• The emergence of a definitive market leader — a corporation or start-up with a clear product or service, an unassailable business model for the sector, and a FAANG-style taste for publicity.
• Time — because some of the most promising new predictive analytics frameworks have not been in existence long enough to prove their value over the long term.
• The willingness of incumbent market leaders to develop AI-driven systems that at least run in parallel to their older analytics models, while their worth is evaluated.
• A hard break from the need to build predictive analytics systems from the ground up, as methodologies for data-gathering gradually become commoditized into SaaS offerings that can more easily be configured into agile frameworks.
• A clearer taxonomy for the value of contributing data streams, including computer vision-based sources for interior and exterior evaluation and interpretation; well-defined data pre-processing pipelines for public-facing commercial or governmental resources (such as planning applications); and clear evidence that these streams correlate to real-world conditions within an acceptable margin of error.

However, those who wait for all these certainties may end up customers rather than providers. As one Oxford university professor recently noted in Pi Lab’s paper on PropTech cited above:

'We should remember Amara’s Law: we tend to overestimate the impact of a new technology in the short run, but we underestimate it in the long run. Sooner or later, the long run will arrive.'