Are you buying an apartment? How to hack competition in the real estate market with data monitoring

In the last couple of years, real estate companies have shifted their focus to the digital world, and now almost all investments have an online system showing what apartments are available. This is very convenient for their potential clients, as they can easily become familiar with the apartments on offer. Things become interesting when all available data is monitored on a weekly basis, and sales progress is analysed.

How can you benefit as an individual client?

Also, this approach can be beneficial to a real estate company through monitoring competition?

The investment is built in the Warsaw eastern district of Praga Południe. This is an area with great potential. Currently, there are a lot of apartment buildings and places that wait for renovation or replacement. However, it is very well connected with the city center (15 minutes by public transport or 10 minutes by car). Another advantage is that there are a lot of points of interests like shopping malls, parks, schools or medical service points.

The monitored investment is built in the eastern Warsaw district of Praga Południe. This is an area with great potential. Currently, there are a lot of apartment buildings and places waiting to be renovated. However, it is very well connected with the city centre (15 minutes by public transport or 10 minutes by car). There are also a lot of facilities nearby, such as shopping malls, parks, schools, or medical service points.

Here are the basic characteristics of the monitored investment:

After collecting the first data from the investment website, we can see the current offers of different apartment types:

This is good to know, but the most interesting data comes from the periodical monitoring of sales progress. Below you can see the timeline representing the number of apartments sold each week:

What else can we see? For example, what is the current state of sales progress?

It is worth remembering that construction began in Q1 2018. Currently, only the building’s foundation is ready, but almost half of the apartments are already sold.

Let’s see what the sales dynamic is by apartment type. We can also try to make a simple prediction of when all apartments will be sold.

What we can see from this visualisation:

Another interesting observation is about apartments that had sold, but later became available again. There are multiple reasons why this happens. It is important to understand how the selling process works. The buyer has to sign an agreement with the developer before requesting a loan from a bank. There is always a possibility that after officially signing for an apartment, the bank can reject your loan request. The buyer can also change their mind and withdraw for other reasons.

In the observed data, there was only one situation like this. A 2-room apartment with a separate kitchen was sold at the end of August 2018, but, in mid-September, became available again.

Companies are aware of this process, so they usually protect their websites and online systems. Some of the techniques are:

In the case described in this article, I monitored one of the locations in Warsaw. The real estate developer is selling apartments in this location using an online system, which contains the following table of apartment availability.

This system didn’t have any captcha or anomaly detection protection. The only impediment here is that the content is loaded dynamically using JavaScript, and there is no open API endpoint where we can just request the data. You are able to view the data only after clicking through the interface.

The solution is a web crawler simulating human behaviour, clicking through the interface. Imagine that you have a robot-employee, who monitors all the information you need, on a weekly basis, doesn’t get tired or bored, and is 100% precise.

const puppeteer = require('puppeteer'); (async() => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://real-estate-developer-address.com/'); // Real URL address is confidential await page.click('#cbNotSale'); await page.waitFor(2000); const nextSelector = '//input[contains(@alt, ">")]' var next = await page.$x(nextSelector) while (next.length > 0) { const cells = await page.$x('//*[@id="gwProducts"]/tbody/tr/td'); var vals = []; var val = null; for (var i = 0; i < cells.length; i++) { val = await page.evaluate(x => x.textContent, cells[i]) if (val == "F1") vals.push([]); // The quickest (dirty) way to recognize row beginning. if (val == "zobacz" || val.trim() == "") continue; // The quickest (dirty) way to recognize row end. vals[vals.length - 1].push(val) } vals.forEach(row => { console.log(row.join(";")) // Robot prints collected rows to the stdout }); await next[0].click() await page.waitFor(2000); next = await page.$x(nextSelector) } await page.screenshot({path: 'example.png'}); await browser.close(); })();

And we can run the crawler with the command below. Just configure a CRON or other job to run this weekly:

node scraper.js > data-YYYY-MM-DD.csv

As a private investor, I need to excel in my investments. There is no place for bad bets, which is why I use all available cards in the deck to make sure I do everything in my data science power. This analysis gave me the information about the investment’s financial performance, and I could predict how much time I had to close the deal for my dream apartment. I had more time for the final decision, and I was more confident during negotiations. Remember that buying an apartment at the latest possible moment gives you short-term alternatives, instead of freezing your capital right away. I had an edge that other buyers did not, so I could easily tackle false claims about the popularity of the apartments, and not feel pressured into buying. Each sales process is a game with unequal distribution of knowledge. Knowledge of sales dynamics will allow you to fight against the odds and close better deals.

Related posts:

You seek out some of the most expert people around you to get advice from. You pay for books, online courses and attend workshops and conferences, all to reduce the likelihood of failure and/or to…

The UN climate science panel has released its summary of the evidence around the tougher climate goal demanded by vulnerable countries. We break it down This is the first time the Intergovernmental…

Week two was a great experience and learned more behind what writing is really all about. We studied the history behind story telling and learned how old the story telling culture is and has changed…