Dream Home Describe API
Creating a Real Estate Voice Search with GoLang
As the proliferation of digital voice assistants adoption speeds up, it is about time to update real estate searches. Real estate has long been one of the most popular internet searches.
Wouldn't it be great to simply click your phone's microphone and state “find me homes on the water in Jacksonville with a boat dock for under one million?” Or simply state “I want a newer condo in near downtown Jacksonville with 2 bedrooms an office and granite countertops.” Certainly easier than having to fill in all of those pesty little dropdown boxes.
At GoMeisters, we have been working on perfecting such a search for about 2 years. This week, we have released our updated native IOS and Android real estate search apps. Known as Dream Realty Home Search, they make full use of our Go micro-services api, “Dream Home Describe” (Go is a language developed by Google for its speed and security – micro-services break down software development into a suite of small and fast manageable services – api stands for application programming interface – a modern way to retrieve data across the internet)
Natural Language Understanding
Facilitating voice search, first requires voice recognition on the part of your mobile device or laptop. Fortunately, this has been pretty much successfully perfected by Google and Apple, so no need to re-invent that wheel.
With most digital voice assistants, after voice recognition, the next step is to understand the “intent” of the user. One could ask Google “How old is Yoko Ono?” or “Where is the closest Starbucks?” Google has to quickly determine what type of request this is. Is it a weather question?, a biographic one?, location?, food, etc. Or possibly with Alexa, “play the Beatles White Album” would obviously be a music request.
Accomplishing a natural language search with a limited domain like “real estate” is considerably more manageable than the generalized abilities of Google and Alexa.
Determining the user's intent in a real estate voice search is not really required: Dream Home Describe (DHD) is limited to one obvious intent: finding real estate.
What is needed is a substantial algorithm that can parse (break text down into manageable pieces) and make sense of what the natural language (namely spoken English in our use case) search is requesting. This algorithm would specifically have the goal of converting the user's searches into an equivalent SQL (structured query language) query. SQL being the query language of the database in which the real estate listings are stored. SQL, also being the most common way real-estate sites retrieve user dropdown box query results.
A simple example of a text-based real-estate SQL query based on a user entering Jacksonville in the location box and choosing 3 beds in the beds dropdown boxes would be “select * from listings where city = “Jackonville” and beds = 3”
So, having been passed the actual text of the user's search and knowing the intent is to find real-estate, what steps does DHD need to take in order to convert the user's request to SQL?
Breaking Down User's Search
We need to break down the search into a number of pre-defined potential real-estate based slots.
To do so, we need to first recognize and eliminate the usual dross that a voice search often commences with like, “I would love to find a home that has....”
Looking further, has the user mentioned anything about beds or bedrooms, or baths, or a price. Has the user mentioned a location, etc? There are many such pre-defined slots to check for in a real estate voice search.
When parsing the search, there are 3 most important tasks:
Understand the location – the primary entity in a real-estate search.
Have an understanding for each word – whether it is to be eliminated as dross or to be used in a slot of the query such as beds, home type , pool?
Keep track of and analyze results to continually improve DHS.
For achieving step 1, DHD has approximately 15 functions implemented to nail down the location. Fortunately, there is some help from the voice recognition services of Google and Apple: locations are quite often recognized and therefore capitalized. Thus, a user search would often be passed to our api as “Looking for single-family homes for less than 300k around Southside Jacksonville with at least 3 bedrooms”
For step 2, the user's search must be run through about 12 functions in search of specific real-estate amenities like beds, baths, price, etc.
In creating the SQL search, beds and baths are easy. Most of the more unique features of the home are described by the realtor in the remarks section of the listing. It is important to have a full-text index on that column in the listing table.
A user may search for a “Home in New York having high ceilings, granite and 3 bedrooms.” DHD would match the “high ceilings” and “granite” from the remarks column.
How to know “granite” and “high ceilings” are potential real-estate terms and not simply unrecognizable words?
The use of several custom database word related tables are needed. There is a table of words that a user may employ in their search should can be eliminated when parsing. Examples would be “please”, “available”, “which”, “search”,etc.
There is a table of real-estate terms which can be classified as ngrams – (one or more continuous words) Examples would be “beach access”, “bungalow”, “club house”, “concrete”, etc – there are many such real-estate ngrams to look for.
There is a table of real-estate related terms that can be spelled several ways – especially compound words.. As it is often necessary to include matching in the remarks column, DHD needs to add some fuzzy logic to account for various realtor spellings of a number of real-estate compound words. Examples would be “age restricted,age-restricted”, “mid century,mid-century,midcentury”, “back yard,backyard,back-yard.”
There is a table of real-estate terms that can be either singular or plural. These words need to be converted to a regex version for more inclusive searching. Terms like “french doors” could be listed by the realtor in the remarks as singular if there were only one and a voice search of “home with french doors” often would not match a singular. Other examples would be fireplace(s), expansive view(s), patio(s), etc.
There is a table of real-estate synonyms. An example would be “remodeled,updated,renovated,upgraded.” If the user states they are looking for a “remodeled home”, by including the accompanying synonyms in the SQL, more relevant search results will appear.
Another such example would be “tear down|tear-down|handyman special|handyman|fixer upper|fixer-upper|fixer|demolition|fixers|fixer uppers.” All synonymous terms that a realtor could use in the remarks column for a “fixer-upper.”
These is also a table of dross search lead-in words. Very important to eliminate any matching dross expressions. “I am looking for”, “Find me a”.
All of the above word-related tables are important to have in a database and not in the search api code. By having such database tables, words and expressions can and will need to be added the more the search is used. Also, as DHD expands into other areas of the country, there will be more indigenous terms to add. An obvious example would be “mountain views.” Certainly no mountains in Florida.
If, after the first pass, no location can be determined and/or there are words un-accounted for, a more general pass is then applied.
Ultimately, a pretty well formed SQL query has been created and it is passed to our Golang search micro-service which processes real estate SQL queries.
The user's search string, the created SQL, the location extracted and the query result count are all saved in a table for analysis and improvements. Artificially intelligent searches need to keep learning.
Need for Speed
Since our voice real estate search requires 2 services, one for parsing the user's request into SQL and the second for processing the SQL, speed is really of the essence.
We first developed Dream Home Describe with php – the common scripting language used extensively by WordPress and many, many other websites. Our search for faster processing finally led us to Google's Go aka Golang.
Much has been said of Go. Here is a list of companies using Go – quite impressive. Go is wickedly fast and secure (and awesome). Also, as there is a substantial amount of pattern matching in natural language processing, we love Go's ability to pre-compile our common regex's.
There has been a substantial increase in speed since adopting a Go micro-services architecture.
Dream Home Describe vs Alexa (et. al)
Although DHD currently has limited conversational ability, speed is still of the essence. We've tested an Alexa based version of a real-estate search which was painfully slow. Alexa must've asked at least 10 questions then finally announced it would be emailing the results . Ultimately, an extremely slow process after which you ended up with an email list that could include many properties.
I think one reason we could only find one Alexa real-estate search as it doesn't seem all that practical. When I'm searching for real-estate I want to see the result immediately on a screen I understand that digital voice assistants can incorporate screens, but maybe it's just a matter of taste and choice. Almost everyone these days has a smart phone. Just click on your Dream Realty app and state your dream home wishes.
“Find me 3 bedroom waterfront condos on Jacksonville Beach with an ocean view” - then BOOM – app responds with voice stating “26 listings found.”
Dream Realty Home Search currently only has homes in the Jacksonville Florida area. You can test our api without downloading the apps by going to www.DreamRealty.com with a Chrome browser. Just scroll to the bottom of the page and click on the microphone.
If you search “3 beds, 3 baths” you will get a response from DHD that location was not found. You can then click on Modify and then state a location without having to repeat your whole search.
Dream Realty Home Search is really a fun way to search for properties whose time, I believe, has come.