Web-app that Curates home-listings by Visual Style using Deep Learning
User uploads a picture of a home (image of the outside of a home) that they really like. The app returns home-listings in the Seattle that have a similar visual style as the uploaded picture.
Note that the returned listings are static listings that were downloaded from Redfin in May-June 2020. The app is not intended to be a stand-alone app. Rather it is intended to be a POC for a feature that home-listing search engines could incorporate.
- Creates image embeddings for a set of home-listings.
- When the user uploads a picture, creates an image embedding for the uploaded picture
- Finds the most similar home-listing by comparing the image embedding of the uploaded picture with the image-embeddings of the home-listings using Cosine similarity metric.
- Image embeddings are created as follows -- Transfer learning from a pre-trained Resnet50 architecture is used -- A classifier for classifying between different home-styles is built on top of the Resnet50. The output of the second last layer of Resnet50 is used as an input to the classifier. -- Although we build a classifier, we do not use the output of the classifier. Instead, we use the output of the second last layer as the Image embedding.
- Streamlit
- Sklearn
- Tensorflow (and Tensorflow keras)
- Pillow
More information about the App can be found in the slides at https://bit.ly/rm_insight20b_42 The app is hosted at: http://www.datastory.work:8501/ A demo of the app is available at https://youtu.be/qbgyS_FKa8g
- data: Contains iamges used for training/testing the recommendation model as well as home-listings used for recommendations.
- notebooks: Contains notebooks used to build and test the recommendation model(s).
- scripts: Contains scripts for scraping images from google as well as splitting images in to training and test setss -visual_home_finder: Contains source code for the Streamlit app HomeSpotter.