Introduction
Hmmm, no Star Wars in the IMDB search results when I search for star; these search results aren’t wrong; but they don’t really take my personal movie preferences into account (this makes sense for a general database like IMDB).
But for fun, lets build a simple search that does take personal preferences into account. If I type star I would expect Star Wars or Star Trek to show up as I like science-fiction moveis ! For people who really like movies about music, A Star is born is probably a good suggestion.
Prerequisites
You’ll need Apache spark, Docker (to run a Elasticsearch 7.10 and Cerebro), and Python in order to follow. If you don’t have these installed right now, not a problem, you can just read along. All the code is available in a Github repository. …
This story is organized into two parts. The first one is about building a movie recommender system based on collaborative filtering using Apache Spark. The recommendation model we’ll build is almost identical to the example model from the Apache Spark ALS model, so we won’t go into too much depth here.
The second is about serving the results recommendation model. Instead of pre-calculating the recommendations for each user or creating a service that can make predictions given a user we’ll store everything in a MemSQL relational database and will use some build-in ML methods to make user prediction in SQL. …
About
