Image for post
Image for post
Searching for Star on IMDB. Given my movie preferences I would have expected Star Wars or Star Trek.


Hmmm, no Star Wars in the IMDB search results when I search for star; these search results aren’t wrong; but they don’t really take my personal movie preferences into account (this makes sense for a general database like IMDB).

But for fun, lets build a simple search that does take personal preferences into account. If I type star I would expect Star Wars or Star Trek to show up as I like science-fiction moveis ! For people who really like movies about music, A Star is born is probably a good suggestion.


You’ll need Apache spark, Docker (to run a Elasticsearch 7.10 and Cerebro), and Python in order to follow. If you don’t have these installed right now, not a problem, you can just read along. All the code is available in a Github repository. …

Image for post
Image for post

Github repo

This story is organized into two parts. The first one is about building a movie recommender system based on collaborative filtering using Apache Spark. The recommendation model we’ll build is almost identical to the example model from the Apache Spark ALS model, so we won’t go into too much depth here.

The second is about serving the results recommendation model. Instead of pre-calculating the recommendations for each user or creating a service that can make predictions given a user we’ll store everything in a MemSQL relational database and will use some build-in ML methods to make user prediction in SQL. …


Pieterjan Criel @pjcr

πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Dad of two πŸ‡§πŸ‡ͺ Ghent 🎈 @Balloon_inc πŸ‘¨β€πŸ’» Coding πŸ§ͺ Data science πŸ“ˆ Graph enthusiast πŸ‘¨β€πŸ’» Engineer @showpad

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store