ACM RecSys Challenge 2019

About the challenge

Trivago is a global hotel search platform focused on reshaping the way travelers search for and compare hotels, while enabling advertisers of hotels to grow their businesses by providing access to a broad audience of travelers via our websites and apps. We provide aggregated information about the characteristics of each accommodation to help travelers to make an informed decision and find their ideal place to stay. Once a choice is made, the users get redirected to the selected booking site to complete the booking.

It’s in the interest of the traveler, advertising booking site, and trivago to suggest suitable accommodations that fit the needs of the traveler best to increase the chance of a redirect (click­out) to a booking site. We face a few challenges when it comes to recommending the best options for our visitors, so it’s important to effectively make use of the explicit and implicit user signals within a session (clicks, search refinement, filter usage) to detect the users’ intent as quickly as possible and to update the recommendations to tailor the result list to these needs.

Goal of the challenge is to develop a session-based and context-aware recommender system to adapt a list of accommodations according to the needs of the user. In the challenge, participants will have to predict which accommodations have been clicked in the search result during the last part of a user session. Afterwards predictions are evaluated offline and scores will be displayed in a leaderboard.

Visit the challenge website for more information about the challenge.

You can read the article in which we describe our work here.

Team members

We partecipated in the challenge as PoliCloud8, a team of 8 MSc students from Politecnico di Milano:

• D'Amico Edoardo
• Gabbolini Giovanni
• Montesi Daniele
• Moreschini Matteo
• Parroni Federico
• Piccinini Federico
• Rossettini Alberto
• Russo Introito Alessio

We worked under the supervision of two PhD students:

• Cesare Bernardis
• Maurizio Ferrari Dacrema

Dataset

The dataset can be found at this link: challenge dataset page. You must be registered to download it.

Run the code

Preprocessing

Run this to preprocess the original data:

python preprocess.py

Files will be saved inside: dataset/preprocessed. It is possibile to work with different modes:

• full: all the samples from train.csv are used as training set and all the test samples are used as test set
• local: 80% of the train.csv is used, the remaining 20% is stashed as validation set
• small: only a small number of sample is taken from the train.csv (you can choose how many, default is 100k samples). This is useful for debugging purposes.

Training and test set

Inside the folder preprocess_utils, there are files used to create the datasets suitable for each model:

Core models

• dataset_xgboost.py: create the dataset for XGBoost
• create_dataset_classification_rnn.py: create the dataset for RNN
• tfranking_dataset_creator_2.py: create the dataset for TensorflowRanking
• dataset_catboost.py.py: create the dataset for Catboost
• dataset_lightGBM.py.py: create the dataset for LightGBM

Support models

• dataset_xgboost_classifier.py: create the dataset for XGBoost classifier
• create_dataset_binary_classification_rnn.py: create the dataset for RNN classifier

Stacking Ensemble

• dataset_stacking.py: create the dataset for the Stacking Ensemble

Features

One of the hardest phase of the competition was feature engineering. We managed to craft around a hundred different features. You can find all of them in the folder extract_features. Each file creates a single feature and contains a description of it.

Models

You can find all the recommenders we developed inside the folder recommenders. They inherit from a common base class called RecommenderBase. This abstract class exposes some useful methods, like:

fit()

Fit the model on the data. Inherited class should extend this method in the appropriate way.

recommend_batch()

Return the list of recommendations.

evaluate()

Return the MRR computed on the local test set.

We developed several models that have been finally ensembled:

• XGBoost
• Tensorflow Ranking
• Recurrent Neural Network
• Catboost
• LightGBM

Read the paper for more details.

Ensemble

We trained a new model in a new training set composed by the predictions of the previously trained models (stacking), together with all the other feature we crafted.

We report below an analysis of the permutation importance of the various features of the ensemble.