dselivanov / Kaggle Outbrain
Companion repository to blog posts https://dsnotes.com/post/2017-01-27-lessons-learned-from-outbrain-click-prediction-kaggle-competition/ and https://dsnotes.com/post/2017-02-07-large-data-feature-hashing-and-online-learning-part-2/
Stars: ✭ 21
Programming Languages
r
7636 projects
Preliminary step
Splitting page_views
In order to conveniently work woth page_views.csv.zip
file we need to split it into chunks, so we can process each chunk independently and in memory.
mkdir page_views_chunks
unzip -p page_views.csv.zip | split --line-bytes=300m --filter='gzip --fast > ./page_views_chunks/$FILE.gz'
Configuration and utilities
You can adjust configuration in conf.R
file - paths to data, number of cores to use, number of partitions, etc. Need to specify path to initial data files and path to page views chunks from step above
Misc functions are in misc.R
file.
Baseline 1
Here we won't use page_views
- only data from clicks_train.csv.zip
, events.csv.zip
. To run baseline you need to run:
-
Rscript 0-0-prepare-baseline-1.R
- prepares dataclicks
,events
,promo
files. -
Rscript 0-1-prepare-baseline-1.R
- creates ans saves model matrix to disk (partition byuuid
). -
Rscript 0-2-run-baseline-1.R
- fit FTRL to model matrix chunks from step above. -
Rscript 0-3-predict-baseline-1.R
- generate submission file (without leak)
Rough timings provided at the top of each file.
Baseline 2
To run baseline you need to run:
-
Rscript 1-0-prepare-baseline-2.R
- preprocesspage_views
- filter not relevant page views and partition byuuid
. -
Rscript 1-1-prepare-baseline-2.R
- creates ans saves model matrix to disk (incluing hashed interactions between page views and advertisement and user context). -
Rscript 1-1-extract-leak.R
- extracts leak -
Rscript 1-2-run-baseline-2.R
- fit FTRL to model matrix chunks from step above. -
Rscript 1-3-predict-baseline-2.R
- generate two submission files - with and without leak
Rough timings provided at the top of each file.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].