RatS

This project serves for analyzing, and transferring your ratings from one movie tracking / rating website to another.

The goal of this project is to have a universal tool which can transfer your ratings from any site to another without the need of any manual steps like configuring an API access or whatever. Just configure your credentials (see steps'below), start the tool and relax.

This also works if your lists are marked as private, as this tool uses a browser to login and get the content.

How do I use this?

Natively on the command line

This project is currently still under development. Please be patient, as I'm only working on this every once in a while.

1. Make sure you have Python3, Firefox and Xvfb installed on your system. This project is designed to run on Linux.

2. Checkout the project git clone https://github.com/StegSchreck/RatS.git && cd RatS

3. Install the requirements with pip for Python3 pip3 install -r requirements.txt

4. Install Geckodriver

   • Use your system's package manager (if it contains Geckodriver)
     • Arch Linux: pacman -S geckodriver
     • MacOS: brew install geckodriver
   • Or execute sudo ./InstallGeckodriver.sh. For this you will need to have tar, wget and curl installed.

5. Set your credentials

   1. Copy the credentials.cfg.orig file to credentials.cfg and insert your credentials for the sites you need there (without any quotation marks etc.).

      Copying the file will conserve the possibility to do a git pull later on without overwriting your credentials.

   2. Alternatively, you can set the credentials via environment variables. For example, if you want to set the credentials for your IMDB and Trakt accounts you would need to set these environment variables:

      export IMDB_USERNAME=[email protected]
      export IMDB_PASWORD=def
      export TRAKT_USERNAME=abc
      export TRAKT_PASWORD=def
      

      It is important to use the variable names completely in uppercase!

   3. The third way is to set the environment variables when running the transfer script, like this: [email protected] IMDB_PASWORD=def TRAKT_USERNAME=abc TRAKT_PASWORD=def python3 transfer_ratings.py --source trakt --destination imdb

   The credentials in environment variables are overruling the ones in the credentials.cfg file.

6. Execute the script with Python3 python3 transfer_ratings.py --source trakt --destination movielens

   This will first parse your ratings in Trakt, save them in a JSON file for later use and then try to find those movies in Movielens an put your rating there. Notice: This will also overwrite rating you already did set there before.

   This script will take some minutes. Relax. You can follow the progress in console output.

   For more information about how to use the script, you can call python3 transfer_ratings.py --help

7. At the end, the script will print out how many movies were successfully posted. Afterwards all the movies which couldn't be found are printed out, so you can check them manually. The failed movie are also exported to a JSON file, so you can easily try them again (see below).

Inside a Docker container

Please note: This is currently not working on Windows. See issue #19

1. Create a credentials configuration in your home folder, e.g.: touch ~/.RatS.cfg

2. Configure your credentials in the file you just created, e.g.:

   [Trakt]
   USERNAME = abc
   PASSWORD = def
   

   Please see the RatS/credentials.cfg.orig in this repository as a template. Please enter your credentials as they are, without any quotation marks.

3. Get the Docker image: docker pull stegschreck/rats

4. Run the script: docker run -it -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg stegschreck/rats python3 transfer_ratings.py --source trakt --destination movielens

   The -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg option will load the credentials file you just created from your home directory into the docker container, so that the script can use it.

   You will see the progress in your console. If you want to run this in the background, you can add the option -d to the docker run command to hide the output.

5. If you want to run the command again, simply run docker start -ai <container-id>. You can find the container id using docker ps -a or by running docker ps -q -l, if you haven't started any other containers in the meanwhile.

6. After the successful run of the transfer script, you may remove the docker container using docker rm <container-id>.

Command line call parameters

1. the first argument (--source) is the site where the ratings are parsed from (see Available Parsers)
2. the second argument (--destination) is the site where the ratings should be posted (inserted) to (see Available Inserters)

You can also omit the destination argument in order to just save the parsing results to a JSON file. You might insert the saved results anytime later. (see below)

Furthermore, you can define multiple destinations, e.g. like this: python3 transfer_ratings.py --source trakt --destination movielens --destination imdb in order to use the same data from the parser in this run for multiple destinations.

Mandatory

-s / --source: the site where the ratings are parsed from (see Available Parsers). This parameter must be given in even if you use the -f option.

Optional

-d / --destination: the site where the ratings should be posted (inserted) to (see Available Inserters). If you don't specify any destination, the ratings are only parsed and saved locally.

-D / --all-destinations: try to insert the parsed ratings to all configured sites for which valid credentials can be found

-f / --file: read ratings from a previous run instead of parsing them

-v / --verbose: increase output verbosity (i.e. print the generated predictions in the console)

-x / --show_browser: show the browser doing his work (this might help for debugging))

-h / --help: Display the help, including all possible parameter options

Currently Available Parsers

• AlloCine
• Criticker
• FilmAffinity
• Filmtipset
• Flixster deactivated until new account feature is implemented at Flixster
• iCheckMovies
  • please configure the conversion value for the like/dislike into a numeric rating yourself in the credentials.cfg
• IMDB (with IMDB account)
• Letterboxd
• Listal
• Movielens
• MoviePilot
• Plex (locally hosted server)
• Rotten Tomatoes
• TMDB (The Movie Database)
• Trakt

Currently Available Inserters

• AlloCine
• Criticker
• FilmAffinity
• Filmtipset
• Flixster deactivated until new account feature is implemented at Flixster
• iCheckMovies
  • please configure the boundaries for the ratings conversion yourself in the credentials.cfg
• IMDB (with IMDB account)
• Letterboxd
• Listal
• Metacritic
• Movielens
• MoviePilot
• Plex (locally hosted server)
• Rotten Tomatoes
• TMDB (The Movie Database)
• Trakt

Trying again with former export data

Depending on what you used in the first run:

... using the command line

You can re-use the data you parsed before, without parsing again. This will help you, if you want to distribute from one source to multiple destinations. The parser tells you in which file he saved his result, the folder is ./RatS/exports. You can use this data by calling the script for example this way:

python3 transfer_ratings.py --source trakt --destination movielens --file 20170721191143_Trakt.json

Please notice, that the --source argument is still needed in order to identify which data to use from the given file for the inserter.

Furthermore: This section is meant to be used with the native command line version. Docker containers work differently.

... using Docker

1. After the first run of the Docker container: (assuming you didn't start any other Docker containers in the meanwhile)

   • save the state of the container (including the saved data): docker commit <container-id> user/RatS
   • lookup the JSON file the script saved into the exports folder: docker run -it user/RatS ls -l RatS/exports/ (take the latest JSON file)

2. Run the container again, e.g. with a different destination docker run -it -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg user/RatS python3 transfer_ratings.py --source trakt --destination imdb --file 20170721191143_Trakt.json

Switching between environments

You can copy the exported JSON from/to the Docker container if you like. I don't recommend this if you have no experience with docker. If you have, you already know what to do here ;)

Problem shooting

Script aborts because of Timeouts

It might occasionally happen, that the script runs into errors caused by the page loading too slow. In the past, this was quite an issue. I now tried to build a mechanism with dynamically increasing timeouts. Therefore, the script sleeps for one second after failing the first try (effective for multiple problematic locations in the code), waiting two seconds after the second try failed, and so on. The maximum number of tries is ten. If this is reached, the error gets re-raised and the script will fail.

Script aborts with WebDriverException

If you recently updated your Firefox, you might encounter the following exception during the login attempt of the parser:

selenium.common.exceptions.WebDriverException: Message: Expected [object Undefined] undefined to be a string

This can be fixed by installing the latest version of Mozilla's Geckodriver by running again the Install Geckodriver command mentioned above.

Script aborts with MoveTargetOutOfBoundsException while inserting to IMDB

IMDB shows special promotional banners for certain movies. If they are too big, they will cause the ratings component to be out of the viewport of the browser and the script fails with something like:

selenium.common.exceptions.MoveTargetOutOfBoundsException: Message: (620, 629) is out of bounds of viewport width (720) and height (538)

For now, the easiest solution is to run the script on your console with the -x flag. This will display the browser to you in a maximized window which then has a bigger viewport and prevent the error.

Login attempt does not work

This can have multiple explanations. One is, that you are using a password which starts or ends with a space character. RatS is currently not capable of dealing with that. If your credentials have a space character in the middle though, it will work fine.

Trakt aborts with AttributeError

While tying to login to Trakt, the script aborts with this error message:

===== Trakt: performing login
Traceback (most recent call last):
[...]
AttributeError: 'NoneType' object has no attribute 'find'

This was observed to happen when the credentials were configured to use the email address for login. Try to set your username instead.

You are missing a feature or noticed something is wrong?

Before submitting a bug ticket, I would ask you to check first whether all dependencies are up-to-date, especially geckodriver.

Please write a ticket, I will have a look as soon as I can.

Where does the name come from?

The name for this project comes from the first letters of "rating synchronisation". It is that simple. It is also a reference to the movie "Departed".

Acknowledgments

This project uses a free license for PyCharm by benefiting from JetBrains' open source community support program.