YouTube Ad Placements

This repository contains code and data to reproduce the findings featured in our stories, "Google Has a Secret Blocklist that Hides YouTube Hate Videos from Advertisers—But It’s Full of Holes," and "Google Blocks Advertisers from Targeting Black Lives Matter YouTube Videos" from our series Google the Giant.

Our methodology is described in "How We Discovered Google’s Hate Blocklist for Ad Placements on YouTube," and "How We Discovered Google’s Social Justice Blocklist for YouTube Ad Placements."

Data that we collected and analyzed are in the data folder.

Jupyter notebooks used for data collection, preprocessing and analysis are in the notebooks folder.

Installation

Python

Make sure you have Python 3.6+ installed, we used Miniconda to create a Python 3.8 virtual environment.

Then install the Python packages:
pip install -r requirements.txt

Notebooks

These notebooks are intended to be run sequentially, but they are not dependent on one another.

0-data-collection.ipynb

How we interacted with the "PlacementSuggestionService" API from "ads.google.com". We sent each term from terms.py through the API. Use this notebook for reference: it is not functional due to the expired or redacted parameters present.

1-data-preprocessing.ipynb

Parsing the API responses and fetching the suggested videos and channels for each term we sent to the API.

2-data-analysis-hate.ipynb

The bulk of stats and tables for our hate methodology.

3-suggestion-analysis.ipynb

Looks at videos and channels suggested by the API for hate terms. We cross reference these suggestions with channels identified as "extremist" or "alternative" by researchers from Dartmouth College, Northeastern University, and University of Exeter and provided to the Markup by the ADL, as well as a channels researchers at EPFL and UFMG identified as "alt-right" or "alt-lite".

4-data-analysis-social-justice.ipynb

The bulk of stats and tables for our social justice methodology.

5-rerun-and-check-status.ipynb

After we shared our findings with Google, we re-run the analysis on data collected on March 31, 2021. Check what changed from the original data we collected on November 20, 2020.

utils.py

Contains functions to parse and decipher the API responses.

terms.py

This contains lists of terms used in the series. This includes hate terms sourced from the SPLC, RationalWiki.org, and Muslim Advocates. social_justice terms sourced from Color of Change, MediaJustice, Mijente, and Muslim Advocates. adhoc terms were submitted for comparison against terms in the other lists. noise contains randomly generated strings.

Refer to the "Data" section below for the API status of each of these terms.

Data

This directory is where inputs, intermediaries and outputs are saved.

data
├─── reference
│   ├── placements_api_example_responses
│   │   ├── blocked.json
│   │   ├── empty.json
│   │   ├── full.json
│   │   └── partially_blocked.json
│   └── what_is_blocked.xlsx
├── input
│   ├── channel_lists
│   ├── hate_terms_background_info.csv
│   ├── placements_api
│   │   ├── adhoc
│   │   ├── blocked
│   │   ├── blocked_basewords
│   │   ├── hate
│   │   ├── noise
│   │   ├── policy
│   │   └── social_justice
│   ├── placements_api_deep3
│   │   ├── we wuz kangz.json
│   │   ├── white ethnostate.json
│   │   └── whitegenocide.json
│   └── video_metadata
│       ├── deep_catalog_wwk_wg_we.csv
│       └── topline_hate_videos.csv
└── output
    ├── channel_overlap.csv
    ├── tables
    ├── placements_api_keyword_status
    │   ├── adhoc.csv
    │   ├── basewords.csv
    │   ├── hate.csv
    │   ├── policy.csv
    │   └── social_justice.csv
    └── placements_api_suggestions
        ├── channels_for_hate_terms.csv
        ├── channels_for_social_justice_terms.csv
        ├── videos_for_hate_terms.csv
        └── videos_for_social_justice_terms.csv