COVID-19 Mobility Data Aggregator. Scraper of Google, Apple, Waze and TomTom COVID-19 Mobility Reports

This is a repository with a data scraper of Mobility Reports and reports in different formats.

Table of contents

1. About data
   • 1. About Google COVID-19 Community Mobility Reports
   • 2. About Apple COVID-19 Mobility Trends Reports
   • 3. About Waze COVID-19 local driving trends
   • 4. About TomTom Trafic Index
2. Data explorer
   • 1. Google Reports
   • 2. Apple Reports
   • 3. Waze Reports
   • 4. TomTom Reports
   • 5. Summary Reports
3. How to run script
   • 1. Installation
   • 2. Usage
4. Contributing
5. Showcases
   • 1. Dashboards and visualizations based on these data
   • 2. Articles and research publications

About data

About Google COVID-19 Community Mobility Reports

In early April 2020, Google started publishing an early release of COVID-19 Community Mobility Reports to provide insights into what has changed in response to work from home, shelter in place, and other policies aimed at flattening the curve of this pandemic. These reports have been developed to be helpful while adhering to our stringent privacy protocols and policies.

These Community Mobility Reports aim to provide insights into what has changed in response to policies aimed at combating COVID-19. The reports chart movement trends over time by geography, across different categories of places such as retail and recreation, groceries and pharmacies, parks, transit stations, workplaces, and residential.

Update interval: twice a week

By downloading or using this data and reports, you agree to Google Terms of Service.

About Apple COVID-19 Mobility Trends Reports

The CSV file shows a relative volume of directions requests per country/region or city compared to a baseline volume on January 13th, 2020.

Day defined as midnight-to-midnight, Pacific time. Cities represent usage in greater metropolitan areas and are stably defined during this period. In many countries/regions and cities, relative volume has increased since January 13th, consistent with normal, seasonal usage of Apple Maps. Day of week effects are important to normalize as you use this data.

Data that is sent from users' devices to the Maps service is associated with random, rotating identifiers so Apple doesn't have a profile of your movements and searches. Apple Maps has no demographic information about Apple users, so it's impossible to make any statements about the representativeness of usage against the overall population.

Update interval: daily

By downloading or using this data, you agree to Apple terms.

About Waze COVID-19 local driving trends

The driven kilometers/miles percent change data being shared comes from the Waze app and is aggregated and anonymized. These insights were generated using differential privacy to protect user privacy. No personally identifiable information, such as an individual’s location, contacts, or movement, is available through this data.

These reports show the increase or decrease in driven kilometers/miles as a percent change compared to a baseline. The changes for each day are compared to a baseline value for that day of the week.

• The baseline is the average value, for the corresponding day of the week, during the 2- week period February 11, 2020 to February 25, 2020.
• The reports show trends over two weeks with the most recent data representing approximately 2-3 days ago.

As with all samples, this may or may not represent the exact behavior of a wider population.

Update interval: on weekdays

About TomTom Traffic Index

Covering 416 cities across 57 countries on 6 continents, Traffic Index ranks urban congestion worldwide and provides free access to city-by-city information. Wondering how we determine the rankings, and what exactly the percentages mean?

A 53% congestion level in Bangkok, for example, means that a 30-minute trip will take 53% more time than it would during Bangkok’s baseline uncongested conditions.

You can turn this 53% into travel time through simple calculations. First: 0.53 x 30 mins = 15.9 mins extra average travel time. Second: 30 mins + 15.9 mins = 45.9 mins total average travel time.

We calculate the baseline per city by analyzing free-flow travel times of all vehicles on the entire road network – recorded 24/7, 365 days a year. This information allows us to also calculate, for example, how much extra time a driver will spend in traffic during rush hour in Bangkok.

We perform calculations for all hours of each day, so you can see congestion levels at any time in any city, including morning and evening peak hours.

Update interval: daily

Data explorer

Google reports:

Raw CSV file (in ZIP archive). Direct link to the original CSV: https://www.gstatic.com/covid19/mobility/Global_Mobility_Report.csv

Data for the worldwide (only 1st level of subregions): CSV, Excel

Detailed reports:

Data for the US: CSV, Excel

Data for Brazil: CSV, Excel

Data for Europe: CSV, Excel

Data for Asia + Africa: CSV, Excel

Data for North and South America + Oceania (Brazil and US excluded): CSV, Excel

Apple reports:

Raw CSV file

Data for the worldwide: CSV, Excel

Data for the US: CSV, Excel

The following transformations have been made here:

• transformed dates from columns to rows
• transformed transportation types from rows to columns
• subtracted 100 from values (such as in Google Mobility Reports)

Note: Data for May 11-12, 2020 and March 12, 2021 is not available

Waze reports:

Raw CSV files: Country-level, City-level

Preprocessed report: Google Sheets, CSV, Excel

TomTom reports:

Scraped reports: Google Sheets, CSV, Excel

diffRatio attribute shows relative difference of average congestion levels in 2020 from standard congestion levels in 2019.

Summary reports:

These are merged Apple and Google reports.

Report by regions: CSV, Excel

Report by countries: Google Sheets, CSV, Excel

Report for the US: CSV, Excel

How to run script

Installation

git clone https://github.com/ActiveConclusion/COVID19_mobility
pip install -r requirements.txt

Usage

# scrape data from specified sources. If no sources are provided, data will be scraped from all available sources
python scraper.py scrape <SOURCES>

# merge mobility reports (Apple and Google)
python scraper.py merge

# Scrape data from all sources and merge reports
python scraper.py run-all

Also, available Jupyter notebook mirror of this script

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Place to discuss use cases for this data

Showcases

Dashboards and visualizations based on these data

1. Dashboard for the US-1
2. Dashboard for the US-2
3. Dashboard for the world
4. Balefire COVID-19 USA Data Explorer
5. Pandemic Traffic in Ireland by David ó Cinnéide
6. New South Wales COVID Tracking Dashboard by Damjan Vlastelica
7. Global COVID Vital Signs
8. [Here can be your great dashboard/visualization]

Articles and research publications

1. Is Your Community Doing Enough To Fight COVID-19? by Molly Liebeskind
2. Project US Mobility and Fuel Demand Under COVID-19
3. COVID-19: Country progress tracker and forward projections
4. Krekel, C., Swanke, S., De Neve, J., & Fancourt, D. (2020). Are Happier People More Compliant? Global Evidence From Three Large-Scale Surveys During Covid-19 Lockdowns
5. Green shoots and mobility: Philippine economic prospects By Diwa C. Guinigundo
6. [Here can be your great article/research publication]