GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → vitorbaptista → Google Covid19 Mobility Reports

vitorbaptista / Google Covid19 Mobility Reports

Licence: mit
Data extraction of Google's COVID-19 Mobility Reports

Labels

htmldatasetscraping

Projects that are alternatives of or similar to Google Covid19 Mobility Reports

Mtnt
Code for the collection and analysis of the MTNT dataset
Stars: ✭ 48 (-41.46%)
Mutual labels:  dataset, scraping
Recursive Cnns
Implementation of my paper "Real-time Document Localization in Natural Images by Recursive Application of a CNN."
Stars: ✭ 80 (-2.44%)
Mutual labels:  dataset
Sketchyscene
SketchyScene: Richly-Annotated Scene Sketches. (ECCV 2018)
Stars: ✭ 74 (-9.76%)
Mutual labels:  dataset
Detect Cms
PHP Library for detecting CMS
Stars: ✭ 78 (-4.88%)
Mutual labels:  scraping
Nimquery
Nim library for querying HTML using CSS-selectors (like JavaScripts document.querySelector)
Stars: ✭ 75 (-8.54%)
Mutual labels:  scraping
Setl
A simple Spark-powered ETL framework that just works 🍺
Stars: ✭ 79 (-3.66%)
Mutual labels:  dataset
Raccoon dataset
The dataset is used to train my own raccoon detector and I blogged about it on Medium
Stars: ✭ 1,177 (+1335.37%)
Mutual labels:  dataset
Vidvrd Helper
To keep updates with VRU Grand Challenge, please use https://github.com/NExTplusplus/VidVRD-helper
Stars: ✭ 81 (-1.22%)
Mutual labels:  dataset
Pointclouddatasets
3D point cloud datasets in HDF5 format, containing uniformly sampled 2048 points per shape.
Stars: ✭ 80 (-2.44%)
Mutual labels:  dataset
Viewstate
ASP.NET View State Decoder
Stars: ✭ 77 (-6.1%)
Mutual labels:  scraping
La3dm
Learning-aided 3D mapping
Stars: ✭ 77 (-6.1%)
Mutual labels:  dataset
Color Names
Large list of handpicked color names 🌈
Stars: ✭ 1,198 (+1360.98%)
Mutual labels:  dataset
Symbolic Musical Datasets
🎹 symbolic musical datasets
Stars: ✭ 79 (-3.66%)
Mutual labels:  dataset
Tju Dhd
A newly built high-resolution dataset for object detection and pedestrian detection (IEEE TIP 2020)
Stars: ✭ 75 (-8.54%)
Mutual labels:  dataset
Atis dataset
The ATIS (Airline Travel Information System) Dataset
Stars: ✭ 81 (-1.22%)
Mutual labels:  dataset
Mmsa
CH-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotations of Modality (ACL2020)
Stars: ✭ 70 (-14.63%)
Mutual labels:  dataset
Facegrab
A tool to collect public images from Facebook and create an image dataset for training computer vision applications like gender recognition, and face detection
Stars: ✭ 76 (-7.32%)
Mutual labels:  dataset
Urbannavdataset
UrbanNav: an Open-Sourcing Localization Data Collected in Asian Urban Canyons, Including Tokyo and Hong Kong
Stars: ✭ 79 (-3.66%)
Mutual labels:  dataset
Openml R
R package to interface with OpenML
Stars: ✭ 81 (-1.22%)
Mutual labels:  dataset
Email Extractor
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (-1.22%)
Mutual labels:  scraping
View All Similar Projects ➔

_This project is archived. Google now provides the full data as a CSV, so it isn't needed anymore. _

Google COVID-19 Mobility Reports

Archive of the Google COVID-19 Mobility Reports' PDFs and their data extracted as a CSV.

Motivation

Google launched a series of reports on their user's mobility during the Coronavirus pandemic. According to their website, the reports "chart movement trends over time by geography, across different categories of places such as retail and recreation, groceries and pharmacies, parks, transit stations, workplaces, and residential."

https://www.google.com/covid19/mobility/

This data is very interesting and important during these times. However, it's only available as a PDF per country (or per State in the USA). This project has two goals:

  1. Provide an archive of the reports
  2. Extract this data into a more convenient format (CSV or JSON)

Where is the data?

How reliable is the CSV?

I've been very careful in checking that the data in the CSV corresponds to the data in the PDF. There are tests to reduce the possibility of bugs being introduced, and I manually checked a few of the results.

That being said, there are no guarantees that the data is error-free. I recommend double checking the CSV with the raw PDFs if you want to be sure. If you find any errors, please create an issue.

Installing

To run it locally, you need pdftotext and a recent Python version (tested on 3.8, but it should work on 3.6+). You can then download any new reports and parse the files by running make.

This will download the main Mobility Reports' HTML page, saving it into data/raw/html, then download any new PDF reports into data/raw/reports, and finally parse them, saving the results to data/processed/mobility_reports.csv.

If you want to convert a single report PDF to CSV, run:

PYTHONPATH=. python mobility_reports/cli.py <PATH_TO_THE_PDF>

This command accepts multiple paths, and outputs the CSV to stdout.

License

The data is copyrighted by Google. Everything else is licensed under the MIT License.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].