Coronavirus Dresden

This script collects official infection statistics published by the city of Dresden and saves them to InfluxDB. From there the data can be processed and visualised using the SQL-like query language InfluxQL and, for instance, Grafana.

Subsequent changes to the published data set can also be detected and routinely logged.

• Data sets are archived here.
• And you can have a look at my dashboard here.

Data source

The raw data provided by the city of Dresden and visualised on their Dashboard is obtained from the following source:

• Overview over all tables and data fields
• Web form for queries
• Download the latest JSON dump

Data is available under an open licence compatible with CC-BY: Landeshauptstadt Dresden, dl-de/by-2-0, opendata.dresden.de.

Install

Get this repository:

git clone https://github.com/jdieg0/coronavirus-dresden.git

If desired, the data archive can be retrieved with:

cd coronavirus-dresden
git submodule update --init --recursive

If you want to load new data in the future, git pull inside the subdirectory:

cd data
git checkout main
git pull

Using a virtual environment of your choice for Python is recommended. An exemplary installation with venv is described below.

venv (recommended)

Set up Python environment:

python3 -m venv venv
source venv/bin/activate

pip install -r requirements.txt

InfluxDB (required)

macOS

brew install influxdb
brew services start influxdb

Ubuntu

wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
source /etc/lsb-release
echo "deb https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
sudo apt-get update && sudo apt-get install influxdb
sudo service influxdb start

Python API

Helpful resources:

• Getting Started with Python and InfluxDB

Grafana (optional)

mac OS

brew install grafana
brew services start grafana

Run

python collect.py

cron

To search for new data regularly, enter:

sudo crontab -e

Add the following line to run the script every 5 minutes (adapt paths to suit your own installation):

*/5 * * * * /root/bin/coronavirus-dresden/venv/bin/python /root/bin/coronavirus-dresden/collect.py --log --archive-json

If you just want to routinely save new published JSON files and are not interested in saving the data to InfluxDB, you can do this by typing:

python collect.py --skip-influxdb

Command line arguments

To display all data collection options, type:

python collect.py --help