dimgold / Etl_with_python
ETL with Python - Taught at DWH course 2017 (TAU)
Stars: ✭ 68
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Etl with python
Griffon Vm
Griffon Data Science Virtual Machine
Stars: ✭ 128 (+88.24%)
Mutual labels: jupyter-notebook, data-science, database, mysql
Csv2db
The CSV to database command line loader
Stars: ✭ 102 (+50%)
Mutual labels: csv, etl, database, mysql
Awesome Business Intelligence
Actively curated list of awesome BI tools. PRs welcome!
Stars: ✭ 1,157 (+1601.47%)
Mutual labels: data-science, etl, database
Elastic
R client for the Elasticsearch HTTP API
Stars: ✭ 227 (+233.82%)
Mutual labels: data-science, etl, database
Openml R
R package to interface with OpenML
Stars: ✭ 81 (+19.12%)
Mutual labels: jupyter-notebook, data-science, database
Aws Data Wrangler
Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Stars: ✭ 2,385 (+3407.35%)
Mutual labels: data-science, etl, mysql
Datax
DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server
Stars: ✭ 116 (+70.59%)
Mutual labels: etl, database, mysql
Pgloader
Migrate to PostgreSQL in a single command!
Stars: ✭ 3,754 (+5420.59%)
Mutual labels: csv, database, mysql
Datacleaner
The premier open source Data Quality solution
Stars: ✭ 391 (+475%)
Mutual labels: data-science, etl, database
Web Database Analytics
Web scrapping and related analytics using Python tools
Stars: ✭ 175 (+157.35%)
Mutual labels: jupyter-notebook, data-science, database
Datasets For Recommender Systems
This is a repository of a topic-centric public data sources in high quality for Recommender Systems (RS)
Stars: ✭ 564 (+729.41%)
Mutual labels: jupyter-notebook, data-science, database
Fifa Fut Data
Web-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB
Stars: ✭ 55 (-19.12%)
Mutual labels: csv, database, mysql
Docker Backup Database
Docker image to periodically backup your database (MySQL, Postgres, or MongoDB) to S3 or local disk.
Stars: ✭ 57 (-16.18%)
Mutual labels: database, mysql
Datascience Projects
A collection of personal data science projects
Stars: ✭ 57 (-16.18%)
Mutual labels: jupyter-notebook, data-science
Rumble
⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Stars: ✭ 58 (-14.71%)
Mutual labels: csv, data-science
Storytelling With Data
Course materials for Dartmouth Course: Storytelling with Data (PSYC 81.09).
Stars: ✭ 59 (-13.24%)
Mutual labels: jupyter-notebook, data-science
Wertik Js
💪 A library that powers your app with GraphQL + Rest API
Stars: ✭ 56 (-17.65%)
Mutual labels: database, mysql
ETL with Python and MySQL
ETL with Python Training - Taught during Data Warehousing course - Tel Aviv University 2017
The Training is planned for ~2 hours and contains 4 notebook files:
-
jupyter-notebook.ipynb - quick Jupiter notebook introduction and tutorial
-
Mysql-io.ipynb - Input/Output to MySQL using
MySQLdbconnector -
ETL with Python.ipynb - ETL with python using
petlpackage -
Extra: CSV-io -
csvlibrary usage examples
Data files:
-
drinks.json - drinks consumption data (source)
-
drinks.zip - zipped json file (used for a zip file example in ETL notebook)
-
assuming you have default build-in world schema of MySQL (for mysql-io exercise). If not you can get it by running CreateWorld.sql - dump of sql scripy for creating world schema for mysql-io exercise.
-
file1.csv/file2.csv/flie3.csv - csv examples for
csvlibrary -
simpsons_phone_book.csv - csv example for
sniffer
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].