tomaszdudek7 / Airflow_project
Licence: mit
scaffold of Apache Airflow executing Docker containers
Stars: ✭ 84
Labels
Projects that are alternatives of or similar to Airflow project
Spacenet building detection
Project to train/test convolutional neural networks to extract buildings from SpaceNet satellite imageries.
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Esda
statistics and classes for exploratory spatial data analysis
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Python script Manual
《Python工具代码速查手册》是我们的python培训教材，主要面向数据分析方向。其中包含了python的常用总结性操作，使用jupyter notebook，利用markdown和script结果对常用操作进行总结，包括了使用方式和脚本。之所以使用notebook形式是可以方便大家编辑，方便大家形成自己的总结笔记。当然各位有更好的操作建议也欢迎向我们团队分享~
Stars: ✭ 84 (+0%)
Mutual labels: jupyter-notebook
Deep Knowledge Tracing Plus
It is the DKT+ model implemented in python3 and tensorflow1.2
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Torchtexttutorial
A short tutorial for Torchtext, the NLP-specific add-on for Pytorch.
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Pylbm
Numerical simulations using flexible Lattice Boltzmann solvers
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Tensor Learning
Python codes for low-rank tensor factorization, tensor completion, and tensor regression techniques.
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Gancs
Compressed Sensing MRI based on Deep Generative Adversarial Network
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Logomaker
Software for the visualization of sequence-function relationships
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Jupyter case studies
Inference case studies in jupyter
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Ml
Machine learning projects, often on audio datasets
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Deepembeding
图像检索和向量搜索，similarity learning,compare deep metric and deep-hashing applying in image retrieval
Stars: ✭ 83 (-1.19%)
Mutual labels: jupyter-notebook
Yet Another Scalable Apache Airflow With Docker Example Setup
This repository contains the code for article https://medium.com/@tomaszdudek/yet-another-scalable-apache-airflow-with-docker-example-setup-84775af5c451. It features a scaffold of Apache Airflow, whose workers instead of executing the job themselves, run Docker containers and wait until they complete.
Requirements
- Linux or MacOS
- Docker and docker-compose
- Python 3.6 (with
docker-pyinstalled)
How to run example
- clone the project
- go to
./airflow/and rundocker build . -t puckel-airflow-with-docker-inside:latest - make sure you have docker-py (tested with version
docker==3.7.0- careful because usingdocker-py==1.10.xwon't work) and runpython build_images.py - go to
./airflowand rundocker-compose up -f docker-compose-linux.yml(or justupwhen using OSX) - launch http://localhost:8080/admin/, ensure that DAG is ON:
- trigger the DAG:
- watch the pipeline as it launches Docker containers:
Modify
- feel free to make PR's
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].