jobAnalytics and searchJobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.
Stars: ✭ 25 (-69.88%)
Aws Airflow StackTurbine: the bare metals that gets you Airflow
Stars: ✭ 352 (+324.1%)
airflow-ciApache Airflow CI pipeline
Stars: ✭ 18 (-78.31%)
Udacity Data Engineering ProjectsFew projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
Stars: ✭ 458 (+451.81%)
astroAstro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Stars: ✭ 79 (-4.82%)
Docker AirflowRepo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
Stars: ✭ 29 (-65.06%)
Airflow OperatorKubernetes custom controller and CRDs to managing Airflow
Stars: ✭ 278 (+234.94%)
airflow-boilerplateA complete development environment setup for working with Airflow
Stars: ✭ 94 (+13.25%)
Goodreads etl pipelineAn end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Stars: ✭ 793 (+855.42%)
airflow-code-editorA plugin for Apache Airflow that allows you to edit DAGs in browser
Stars: ✭ 195 (+134.94%)
airflow-dbtApache Airflow integration for dbt
Stars: ✭ 233 (+180.72%)
Agile data code 2Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Stars: ✭ 413 (+397.59%)
XeneA distributed workflow runner focusing on performance and simplicity.
Stars: ✭ 56 (-32.53%)
aircalVisualize Airflow's schedule by exporting future DAG runs as events to Google Calendar.
Stars: ✭ 66 (-20.48%)
ElyraElyra extends JupyterLab Notebooks with an AI centric approach.
Stars: ✭ 839 (+910.84%)
torchxTorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Stars: ✭ 165 (+98.8%)
helpdeskYet another helpdesk based on multiple providers
Stars: ✭ 14 (-83.13%)
ap-airflowAstronomer Core Docker Images
Stars: ✭ 87 (+4.82%)
Data Pipelines With Apache AirflowDeveloped a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3
Stars: ✭ 50 (-39.76%)
udacity-data-eng-proj2A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.
Stars: ✭ 25 (-69.88%)
Incubator DolphinschedulerApache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.
Stars: ✭ 6,916 (+8232.53%)
ecs-airflowCloudformation templates for deploying Airflow in ECS
Stars: ✭ 37 (-55.42%)
Airflow CookbookAirflow workflow management platform chef cookbook.
Stars: ✭ 58 (-30.12%)
openverse-catalogIdentifies and collects data on cc-licensed content across web crawl data and public apis.
Stars: ✭ 27 (-67.47%)
AirflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows
Stars: ✭ 24,101 (+28937.35%)
airflow-tutorialUse Airflow to move data from multiple MySQL databases to BigQuery
Stars: ✭ 96 (+15.66%)
ObjinsyncContinuously synchronize directories from remote object store to local filesystem
Stars: ✭ 29 (-65.06%)
Dag FactoryDynamically generate Apache Airflow DAGs from YAML configuration files
Stars: ✭ 385 (+363.86%)
Terraform Aws AirflowTerraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor
Stars: ✭ 69 (-16.87%)
viewflowViewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.
Stars: ✭ 110 (+32.53%)
ml-opsGet your MLOps (Level 1) platform started and going fast.
Stars: ✭ 81 (-2.41%)
Airflow Rest Api PluginA plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces
Stars: ✭ 281 (+238.55%)
incubator-liminalApache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
Stars: ✭ 117 (+40.96%)
Airflow ToolkitAny Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]
Stars: ✭ 51 (-38.55%)
opentrials-airflowConfiguration and definitions of Airflow for OpenTrials
Stars: ✭ 18 (-78.31%)
FastETLPlugins do Airflow para implementação de pipelines de dados
Stars: ✭ 31 (-62.65%)
DatabookA facebook for data
Stars: ✭ 26 (-68.67%)
dbt-on-airflowNo description or website provided.
Stars: ✭ 30 (-63.86%)
DataspherestudioDataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Stars: ✭ 1,195 (+1339.76%)
DiscreetlyETLy is an add-on dashboard service on top of Apache Airflow.
Stars: ✭ 60 (-27.71%)
Argo WorkflowsWorkflow engine for Kubernetes
Stars: ✭ 10,024 (+11977.11%)
bigkubeMinikube for big data with Scala and Spark
Stars: ✭ 16 (-80.72%)