GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → teamclairvoyant → apache-airflow-cloudera-parcel

teamclairvoyant / apache-airflow-cloudera-parcel

Licence: Apache-2.0 license
Parcel for Apache Airflow

Programming Languages

Dockerfile
14818 projects
shell
77523 projects
python
139335 projects - #7 most used programming language

Labels

airflowclouderaparcel

Projects that are alternatives of or similar to apache-airflow-cloudera-parcel

pipeline
PipelineAI Kubeflow Distribution
Stars: ✭ 4,154 (+25862.5%)
Mutual labels:  airflow
twitch-extension-starter
Kickstarts your Twitch Extension using React
Stars: ✭ 38 (+137.5%)
Mutual labels:  parcel
incremental training
Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'
Stars: ✭ 110 (+587.5%)
Mutual labels:  airflow
aircan
💨🥫 A Data Factory system for running data processing pipelines built on AirFlow and tailored to CKAN. Includes evolution of DataPusher and Xloader for loading data to DataStore.
Stars: ✭ 24 (+50%)
Mutual labels:  airflow
kedro-airflow-k8s
Kedro Plugin to support running pipelines on Kubernetes using Airflow.
Stars: ✭ 22 (+37.5%)
Mutual labels:  airflow
Insight-GDELT-Feed
A way for home buyers to know about factors affecting a state
Stars: ✭ 43 (+168.75%)
Mutual labels:  airflow
react-redux-boilerplate
A React boilerplate based on Redux, React Router, styled components and Parcel
Stars: ✭ 62 (+287.5%)
Mutual labels:  parcel
parcel-vuejs-starterkit
⚡ parcel-bundle + vuejs + vue-router demo
Stars: ✭ 13 (-18.75%)
Mutual labels:  parcel
fab-oidc
Flask-AppBuilder SecurityManager for OpenIDConnect
Stars: ✭ 28 (+75%)
Mutual labels:  airflow
auto-parcel
A fast annotation processor to make your objects `Parcelable` without writing any of the boilerplate.
Stars: ✭ 80 (+400%)
Mutual labels:  parcel
AirflowETL
Blog post on ETL pipelines with Airflow
Stars: ✭ 20 (+25%)
Mutual labels:  airflow
purescript-parcel-example
An example of how PureScript outputs are just CommonJS modules that you can use in any way you want.
Stars: ✭ 19 (+18.75%)
Mutual labels:  parcel
airflow-site
Apache Airflow Website
Stars: ✭ 95 (+493.75%)
Mutual labels:  airflow
parcel-plugin-externals
Parcel plugin for declaring externals. These externals will not be bundled. 📦
Stars: ✭ 47 (+193.75%)
Mutual labels:  parcel
T-Watch
Real Time Twitter Sentiment Analysis Product
Stars: ✭ 20 (+25%)
Mutual labels:  airflow
parcel-vue-demo
📦 parcel demo/example for vue
Stars: ✭ 19 (+18.75%)
Mutual labels:  parcel
kedro-airflow
Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.
Stars: ✭ 121 (+656.25%)
Mutual labels:  airflow
k3ai
A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.
Stars: ✭ 105 (+556.25%)
Mutual labels:  airflow
fairflow
Functional Airflow DAG definitions.
Stars: ✭ 38 (+137.5%)
Mutual labels:  airflow
svelte-box
A truffle box for svelte
Stars: ✭ 60 (+275%)
Mutual labels:  parcel
View All Similar Projects ➔

Airflow Parcel

This repository allows you to install Apache Airflow as a parcel deployable by Cloudera Manager.

Requirements

  • A supported operating system.
  • MySQL or PostgreSQL database in which to store Airflow metadata.

Currently Supported Versions of Airflow

  • Airflow 1.9.0
  • Airflow 1.10.3

Currently Supported Operating Systems

  • CentOS/RHEL 6 & 7
  • Debian 8
  • Ubuntu 14.04, 16.04, & 18.04

Installing the Parcel

  1. First, install the Airflow CSD. Then you can skip steps #1 and #2.
  2. In Cloudera Manager, go to Hosts -> Parcels -> Configuration.
  3. Add http://archive.clairvoyantsoft.com/airflow/parcels/latest/ to the Remote Parcel Repository URLs if it does not yet exist.
  4. In Cloudera Manager, go to Hosts -> Parcels. Airflow parcels and their respective versions will be available within the Parcels page.
  5. Download, Distribute, Activate the required parcels to use them.

Building the Parcel

  1. Install Docker and Python.
  2. Run the script build_airflow_parcel.sh by executing:
./build_airflow_parcel.sh --airflow <airflow_version> --python <python_version> --parcel <parcel_version>
  1. Output will be placed in the target/ directory.
  2. Use ./serve_parcel.sh to serve this directory via HTTP, or move the entire directory contents to your own webserver.

Resources:

  1. https://github.com/teamclairvoyant/apache-airflow-cloudera-csd
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].