180 Open source projects that are alternatives of or similar to DataflowTemplates

A Scala API for Apache Beam and Google Cloud Dataflow.

Export a whole BigQuery table to Google Datastore with Apache Beam/Google Dataflow

A Python framework for data processing on GCP.

Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.

GCP Variant Transforms

Tokenize Japanese text on BigQuery with Kuromoji in Apache Beam/Google Dataflow at scale

Quix Notebook Manager

An example mini data warehouse for python project stats, template for new projects

Stream data into Google BigQuery concurrently using InsertAll()

Load data from Kafka to any data warehouse

Another OLAP database

GRPC SQL and Spanner persistence layer

A collection of Magnolia add-on modules

Google BigQuery Datasource Plugin for Grafana.

ScalikeJDBC extension for Google BigQuery

a bot that generates realistic replies using a combination of pretrained GPT-2 and BERT models

Hive Storage Handler for interoperability between BigQuery and Apache Hive

Universalizando o acesso a dados no Brasil. Docs: https://basedosdados.github.io/mais/

NPM package for copying and converting Cloud Firestore data to BigQuery.

📊 Cube — Open-Source Analytics API for Building Data Apps

Logica is a logic programming language that compiles to StandardSQL and runs on Google BigQuery.

LINQ to BigQuery is C# LINQ Provider for Google BigQuery. It also enables Desktop GUI Client with LINQPad and plug-in driver.

A collection of tools for extracting FHIR resources and analytics services on top of that data.

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

DIY commercial datasets on Google Cloud Platform

Mercari Dataflow Template

RTLSDR ADS-B dump1090 to Google BigQuery

Export PostgreSQL tables to Google BigQuery

Automated feature engineering for geospatial data

GIS Visualisation for Amazon Athena and BigQuery

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

Google Search Console Logger for Google App Engine

A distributed locking library built on top of Cloud Spanner and TrueTime.

Cloud Dataflow Tutorial for Beginners

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Website analytics, JavaScript error tracking + analytics, tag manager, data ingest endpoint creation (tracking pixels). GDPR + CCPA compliant.

Common solutions and tools developed by Google Cloud's Professional Services team

Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub

Tools for building reactive user interfaces in ClojureScript.

Pandas Google BigQuery

An interface to Google's BigQuery from R.

Embulk output plugin to load/insert data into Google BigQuery

Starlake is a Spark Based On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. What datasets do you want to be added to Ethereum ETL? Vote here: https://blockchain-etl.convas.io.

Managed Machine Learning Systems and Internet of Things Live Lesson

Run templatable playbooks of SQL scripts in series and parallel on Redshift, PostgreSQL, BigQuery and Snowflake

Simulink toolbox to rapidly prototype robot controllers

DDL parase and Convert to BigQuery JSON schema and DDL statements

hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

Powerful product analytics for data teams, with full control over data & models.

tbls is a CI-Friendly tool for document a database, written in Go.

Dnai Editor - Visual Scripting (Node Editor)

Google-provided Cloud Dataflow template pipelines for solving simple in-Cloud data tasks

The Proxima platform.

Simple Python client for interacting with Google BigQuery.

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

Export your events from Amplitude to Google BigQuery/Google Cloud Storage

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub