GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → GoogleCloudPlatform → spark-on-k8s-gcp-examples

GoogleCloudPlatform / spark-on-k8s-gcp-examples

Licence: Apache-2.0 license
Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub

Programming Languages

java
68154 projects - #9 most used programming language
shell
77523 projects

Labels

kubernetesbigquerysparkgcscloud-pubsubgcs-connector

Projects that are alternatives of or similar to spark-on-k8s-gcp-examples

astro
Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Stars: ✭ 79 (+119.44%)
Mutual labels:  bigquery, gcs
google-cloud
A collection of Google Cloud Platform (GCP) plugins
Stars: ✭ 34 (-5.56%)
Mutual labels:  bigquery, gcs
Quix
Quix Notebook Manager
Stars: ✭ 184 (+411.11%)
Mutual labels:  bigquery
gsc-logger
Google Search Console Logger for Google App Engine
Stars: ✭ 38 (+5.56%)
Mutual labels:  bigquery
tame-gcs
📂 A small library with a limited set of Google Cloud Storage operations 🦀
Stars: ✭ 37 (+2.78%)
Mutual labels:  gcs
Geomancer
Automated feature engineering for geospatial data
Stars: ✭ 194 (+438.89%)
Mutual labels:  bigquery
firestore-to-bigquery-export
NPM package for copying and converting Cloud Firestore data to BigQuery.
Stars: ✭ 26 (-27.78%)
Mutual labels:  bigquery
Bitcoin Etl
ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ
Stars: ✭ 174 (+383.33%)
Mutual labels:  bigquery
pubsub cli
super handy google cloud Pub/Sub CLI
Stars: ✭ 32 (-11.11%)
Mutual labels:  cloud-pubsub
hive compared bq
hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.
Stars: ✭ 27 (-25%)
Mutual labels:  bigquery
bigquery-data-lineage
Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.
Stars: ✭ 112 (+211.11%)
Mutual labels:  bigquery
flight2bq
RTLSDR ADS-B dump1090 to Google BigQuery
Stars: ✭ 33 (-8.33%)
Mutual labels:  bigquery
Mprove
Open source Business Intelligence tool 🎉
Stars: ✭ 212 (+488.89%)
Mutual labels:  bigquery
tag-manager
Website analytics, JavaScript error tracking + analytics, tag manager, data ingest endpoint creation (tracking pixels). GDPR + CCPA compliant.
Stars: ✭ 279 (+675%)
Mutual labels:  bigquery
Bigquery Grafana
Google BigQuery Datasource Plugin for Grafana.
Stars: ✭ 188 (+422.22%)
Mutual labels:  bigquery
scalikejdbc-bigquery
ScalikeJDBC extension for Google BigQuery
Stars: ✭ 18 (-50%)
Mutual labels:  bigquery
Scio
A Scala API for Apache Beam and Google Cloud Dataflow.
Stars: ✭ 2,247 (+6141.67%)
Mutual labels:  bigquery
Pandas Gbq
Pandas Google BigQuery
Stars: ✭ 243 (+575%)
Mutual labels:  bigquery
managed ml systems and iot
Managed Machine Learning Systems and Internet of Things Live Lesson
Stars: ✭ 35 (-2.78%)
Mutual labels:  bigquery
gobroker
golang wrapper for all (to-be) kinds of message brokers
Stars: ✭ 15 (-58.33%)
Mutual labels:  cloud-pubsub
View All Similar Projects ➔

This is not an official Google product.

Accessing GCP Products in Spark

This repository is for examples showing how to access Google Cloud Platform products such as Google Cloud Storage (GCS), BigQuery, and Cloud PubSub as data sources or sinks in Spark on Kubernetes applications. Please refer to the README in each example directory for details on how to build and run the example.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].