All Projects → ocadaruma → scalikejdbc-bigquery

ocadaruma / scalikejdbc-bigquery

Licence: Apache-2.0 license
ScalikeJDBC extension for Google BigQuery

Programming Languages

scala
5932 projects

Projects that are alternatives of or similar to scalikejdbc-bigquery

Spark Bigquery Connector
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
Stars: ✭ 126 (+600%)
Mutual labels:  bigquery
Geomancer
Automated feature engineering for geospatial data
Stars: ✭ 194 (+977.78%)
Mutual labels:  bigquery
managed ml systems and iot
Managed Machine Learning Systems and Internet of Things Live Lesson
Stars: ✭ 35 (+94.44%)
Mutual labels:  bigquery
Mara Example Project 2
An example mini data warehouse for python project stats, template for new projects
Stars: ✭ 154 (+755.56%)
Mutual labels:  bigquery
Quix
Quix Notebook Manager
Stars: ✭ 184 (+922.22%)
Mutual labels:  bigquery
Hadoop Connectors
Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.
Stars: ✭ 218 (+1111.11%)
Mutual labels:  bigquery
Professional Services
Common solutions and tools developed by Google Cloud's Professional Services team
Stars: ✭ 1,923 (+10583.33%)
Mutual labels:  bigquery
bigquery-data-lineage
Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.
Stars: ✭ 112 (+522.22%)
Mutual labels:  bigquery
Bigquery Grafana
Google BigQuery Datasource Plugin for Grafana.
Stars: ✭ 188 (+944.44%)
Mutual labels:  bigquery
hive compared bq
hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.
Stars: ✭ 27 (+50%)
Mutual labels:  bigquery
Gpt2 Bert Reddit Bot
a bot that generates realistic replies using a combination of pretrained GPT-2 and BERT models
Stars: ✭ 158 (+777.78%)
Mutual labels:  bigquery
Scio
A Scala API for Apache Beam and Google Cloud Dataflow.
Stars: ✭ 2,247 (+12383.33%)
Mutual labels:  bigquery
Pandas Gbq
Pandas Google BigQuery
Stars: ✭ 243 (+1250%)
Mutual labels:  bigquery
Go Bqstreamer
Stream data into Google BigQuery concurrently using InsertAll()
Stars: ✭ 133 (+638.89%)
Mutual labels:  bigquery
firestore-to-bigquery-export
NPM package for copying and converting Cloud Firestore data to BigQuery.
Stars: ✭ 26 (+44.44%)
Mutual labels:  bigquery
Mais
Universalizando o acesso a dados no Brasil. Docs: https://basedosdados.github.io/mais/
Stars: ✭ 122 (+577.78%)
Mutual labels:  bigquery
Mprove
Open source Business Intelligence tool 🎉
Stars: ✭ 212 (+1077.78%)
Mutual labels:  bigquery
gsc-logger
Google Search Console Logger for Google App Engine
Stars: ✭ 38 (+111.11%)
Mutual labels:  bigquery
tag-manager
Website analytics, JavaScript error tracking + analytics, tag manager, data ingest endpoint creation (tracking pixels). GDPR + CCPA compliant.
Stars: ✭ 279 (+1450%)
Mutual labels:  bigquery
flight2bq
RTLSDR ADS-B dump1090 to Google BigQuery
Stars: ✭ 33 (+83.33%)
Mutual labels:  bigquery

scalikejdbc-bigquery

Build Status

ScalikeJDBC extension for Google BigQuery

Prerequisites

  • Java 8
  • Scala 2.11.x / 2.12.x / 2.13.x
  • ScalikeJDBC 2.4.x or newer

Installation

libraryDependencies ++= Seq(
  "com.mayreh" %% "scalikejdbc-bigquery" % "0.1.4",
  "com.google.cloud" % "google-cloud-bigquery" % "1.51.0",
  "org.scalikejdbc" %% "scalikejdbc" % scalikejdbcVersion // specify scalikejdbc version you want. 
)

Usage

import com.google.auth.oauth2.GoogleCredentials
import com.google.cloud.bigquery.{BigQueryOptions, DatasetId}
import scalikejdbc._, bigquery._

// instantiate BigQuery service and DatsetId
val credentials = GoogleCredentials.fromStream(new FileInputStream("/path/to/key.json"))
val bigQuery = BigQueryOptions.newBuilder()
  .setCredentials(credentials)
  .setProjectId("your-gcp-project-id")
  .build()
  .getService
  
val dataset = DatasetId.of("your-gcp-project-id", "your-dataset")

// build query by QueryDSL then execute
val executor = new QueryExecutor(bigQuery, QueryConfig())

val response = bq {
  selectFrom(User in dataset as u)
    .where.eq(u.id, 42)
}.map(User(_)).single.run(executor)

response.result // => Option[User]

See also integration test.

Current status

This project is still in an early stage. Newer releases can include breaking changes.

Be careful using in production.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].