All Projects → mkuthan → Example Spark

mkuthan / Example Spark

Licence: other
Spark, Spark Streaming and Spark SQL unit testing strategies

Programming Languages

scala
5932 projects

Projects that are alternatives of or similar to Example Spark

Real Time Stream Processing Engine
This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.
Stars: ✭ 37 (-81.95%)
Mutual labels:  spark, spark-streaming
Waterdrop
Production Ready Data Integration Product, documentation:
Stars: ✭ 1,856 (+805.37%)
Mutual labels:  spark, spark-streaming
Utils4s
scala、spark使用过程中,各种测试用例以及相关资料整理
Stars: ✭ 1,070 (+421.95%)
Mutual labels:  spark, spark-streaming
Angel
A Flexible and Powerful Parameter Server for large-scale machine learning
Stars: ✭ 6,458 (+3050.24%)
Mutual labels:  spark, spark-streaming
Example Spark Kafka
Apache Spark and Apache Kafka integration example
Stars: ✭ 120 (-41.46%)
Mutual labels:  spark, spark-streaming
Mobius
C# and F# language binding and extensions to Apache Spark
Stars: ✭ 929 (+353.17%)
Mutual labels:  spark, spark-streaming
Spark States
Custom state store providers for Apache Spark
Stars: ✭ 83 (-59.51%)
Mutual labels:  spark, spark-streaming
Coolplayspark
酷玩 Spark: Spark 源代码解析、Spark 类库等
Stars: ✭ 3,318 (+1518.54%)
Mutual labels:  spark, spark-streaming
Spark Streaming With Kafka
Self-contained examples of Apache Spark streaming integrated with Apache Kafka.
Stars: ✭ 180 (-12.2%)
Mutual labels:  spark, spark-streaming
Kinesis Sql
Kinesis Connector for Structured Streaming
Stars: ✭ 120 (-41.46%)
Mutual labels:  spark, spark-streaming
Sparta
Real Time Analytics and Data Pipelines based on Spark Streaming
Stars: ✭ 513 (+150.24%)
Mutual labels:  spark, spark-streaming
Azure Event Hubs Spark
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
Stars: ✭ 140 (-31.71%)
Mutual labels:  spark, spark-streaming
Cdap
An open source framework for building data analytic applications.
Stars: ✭ 509 (+148.29%)
Mutual labels:  spark, spark-streaming
Learning Spark
零基础学习spark,大数据学习
Stars: ✭ 37 (-81.95%)
Mutual labels:  spark, spark-streaming
Learningspark
Scala examples for learning to use Spark
Stars: ✭ 421 (+105.37%)
Mutual labels:  spark, spark-streaming
Pyspark Examples
Code examples on Apache Spark using python
Stars: ✭ 58 (-71.71%)
Mutual labels:  spark, spark-streaming
Gimel
Big Data Processing Framework - Unified Data API or SQL on Any Storage
Stars: ✭ 216 (+5.37%)
Mutual labels:  spark, spark-streaming
Data Accelerator
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Stars: ✭ 247 (+20.49%)
Mutual labels:  spark, spark-streaming
Spark Mllib Twitter Sentiment Analysis
🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib
Stars: ✭ 113 (-44.88%)
Mutual labels:  spark, spark-streaming
Spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Stars: ✭ 1,721 (+739.51%)
Mutual labels:  spark, spark-streaming

Spark, Spark Streaming and Spark SQL unit testing strategies

Build Status Coverage Status

Features

  • ClockWrapper for efficient clock management in Spark Streaming jobs.
  • Base traits for testing Spark, Spark Streaming and Spark SQL to eliminate boilerplate code.
  • Sample applications to show how to make your code testable.
  • All tests can be run or debugged directly from IDE, or using SBT.
  • All test fixtures are prepared as in-memory data structures.
  • SBT is configured to avoid problems with multiple Spark contexts in the same JVM SPARK-2243.
  • SBT is configured to prepare project assembly for deployment on the cluster.

References

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].