stefanobaghino / Spark Twitter Stream Example
"Sentiment analysis" on a live Twitter feed with Apache Spark and Apache Bahir
Stars: ✭ 73
Programming Languages
scala
5932 projects
Projects that are alternatives of or similar to Spark Twitter Stream Example
Flink Learning
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
Stars: ✭ 11,378 (+15486.3%)
Mutual labels: spark, streaming
Azure Event Hubs
☁️ Cloud-scale telemetry ingestion from any stream of data with Azure Event Hubs
Stars: ✭ 233 (+219.18%)
Mutual labels: spark, streaming
Spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Stars: ✭ 1,721 (+2257.53%)
Mutual labels: spark, streaming
Sparta
Real Time Analytics and Data Pipelines based on Spark Streaming
Stars: ✭ 513 (+602.74%)
Mutual labels: spark, streaming
Azure Event Hubs Spark
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
Stars: ✭ 140 (+91.78%)
Mutual labels: spark, streaming
Laravel Twitter Streaming Api
Easily work with the Twitter Streaming API in a Laravel app
Stars: ✭ 153 (+109.59%)
Mutual labels: streaming, twitter
kafka-spark-streaming-zeppelin-docker
One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)
Stars: ✭ 82 (+12.33%)
Mutual labels: streaming, spark
kafka-compose
🎼 Docker compose files for various kafka stacks
Stars: ✭ 32 (-56.16%)
Mutual labels: twitter, spark
Data Accelerator
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Stars: ✭ 247 (+238.36%)
Mutual labels: spark, streaming
Badusb botnet
👥😈 Infect a pc with badusb and establish a connection through telegram.
Stars: ✭ 32 (-56.16%)
Mutual labels: streaming, twitter
Mobius
C# and F# language binding and extensions to Apache Spark
Stars: ✭ 929 (+1172.6%)
Mutual labels: spark, streaming
W2v
Word2Vec models with Twitter data using Spark. Blog:
Stars: ✭ 64 (-12.33%)
Mutual labels: spark, twitter
Akka Http
The Streaming-first HTTP server/module of Akka
Stars: ✭ 1,163 (+1493.15%)
Mutual labels: streaming
Athenax
SQL-based streaming analytics platform at scale
Stars: ✭ 1,178 (+1513.7%)
Mutual labels: streaming
Usersessionbehaviorofflineanalysis
四川大学拓思爱诺用户session行为数据离线分析项目
Stars: ✭ 69 (-5.48%)
Mutual labels: spark
Twitter Webhook Boilerplate Node
A simple Node.js app using Express 4 for Twitter DMs and webhooks.
Stars: ✭ 69 (-5.48%)
Mutual labels: twitter
Skraper
Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, IFunny, VK, Pikabu)
Stars: ✭ 72 (-1.37%)
Mutual labels: twitter
Spark Twitter Stream Example
A few lines of code to demo how streaming works with Spark, in particular using the extensions provided by Apache Bahir to read a live stream of tweets, which will be processed to assign it a sentiment score (using a very naive algorithm).
To make it work on your installation, be sure to add a twitter4j.properties under src/main/resources that includes the following information:
oauth.consumerKey=***
oauth.consumerSecret=***
oauth.accessToken=***
oauth.accessTokenSecret=***
Visit apps.twitter.com to get your own API keys.
To submit the job to an existing Spark installation you can package the job with the following command:
sbt package
and then submit it with the following command:
$SPARK_HOME/bin/spark-submit \
--master $SPARK_MASTER \
--jars $DEPENDENCIES \
--class me.baghino.spark.streaming.twitter.example.TwitterSentimentScore \
target/scala-2.11/spark-twitter-stream-example-assembly-1.0.0.jar
The Spark classpath should include org.apache.bahir:spark-streaming-twitter_2.11:2.0.1, org.twitter4j:twitter4j-core:4.0.4 and org.twitter4j:twitter4j-stream:4.0.4.
After running the sbt package command you'll find the required JARs in your local Ivy cache ($HOME/.ivy2/cache/).
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].