pushshift / Reddit_sse_stream
Licence: mit
A Server Side Event stream to deliver Reddit comments and submissions in near real-time to a client.
Stars: ✭ 39
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Reddit sse stream
Flask reddit
Reddit clone in flask + python + nginx + https. View site:
Stars: ✭ 482 (+1135.9%)
Mutual labels: flask, reddit
Flinkstreamsql
基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
Stars: ✭ 1,682 (+4212.82%)
Mutual labels: stream, bigdata
Azure Event Hubs Spark
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
Stars: ✭ 140 (+258.97%)
Mutual labels: stream, bigdata
flink-learn
Learning Flink : Flink CEP,Flink Core,Flink SQL
Stars: ✭ 70 (+79.49%)
Mutual labels: stream, bigdata
Jiosaavnapi
An unofficial API for JioSaavn written in Python 3
Stars: ✭ 123 (+215.38%)
Mutual labels: stream, flask
http-event-stream
📡 Modern spec-compliant Server Sent Events stream implementation.
Stars: ✭ 16 (-58.97%)
Mutual labels: stream, sse
Spark Movie Lens
An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset
Stars: ✭ 745 (+1810.26%)
Mutual labels: bigdata, flask
Pymap webapp
A webapp version for Raster Map Download Helper
Stars: ✭ 34 (-12.82%)
Mutual labels: flask
Pulsar Beam
Pulsar Beam is a streaming service via HTTP built on Apache Pulsar.
Stars: ✭ 37 (-5.13%)
Mutual labels: sse
Botvid 19
Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku
Stars: ✭ 34 (-12.82%)
Mutual labels: flask
Flask Json
Flask-JSON is a Flask extension providing better JSON support.
Stars: ✭ 34 (-12.82%)
Mutual labels: flask
Tensorflowjsclientsideprediction
An example of a client side prediction made with TensorFlowJS
Stars: ✭ 34 (-12.82%)
Mutual labels: flask
Node Streamspeed
A simple way to keep track of the speed of your node streams.
Stars: ✭ 33 (-15.38%)
Mutual labels: stream
Reddit Swiftui
A cross-platform Reddit client built in SwiftUI
Stars: ✭ 967 (+2379.49%)
Mutual labels: reddit
Big Album Art
[RETIRED] A Flask app to display almost-fullscreen album art for your currently playing Spotify songs. Enjoy the visuals!
Stars: ✭ 38 (-2.56%)
Mutual labels: flask
Cookiecutter Flask Pythonic
Pythonic starter boilerplate for Flask
Stars: ✭ 37 (-5.13%)
Mutual labels: flask
Reddit SSE Stream
Link: http://stream.pushshift.io/
What is this?
This is an SSE stream that you can connect to using a browser or other programs to get a live feed of near real-time Reddit data (couple seconds delayed).
There is a caveat however -- SSE streams currently only work via http (no SSL). However, it does support compression! I highly encourage you to connect using compression headers to conserve bandwidth if you use the full feed.
curl --verbose --compressed 'http://stream.pushshift.io/?type=comments'
Parameters Supported
| Parameter | Description | Possible Values |
|---|---|---|
| type | Restrict feed to a particular event type. Leave out to receive all event types. | comments,submissions |
| author | Restrict to certain authors. Separate multiple authors with a comma. | String |
| domain } Restrict submissions to a domain or group of domains (Comma-delimited) | String | |
| submission | Restrict to certain subreddits. Separate multiple subreddits with a comma. | String |
| submission_backfill | Starts stream X submissions ago (up to 25,000 submissions) | INT (1-100000) |
| comment_backfill | Starts stream X comments ago (up to 100,000 submissions) | INT (1-100000) |
| submission_start_id | Starts stream at specific submission ID (Used for continuation if the stream is broken | INT |
| comment_start_id | Starts stream at specific comment ID (Used for continuation if the stream is broken) | INT |
| over_18 | Restrict submissions in stream to NSFW or SFW content | Boolean (true or false) |
| is_self | Restrict submissions in stream to self posts or linked posts | Boolean (true or false) |
| filter | Restrict the returned JSON object to specific keys (comma-delimited for multiple values) | String |
Examples:
Get only submissions from news and politics:
curl --verbose --compressed 'http://stream.pushshift.io/?type=submissions&subreddit=politics,news'
Get only comments from automoderator:
curl --verbose --compressed 'http://stream.pushshift.io/?type=comments&author=automoderator'
Notice: When connecting with compression. It generally takes a little bit of time before it gets started. Also, only one connection per IP is supported. If you need additional streams, please contact me.
Event Types:
| Event | Description | Currently Implemented |
|---|---|---|
| rc | A new Reddit comment in json format | Yes |
| rs | A new Reddit submission in json format | Yes |
| rr | A new Reddit subreddit in json format | No |
| keepalive | A keepalive message (every 30 seconds) | Yes |
Keepalive Messages:
Currently, a keepalive event is sent to the client every 30 seconds regardless if previous data was sent. The keepalive event has the following information:
| Key | Value Description |
|---|---|
| tcs | Total Comments Sent |
| tss | Total Submissions Sent |
| tbs | Total bytes sent (uncompressed) |
Python Consumer Example
pip install sseclient (From this wonderful github: https://github.com/btubbs/sseclient)
Take a look at the consumer_example.py file that shows how to interact with the stream using Python and sseclient.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].