All Projects → streamwall → stream-list-updater

streamwall / stream-list-updater

Licence: MIT license
Automation for updating an index of live George Floyd protest streams

Programming Languages

javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to stream-list-updater

Scrape Twitter
🐦 Access Twitter data without an API key. [DEPRECATED]
Stars: ✭ 166 (+1006.67%)
Mutual labels:  scraper, streams
BakingSheet
Easy datasheet management for C# and Unity. Supports Excel, Google Sheet, JSON and CSV format.
Stars: ✭ 144 (+860%)
Mutual labels:  spreadsheet
dorkscout
DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targets
Stars: ✭ 189 (+1160%)
Mutual labels:  scraper
MalScraper
Scrape everything you can from MyAnimeList.net
Stars: ✭ 132 (+780%)
Mutual labels:  scraper
ShareScreenServer
share android screen live on udp
Stars: ✭ 21 (+40%)
Mutual labels:  live
immo-feed
A extensible app for scraping property listings
Stars: ✭ 35 (+133.33%)
Mutual labels:  scraper
async-stream-generator
Pipe ES6 Async Generators through Node.js Streams
Stars: ✭ 48 (+220%)
Mutual labels:  streams
nest-crawler
An easiest crawling and scraping module for NestJS
Stars: ✭ 45 (+200%)
Mutual labels:  scraper
TinderBotz
Automated Tinder bot and scraper using selenium in python.
Stars: ✭ 265 (+1666.67%)
Mutual labels:  scraper
philan.net
Public Donation Management webservice for Philanthropist.
Stars: ✭ 35 (+133.33%)
Mutual labels:  spreadsheet
streamalg
Extensible stream pipelines with object algebras.
Stars: ✭ 26 (+73.33%)
Mutual labels:  streams
gesetze-tools
Scripts to maintain German law git repository
Stars: ✭ 99 (+560%)
Mutual labels:  scraper
Image-Scraper
Fast concurrent image scraper
Stars: ✭ 35 (+133.33%)
Mutual labels:  scraper
TikTokDownloader PyWebIO
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具,支持API调用,在线批量解析及下载。
Stars: ✭ 919 (+6026.67%)
Mutual labels:  scraper
twitch-youtube-restream-chat-overlay
Access the YouTube Live chat and route it to your OBS or VMix Browser source.
Stars: ✭ 52 (+246.67%)
Mutual labels:  live
ceiba-dl
NTU CEIBA 資料下載工具
Stars: ✭ 80 (+433.33%)
Mutual labels:  scraper
js-harb
❌ Host of Archaic Representations of Books (now merged in http://github.com/sheetjs/js-xlsx )
Stars: ✭ 55 (+266.67%)
Mutual labels:  spreadsheet
forum live-tangshan
django后台,移动app,功能有登录,支付、充值,礼物贴现申请,直播,论坛,好友,地区选择功能。
Stars: ✭ 38 (+153.33%)
Mutual labels:  live
keikai-tutorial
A tutorial of a web spreadsheet component, keikai
Stars: ✭ 19 (+26.67%)
Mutual labels:  spreadsheet
youtube-trending-videos-scraper
A scraper for videos that are trending on YouTube (https://www.youtube.com/feed/trending)
Stars: ✭ 21 (+40%)
Mutual labels:  scraper

Stream List Updater

A script to update the status of live streams listed in the 2020 George Floyd Protest Tracker by twitch.tv/woke.

These tools use HTML scraping and hacks to get rapid information out of streaming websites with heterogeneous APIs. The methods employed will become stale over time.

Installation / Setup

  1. Run npm install
  2. Create a service account and grant it edit access to the sheet.
  3. Save the service account JSON key as "creds.json" in this directory.
  4. Get a YouTube API Key if you'd like to monitor YouTube live streams.

Scripts

Update stream spreadsheet

This script checks links in the spreadsheet to determine if streams are live or offline. It works for the following services:

  • YouTube (requires API key)
  • Facebook Live
  • Twitch.tv
  • Periscope
  • Instagram

A browser window will open and automatically load up stream URLs. You will need to fill in CAPTCHAs occasionally.

UPDATE_SECONDS=<age in seconds to consider check stale> CHECK_INTERVAL=<seconds between individual site checks> SLEEP_SECONDS=30 SHEETS='<sheets id from url>,<tab name 1>,<tab name 2>' PREV_STREAMS_SHEET_ID=<sheets id from url> PREV_STREAMS_TAB_NAME=<tabname> STREAM_EXPIRE_SECONDS=14400 YT_API_KEY=... IG_USER=... IG_PASS=... npm start

Collect URLs from Twitch chat

SHEET_ID=<sheets id from url> TAB_NAME=... npm run twitch-urls

Automatically publish moderated streams and announce to Discord

To set up Twitter bot:

  1. Create a new Twitter app key, setting callback URL to http://localhost:3000/callback.
  2. Authenticate with Twitter:
    1. Run node ./scripts/setupTwitter.js CONSUMER_KEY CONSUMER_SECRET > ../twitter-creds.json and open http://localhost:3000 in a web browser.
    2. Click the link and authorize your Twitter app.
FROM_SHEETS='<sheets id from url>,<tab name 1>,<tab name 2>' TO_SHEET_ID=<sheets id from url> TO_TAB_NAME=... FLAGGED_SHEET_ID=<sheets id from ulr> FLAGGED_TAB_NAME=... PUBLISHED_COL_NAME=... VETTED_COL_NAME=... ANNOUNCE_WEBHOOK_URL=<webhook url from discord> ANNOUNCE_DETAILS_WEBHOOK_URL=<webhook url from discord> SLEEP_SECONDS=30 npm run link-publisher
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].