716 Open source projects that are alternatives of or similar to newspaperjs

API for fetching data from news websites.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Let's you to access your FB account from the command line and returns various things number of unread notifications, messages or friend requests you have.

Web Scraping Framework

🌎 🏨 Scrape Booking.com 🏨 🌎

Twitter Troll & Fake News Hunter - Crawls news websites and twitter to identify fake news

A Dragnet that also extract author, headline, date, keywords from context

Create agents that monitor and act on your behalf. Your agents are standing by!

Be nice on the web

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

This repository contains all the code I use in my YouTube tutorials.

Implementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021

News aggregator for the press releases of the Bulgarian government sites written in ASP.NET Core

bingip2hosts is a Bing.com web scraper that discovers websites by IP address

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

PHP Metacritic API - Mirrored by my GitLab

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Scrape everything you can from MyAnimeList.net

This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.

Scrape Instagram content and stories anonymously, using a new technique based on the har file (No Token + No public API).

An R web crawler and scraper

Creating Scrapy scrapers via the Django admin interface

A python web scraper for public email lists.

Google News Scraper for languages like Japanese, Chinese... [VPN Support]

robots.txt file parsing and checking for R

Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

Tools for downloading agendas, minutes and other documents produced by local government

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

Command line program to download documents from web portals.

🔎 A simple yet powerful module to retrieve organic search results and much more from Google.

Code relating to scraping public police data.

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

A simple web scraper built around the JavaFX WebEngine

Unofficial API of Inshorts written in Flask

A Feed Aggregator that Knows What You Want to Read.

Search and download mangas from the command line

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

A simple web scraping engine supporting concurrent and anonymous scraping

百度贴吧分布式爬虫，用于贴吧数据挖掘。从贴吧维度和用户维度进行数据分析

Create and **automatically** update a list of all videos on a YouTube channel (in txt/csv/md form) via YouTube bot with end-to-end web scraping - no API tokens required. Multi-threaded support for YouTube videos list updates.

Analyze the margin of error in U.S. census data

A monadic web scraping library

A simple Python wrapper for the archive.is capturing service

Scrapes VK user's photos

A free TVTime API based on scraping TVTime website. No API key required

Blog do Brasil.IO

A web scraper starter project

For viewing a daily issue of the Guardian and Observer newspapers. `main` branch should be stable, current work is in `dev` branch.

Transcribe news audio in realtime

An app that fetches latest news, headlines

Daily News For You is an online news portal developed by Django and SQLite

Python module(s) to get stock data, options data and news.

📚 Don't waste time searching for good dev blog posts. Get the latest news here.

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

🔍 Chráňte vašich blízkych pred nedôveryhodným 🇸🇰 a 🇨🇿 obsahom

Access parts of your account unavailable through normal YouTube API access.

A Go scraper that validates security.txt files and outputs them in the disclose.io JSON format.

An Introduction to Web Scraping

An easy-to-use python client for Google News feeds.