973 Open source projects that are alternatives of or similar to trafilatura

A Dragnet that also extract author, headline, date, keywords from context

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Build a Flask web application to help users retrieve key restaurant information and feature-based reviews (generated by applying market-basket model – Apriori algorithm and NLP on user reviews).

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

Web Scraping, Document Deduplication & GPT-2 Fine-tuning with a newly created scam dataset.

Explaining textual analysis tools in Python. Including Preprocessing, Skip Gram (word2vec), and Topic Modelling.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

News, full-text, and article metadata extraction in Python 3. Advanced docs:

News aggregator for the press releases of the Bulgarian government sites written in ASP.NET Core

Text corpus for Bahasa Malaysia, https://malaya.readthedocs.io/en/latest/Dataset.html

Google News Scraper for languages like Japanese, Chinese... [VPN Support]

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

📈 Regression and Classification with UC Davis student quiz data and exam data

News extraction and scraping. Article Parsing

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

Scrape any website, article or RSS/Atom Feed with ease!

A framework for creating semi-automatic web content extractors

A Happy and lightweight Python Package that Provides an API to search for articles on Google News and returns a JSON response.

New York Times Word Innovation Types dataset

🤔一个新闻网页正文通用抽取器，包括标题、作者和日期。

🦛 scrapes websites and generates rss feeds

A curated list of NLP resources for Hungarian

This repository contains code and bonus content which will be added from time to time for the book "Learning Social Media Analytics with R" by Packt

Text preprocessing, representation and visualization from zero to hero.

🤖 A Versatile Telegram Bot

PHP Scraper - an highly opinionated web-interface for PHP

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Fast readability scores for text data

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

KH Coder: for Quantitative Content Analysis or Text Mining

SmartReader is a library to extract the main content of a web page, based on a port of the Readability library by Mozilla

Web Scraping Framework

Extract clean(er), readable text from web pages via Mercury Web Parser.

Scape top GitHub repositories and users based on keywords

Simple python script to parse twitter feed to generate a rss feed.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

PHP Library for detecting CMS

Text Mining for Indonesian Online News Articles About Corona

Reworked https://www.readability.com/ parsing library (now https://mercury.postlight.com/ is living alternative)

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

A Los Angeles Times analysis of California's costly power glut

Official releases of the PROIEL treebank of ancient Indo-European languages

The aim of this repository is to show a baseline model for text classification by implementing a LSTM-based model coded in PyTorch. In order to provide a better understanding of the model, it will be used a Tweets dataset provided by Kaggle.

James Joyce's novel Ulysses in TEI XML. Work-in-progress.

Codes for text-mined solid-state reactions dataset

A keyphrase extractor for Persian

**[ARCHIVED]** website changes tracker 🔍

Crawlzone is a fast asynchronous internet crawling framework for PHP.

Pillaging the seven seas for torrents, pieces of eight and other bounty.

📊 Python tool to scrape real-time information about ETFs from the web and mixing them together by proportionally distributing their assets allocation

News App developed with Flutter featuring beautiful UI, category-based news, story for faster news reading, inbuilt article viewer, share feature, and more.

Use Puppeteer to download pens from Codepen.io as single html pages

Exercises, data, and more for our 2017 summer workshop (funded by the Estes Fund and in partnership with Project Jupyter and Berkeley's D-Lab)

Tool to scrape linkedin

[INACTIVE] News on GitHub, featured by Huei Tan 😄

Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules