2702 Open source projects that are alternatives of or similar to Autoscraper

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

PHP Scraper - an highly opinionated web-interface for PHP

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Crawly, a high-level web crawling & scraping framework for Elixir.

An R web crawler and scraper

This repository contains all the code I use in my YouTube tutorials.

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

🌎 🏨 Scrape Booking.com 🏨 🌎

Be nice on the web

No description or website provided.

Goal-Oriented Action Planning implementation in Python

Distributed crawler powered by Headless Chrome

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

Create agents that monitor and act on your behalf. Your agents are standing by!

Declarative web scraping

Creating Scrapy scrapers via the Django admin interface

Elegant Scraper and Crawler Framework for Golang

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Universal scrapping tool, which allows you to extract data using multiple environments

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

scrapers for building your own image databases

[Unmaintained] A simple and clean video/music/image downloader 👾

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

A Devtools driver for web automation and scraping

A framework for creating semi-automatic web content extractors

Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

Web Scraping Framework

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

No description or website provided.

🌓 Bringing pjreddie's DarkNet out of the shadows #yolo

A Dragnet that also extract author, headline, date, keywords from context

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

No description or website provided.

Command line program to download documents from web portals.

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

Command Line Artificial Intelligence or CLAI is an open-sourced project from IBM Research aimed to bring the power of AI to the command line interface.

A PHP Scraping Class

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

News extraction and scraping. Article Parsing

PHP Metacritic API - Mirrored by my GitLab

Scape top GitHub repositories and users based on keywords

🕸 List of mini projects that involve web scraping 🕸

bingip2hosts is a Bing.com web scraper that discovers websites by IP address

API for fetching data from news websites.

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

🚆This web scraper builds a dataset for São Paulo subway operation status

High performance crawler for Nodejs

Deep Learning Computer Vision Algorithms for Real-World Use

A Scraper, Downloader, & Recorder for static open directories.

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

Scraping the status of WhatsApp contacts