693 Open source projects that are alternatives of or similar to Phpscraper

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A framework for creating semi-automatic web content extractors

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Scape top GitHub repositories and users based on keywords

PHP Library for detecting CMS

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

Zillow Scraper for Python using Selenium

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

Scraping the status of WhatsApp contacts

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Command line program to download documents from web portals.

MetaData html scraper and parser for Node.js (supports Promises and callback style)

A console application that can scrape the Azur Lane wiki and export the data to Json files

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.

No description or website provided.

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

A PHP Scraping Class

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

A Python Tool For google Hacking

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

A Devtools driver for web automation and scraping

A collection of awesome web crawler,spider in different languages

✂️ High performance, multi-threaded image scraper

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

No description or website provided.

🕸 List of mini projects that involve web scraping 🕸

Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

🚆This web scraper builds a dataset for São Paulo subway operation status

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

A Scraper, Downloader, & Recorder for static open directories.

A collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

NBA Stats API via Basketball Reference

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Faster requests on Python 3

Search for and retrieve US Patent and Trademark Office Patent Data

A Router WiFi key recovery/cracking tool with a twist.

Declarative web scraping

Distributed crawler powered by Headless Chrome

Extract structured data from web sites. Web sites scraping.

[Unmaintained] A simple and clean video/music/image downloader 👾

Powerful web scraping framework for Crystal

Creating Scrapy scrapers via the Django admin interface

Crawly, a high-level web crawling & scraping framework for Elixir.

Go cascadia package command line CSS selector

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Fetch user's data across social media

ProxyCrawl Python library for scraping and crawling

Python Package for scraping NHL Play-by-Play and Shift data

A list of scrapers from around the web.

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges