1802 Open source projects that are alternatives of or similar to Serpscrap

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

PHP Scraper - an highly opinionated web-interface for PHP

Maintained collection of OSINT related resources. (All Free & Actionable)

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

🕸 List of mini projects that involve web scraping 🕸

A Python Tool For google Hacking

✂️ High performance, multi-threaded image scraper

Pahe.ph [Pahe.in] Movies Website Scraper

Scrape job websites into a single spreadsheet with no duplicates.

Crawling and scraping the Web for fun and profit

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

**[ARCHIVED]** website changes tracker 🔍

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

📷 Automate full website screenshots and PDF generation with multiple viewport support.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

ProxyCrawl Python library for scraping and crawling

Linux命令大全搜索工具，内容包含Linux命令手册、详解、学习、搜集。https://git.io/linux

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Distributed crawler powered by Headless Chrome

Crawly, a high-level web crawling & scraping framework for Elixir.

Social media research and promotion, semi-autonomous CLI bot

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

This class can retrieve search results from Google.

🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.

Lightweight package to query popular search engines and scrape for result titles, links and descriptions

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

scrapers for building your own image databases

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

reborn of https://bitbucket.org/rflechner/scrapysharp

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

No description or website provided.

Command line program to download documents from web portals.

A PHP Scraping Class

Scraping the status of WhatsApp contacts

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

A Scraper, Downloader, & Recorder for static open directories.

Universal scrapping tool, which allows you to extract data using multiple environments

Scrape a Facebook profile and turn it into a JSON file

Aiohttp web server API, which scrapes Google and returns scrape results as response. Supports proxies, multiple geos and number of results.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.

Declarative web scraping

Extract structured data from web sites. Web sites scraping.

[Unmaintained] A simple and clean video/music/image downloader 👾

No description or website provided.

Creating Scrapy scrapers via the Django admin interface

Google Search SERP Scraper

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Search for and retrieve US Patent and Trademark Office Patent Data

simple multi-level scraper json input/output for Cheerio

Elegant Scraper and Crawler Framework for Golang

A client library for accessing the USPTO Open Data APIs, written in Python.

An unofficial DuckDuckGo search API.

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!