826 Open source projects that are alternatives of or similar to Scrapedin

Crawler for Nintendo Switch eShop

Elegant Scraper and Crawler Framework for Golang

LinkedinScraper is an another information gathering tool written in python. You can scrape employees of companies on Linkedin.com and then create these employee names, titles and emails.

High performance crawler for Nodejs

Crawly, a high-level web crawling & scraping framework for Elixir.

Simple yet powerful automation stuffs.

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

Ruby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.

台灣股票即時爬蟲。Taiwan Stock Exchange Real Time Crawler

一个灵活、友好的爬虫框架

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

A Python Tool For google Hacking

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

Crawlera middleware for Scrapy

爬取菜鸟教程网站并转PDF__python_crawer_by_chrome

👩 美女写真套图爬虫（一）

A non API python program to crawl public photos, posts or followers

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

📰 A responsive interface of Hacker News with summaries and thumbnails.

An exporter from a LinkedIn profile to JSON Résumé

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Readability / Html Content / Article Extractor & Web Scrapping library written in PHP

Crawler (Bot) searching for credential leaks on different paste sites.

一个基于webmagic框架二次开发的java爬虫框架实战，已实现能爬取腾讯，搜狐，今日头条（单独集成功能）等资讯内容，配合elasticsearch框架用法，实现了自动爬虫，已投入线上生产使用。

swiss army knife for hackers

Easily create XML sitemaps for your website.

Web Application Security Scanner Framework

Html网页正文提取

本项目不再维护，详情可加群了解 https://t.me/onecomicbook

📊 Financial markets data library implemented in go.

Download videos from pornhub.

The Metadata Platform for the Modern Data Stack

An OSINT tool that allows you to draw out relationships between people on LinkedIn via endorsements/skills.

line-bot-tutorial use python flask

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

今日头条机器人，支持用户登陆、关注、取消关注、获取关注粉丝、发文、发悟空问答、点赞、评论、采集各种类型新闻讯息等，使用今日头条网页版API实现

磁力網站U3C3介紹以及域名更新

This is a Multi-thread crawler for Tumblr.

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

The simple, easy to use command line web crawler.

Script that crawls meta data from ICLR OpenReview webpage. Tutorials on installing and using Selenium and ChromeDriver on Ubuntu.

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Scrape a Facebook profile and turn it into a JSON file

A fast tool to fetch URLs from HTML attributes by crawl-in.

知乎模拟登录，支持提取验证码和保存 Cookies

This is Social login library in which you can login through Facebook , LinkedIn and Google

A Simple Proxy Scraper

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

JavGo是一个集合影片管理，影片刮削，视频处理，资源搜索等综合一体的全功能影音软件，支持爬取javbus，jav321，javdb，javlibrary进行刮削，支持db，bus的磁力搜索，支持获取library的影片评论。

Recursive and multi-threaded broken link checker

模拟登录各类网站，操作 API 完成各种不可描述的事情

Stream anime from kaa.si and sync with anilist

A social networking service scraper in Python

What do people have in their dotfiles?

Information Gathering Instagram.

🔗 OAuth 2.0 implementation for various providers in one place.

Extracting data of Nepal Stock Exchange Ltd. (NEPSE)

🌭💦 91porn爬虫在线API接口（永久有效） 及 在线web预览