804 Open source projects that are alternatives of or similar to Webspider

Lets Map Your Network enables you to visualise your physical network in form of graph with zero manual error

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

📰 A responsive interface of Hacker News with summaries and thumbnails.

QGIS embbeded WMS/WFS/WCS asynchronous scalable http server

Gathers scalable tensorflow and infrastructure deployment

🐛一只登录正方教务管理系统，爬取数据的小爬虫

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

今日头条科技新闻接口爬虫

Bittorrent dht network spider

🍰 一个可视化的爬虫平台

爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载

天气爬虫（全国城镇天气自动定时抓取更新，并开放RESTful查询接口），附带代理IP池定时更新并检测其可用性

Django Celery DB Backend that keeps enough info to retry a task.

Convenient utilities for adding OpenTracing support in your python projects

🚀 A Simple API Gateway for Building Security and Flexible Microservices.

RESTful PACS server with plugins

🕷 A lightning fast multithreaded network scanner framework with modules.

各大网站逆向demo。企名片、震坤行工业超市、天翼云登录、物超所值、瓜子二手车、马蜂窝、中华诗词库、澳门彩票、药智网、福建省招标投标在线监管平台、全国公共资源交易平台、问卷星、中国人民银行条法司、中华人民共和国公安部、AqiStudy、巨量星图、HeyTap、掌上高考、船讯网、百度指数、今日头条、知乎、七麦数据、途牛、七猫小说、企查查、同花顺、网易云音乐、拉勾招聘、玩物得志、房天下

磁力網站U3C3介紹以及域名更新

HTTP-based DSL for for validating RESTful APIs

html5 斗地主游戏

High performance crawler for Nodejs

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

A simple and flexible web crawler framework for java.

Türkçe olarak hazırlanmış Python programlama dili ile ilgili içeriklerin derlendiği sayfa.

📺 Youtube字幕下载脚本

一款信息泄漏利用工具，适用于.git/.svn源代码泄漏和.DS_Store泄漏

知乎模拟登录，支持提取验证码和保存 Cookies

Workshop on Template Injection (6 exercises) covering Twig, Jinja2, Tornado, Velocity and Freemaker engines.

A controllable timing task widgets with Celery

新浪微博，微信，知乎，头条爬虫，支持新浪登录打码获取cookie实现登录

DHT磁力链接magnet BT搜索引擎，纯Python开发

Gets info from github and transfers into json styled data

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

🎯Python 3 网络爬虫实战、数据分析合集 | 当当 | 网易云音乐 | unsplash | 必胜客 | 猫眼 |

📊 Request statistics middleware that stores response times, status code counts, etc

📂 Source Code for (some of) the Programming Tutorials from my Italian YouTube Channel and website ProgrammareInPython.it. This is just a small portion of the content: please visit the website for more.

🎊 Design and implement of lightweight crawler framework.

No description or website provided.

A fast tool to fetch URLs from HTML attributes by crawl-in.

HTTP requests library for WASM Apps

Web Scan Lazy Tools - Python Package

An AliExpress spider for Node

Full stack, modern web application generator. Using Flask, CouchDB as database, Docker, Swagger, automatic HTTPS and more.

Extracting and parsing structured data with jQuery Selector, XPath or JsonPath from common web format like HTML, XML and JSON.

A django admin site for scrapy

💂 Wrap NodeJS request module to retry http requests in case of errors

BeGoneAds is a script that puts some popular hosts file lists into the systems hosts file as a adblocker measure.

Clearly see and debug your celery cluster in real time!

图书馆书蜗App自动化脚本（抢坐 & 续借）

downloads photos and videos from twitter

Web-crawler-engineer-for-Python

Sticker is a powerful yet boilerplate-free alternative to writing your web API.

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

📷 web scrapping in python: multiple libraries -requests, beautifulsoup, mechanize, selenium

一个灵活、友好的爬虫框架

中国银行外汇牌价爬虫 / API (Bank of China - Foreign Exchange - Spider/ API)

Patch pymysql to support tornado asynchronous

Request ID propagation for ASGI apps

Korbit API wrapper for Python