682 Open source projects that are alternatives of or similar to Webvideobot

爬虫代理IP池服务，可供其他爬虫程序通过restapi获取

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

Simple, clear and fast Web Crawler framework build on python3.6+, powered by asyncio.

一个功能全面的微信读书爬虫 wereader

爬取CSDN上的博客文章

📖 爬取 Stackoverflow 100万 条问答并简单分析

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

简简单单spider

Some toolkits implements part of BT Protocol, like DHT spider.

yspider -- 轻量级爬虫系统

Last-Statement-of-Death-Row, 人之将死，其言也善

Bitextor generates translation memories from multilingual websites.

This package is a complete tool for creating a large dataset of images (specially designed -but not only- for machine learning enthusiasts). It can crawl the web, download images, rename / resize / covert the images and merge folders..

GUI based offensive penetration testing tool (Open Source)

网易云爬虫解决方案

Get the lyrics for the song currently playing on Spotify

[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+jwt+json日志库zap(支持日志收集到kafka或mongo)+消息队列kafka+微信支付宝支付gopay+api加密+api反向代理+go modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+livereload热加载)

知乎爬虫程序，定时跟踪问题数据，定时推送热门话题

🌟⏳🌟 各种网站的签到（停止维护）

GUI of AlipayOrdersSupervisor, implemented in Java and Swing

A Strong, Fast and Flexible Pixiv Client based on .NET Core and WPF

Download Videos Skill Share per ID or per Class

Node.js seed downloader (Node.js 种子神器)

owllook-小说搜索引擎

To crawl all csgo skins from website.

新浪微博相册大图多线程爬虫。

Alternative for VK Audio API

tumblr解析网站

Scrapy-based Crawlers for news of Taiwan

爬取微信公众号文章

An online tool (crawler) to analyze users performance in online judges (coding competition websites). Supported OJ: POJ, HDU, ZOJ, HYSBZ, CodeForces, UVA, ICPC Live Archive, FZU, SPOJ, Timus (URAL), LeetCode_CN, CSU, LibreOJ, 洛谷, 牛客OJ, Lutece (UESTC), AtCoder, AIZU, CodeChef, El Judge, BNUOJ, Codewars, UOJ, NBUT, 51Nod, DMOJ, VJudge

An Instagram bot developed using the Selenium Framework

Verify that a request is from Google crawlers using Google's DNS verification steps

🔍 简单的搜索引擎, django 框架

A php crawler that finds emails on the internets

Find web directories without bruteforce

Xenomorph Crawler, a Concise, Declarative and Observable Distributed Crawler(Node / Go / Java / Rust) For Web, RDB, OS, also can act as a Monitor(with Prometheus) or ETL for Infrastructure 💫 多语言执行器，分布式爬虫

基于python语言开发的千万级别搜索引擎

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

A Moodle downloader that downloads course content fast from Moodle (eg. lecture pdfs)

An Open Source Search Engine

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

My Python Demo

It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.

抖音爬虫. 通过手机代理爬取用户的作品和用户的喜欢

用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站

用严肃的数据来回答“什么样的企业会到什么样的大学招聘”？

Sample of using proxies to crawl baidu search results.

A configurable web spider with a easy-to-use web console

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

POOPAK - TOR Hidden Service Crawler

대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.

Python: An all-in-one Web Crawler, Web Parser and Web Scrapping library!

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Headless Chrome For Java （Java 爬虫）

It covers the blockade principle of most anti-climbing strategies and corresponding solutions.👽👽👽👽（涵盖了大部分的反爬策略的封锁原理以及对应的解决方案。）

一个获取知乎用户主页信息的多线程Python爬虫程序。

capture pictures from website like sina, lofter, huaban and so on