682 Open source projects that are alternatives of or similar to Webvideobot

🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi

爬取amazon/bestbuy/costco/6pm 的商品详情

🐝 Web vertical crawler framework for fun

Pholcus is a distributed high-concurrency crawler software written in pure golang

tumblr解析网站

Scrapy-based Crawlers for news of Taiwan

Implements the crawler package into Symfony

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

massive SQL injection vulnerability scanner

Bilibili.com视频批量下载工具包

An online tool (crawler) to analyze users performance in online judges (coding competition websites). Supported OJ: POJ, HDU, ZOJ, HYSBZ, CodeForces, UVA, ICPC Live Archive, FZU, SPOJ, Timus (URAL), LeetCode_CN, CSU, LibreOJ, 洛谷, 牛客OJ, Lutece (UESTC), AtCoder, AIZU, CodeChef, El Judge, BNUOJ, Codewars, UOJ, NBUT, 51Nod, DMOJ, VJudge

An Instagram bot developed using the Selenium Framework

Verify that a request is from Google crawlers using Google's DNS verification steps

A Tumblr Blog Backup Application

Dynamic meta tags in your AngularJS single page application

Elasticsearch File System Crawler (FS Crawler)

又一个 java 内容（pa）获取（chong）工具

百度云网盘搜索引擎，包含爬虫 & 网站

My Python Script

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

An Open Source Search Engine

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Seeker - another job board aggregator.

My Python Demo

It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.

越来越多的网站具有反爬虫特性，有的用图片隐藏关键数据，有的使用反人类的验证码，建立反反爬虫的代码仓库，通过与不同特性的网站做斗争（无恶意）提高技术。（欢迎提交难以采集的网站）（因工作原因，项目暂停）

🕷招聘网站爬虫合集，不定期更新分支

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!

POOPAK - TOR Hidden Service Crawler

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API

대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.

Python: An all-in-one Web Crawler, Web Parser and Web Scrapping library!

Cross-platform persistent and distributed web crawler 🔗

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。 [AD]中文知识图谱门户

哔哩哔哩的API调用模块

Fawkes is a tool to search for targets vulnerable to SQL Injection. Performs the search using Google search engine.

基金投资管理回测引擎

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Headless Chrome For Java （Java 爬虫）

It covers the blockade principle of most anti-climbing strategies and corresponding solutions.👽👽👽👽（涵盖了大部分的反爬策略的封锁原理以及对应的解决方案。）

一个获取知乎用户主页信息的多线程Python爬虫程序。

capture pictures from website like sina, lofter, huaban and so on

A scalable web crawler framework for Java.

HTTP API for Scrapy spiders

一些爬虫的代码

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

爬虫, http代理, 模拟登陆!

A cross platform UI crawler which scans view trees then generate and execute UI test cases.

This repository is no longer maintained.

Project thu thập điểm chuẩn đại học 2014 - 2018 và phân tích dữ liệu

Golang pkg to quickly return a preview of a webpage (title/description/images)

Crawler for LinkedIn full profiles 2019

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

Python爬虫，京东自动登录，在线抢购商品

Some scrapy and web.py exmaples

爬虫所需要的IP代理，抓取九个网站的代理IP检测/清洗/入库/更新，添加调用接口

Download images from Google, Bing, Baidu. 谷歌、百度、必应图片下载.