GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → crackcomm → crawl

crackcomm / crawl

Licence: Apache-2.0 license
Lightweight library for scalable crawlers in Go.

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects

Labels

crawlercrawl

Projects that are alternatives of or similar to crawl

Infospider
INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。
Stars: ✭ 5,984 (+29820%)
Mutual labels:  crawl
Andvaranaut
A dungeon crawler
Stars: ✭ 103 (+415%)
Mutual labels:  crawl
Crack Js Spider
破解JS反爬虫加密参数，已破解中国裁判文书网（2020-06-30更新），淘宝密码，天安保险登录，b站登录，房天下登录，WPS登录，微博登录，有道翻译，网易登录，微信公众号登录，空中网登录，今目标登录，学生信息管理系统登录，共赢金融登录，重庆科技资源共享平台登录，网易云音乐下载，一键解析视频链接，财联社登录。
Stars: ✭ 175 (+775%)
Mutual labels:  crawl
Recruit
这个项目的目的是整合招聘信息，并做一定处理。
Stars: ✭ 13 (-35%)
Mutual labels:  crawl
Weixin
微信小游戏辅助合集（加减大师、包你懂我、大家来找茬腾讯版、头脑王者、好友画我、悦动音符、我最在行、星途WeGoing、猜画小歌、知乎答题王、腾讯中国象棋、跳一跳、题多多黄金版）
Stars: ✭ 1,216 (+5980%)
Mutual labels:  crawl
Dl coursera
A simple, fast, and reliable Coursera crawling & downloading tool
Stars: ✭ 115 (+475%)
Mutual labels:  crawl
Fbcrawl
A Facebook crawler
Stars: ✭ 536 (+2580%)
Mutual labels:  crawl
crawler-chrome-extensions
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (+165%)
Mutual labels:  crawl
Pspider
一个简单的分布式爬虫框架
Stars: ✭ 102 (+410%)
Mutual labels:  crawl
Proxy pool
Python爬虫代理IP池(proxy pool)
Stars: ✭ 13,964 (+69720%)
Mutual labels:  crawl
Nodespider
[DEPRECATED] Simple, flexible, delightful web crawler/spider package
Stars: ✭ 33 (+65%)
Mutual labels:  crawl
Novel Plus
小说精品屋-plus是一个多端（PC、WAP）阅读、功能完善的原创文学CMS系统，由前台门户系统、作家后台管理系统、平台后台管理系统、爬虫管理系统等多个子系统构成，支持多模版、会员充值、订阅模式、新闻发布和实时统计报表等功能，新书自动入库，老书自动更新。
Stars: ✭ 1,122 (+5510%)
Mutual labels:  crawl
Python3 Spider
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
Stars: ✭ 2,129 (+10545%)
Mutual labels:  crawl
Grab Site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Stars: ✭ 680 (+3300%)
Mutual labels:  crawl
Awesome crawl
腾讯新闻、知乎话题、微博粉丝，Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等
Stars: ✭ 246 (+1130%)
Mutual labels:  crawl
Thea11ymachine
The A11y Machine is an automated accessibility testing tool which crawls and tests pages of any web application to produce detailed reports.
Stars: ✭ 574 (+2770%)
Mutual labels:  crawl
Geetest
滑动验证码，希望对你们有所帮助❤️
Stars: ✭ 114 (+470%)
Mutual labels:  crawl
gathertool
gathertool是golang脚本化开发库，目的是提高对应场景程序开发的效率；轻量级爬虫库，接口测试&压力测试库，DB操作库等。
Stars: ✭ 36 (+80%)
Mutual labels:  crawl
gospider
⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架
Stars: ✭ 183 (+815%)
Mutual labels:  crawl
Bitextor
Bitextor generates translation memories from multilingual websites.
Stars: ✭ 168 (+740%)
Mutual labels:  crawl
View All Similar Projects ➔

crawl

GoDoc Circle CI

Lightweight library for crawlers in Go.

HTML parsing and extracting is done thanks to goquery.

Usage

You can take a look at example crawler code.

License

                             Apache License
                       Version 2.0, January 2004
                    http://www.apache.org/licenses/
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].