1184 Open source projects that are alternatives of or similar to Go_spider

Scrapy-based Crawlers for news of Taiwan

A python-based isomiR quantification and analysis pipeline

备份豆瓣计划

获取最新可备案域名列表爬虫

An online tool (crawler) to analyze users performance in online judges (coding competition websites). Supported OJ: POJ, HDU, ZOJ, HYSBZ, CodeForces, UVA, ICPC Live Archive, FZU, SPOJ, Timus (URAL), LeetCode_CN, CSU, LibreOJ, 洛谷, 牛客OJ, Lutece (UESTC), AtCoder, AIZU, CodeChef, El Judge, BNUOJ, Codewars, UOJ, NBUT, 51Nod, DMOJ, VJudge

Pholcus is a distributed high-concurrency crawler software written in pure golang

Turn Java 8 Streams into Unix like pipelines

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

When you solve the problem of Baekjoon Online Judge, it automatically commits and pushes to the remote repository.

一个简单的豆瓣信息爬虫😄

Verify that a request is from Google crawlers using Google's DNS verification steps

[ Closed ] 🎨 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.

一个获取知乎用户主页信息的多线程Python爬虫程序。

Catalyst.Segmentation

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

A golang spider

又一个 java 内容（pa）获取（chong）工具

A simple Spark-powered ETL framework that just works 🍺

A background job scheduler for any queue backend

Let's get the ball rolling on some Container-driven CI & CD

A simple cron-like scheduler for a single command

It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.

Testing Using Python Demo. 使用Python测试脚本demo。

A python web scraper for public email lists.

POOPAK - TOR Hidden Service Crawler

The Resources for "Natural Language to Logical Form" ; "自然语言转逻辑形式"研究资料收集。

A document viewer; fuzzy match incremental search.

The Last Web Recon Tool You'll Need

Python: An all-in-one Web Crawler, Web Parser and Web Scrapping library!

🕷招聘网站爬虫合集，不定期更新分支

Seeker - another job board aggregator.

Jenkins-Plugin to create a multi-branch-project from gitlab

Crawls a website, gets PageSpeed Insights data for each page, and exports an HTML report.

Download Videos Skill Share per ID or per Class

越来越多的网站具有反爬虫特性，有的用图片隐藏关键数据，有的使用反人类的验证码，建立反反爬虫的代码仓库，通过与不同特性的网站做斗争（无恶意）提高技术。（欢迎提交难以采集的网站）（因工作原因，项目暂停）

MLBox is a powerful Automated Machine Learning python library.

Tired of pushing to test your .gitlab-ci.yml?

As you can see, a kuaishou crawler

A SingleCell RNASeq pre-processing snakemake workflow

整合第三方网站的视频和弹幕资源, 为白嫖党提供最佳看番追剧体验

Your web font utility belt. It can subset web fonts. It can find unicode-ranges for you automatically. It makes julienne fries.

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

Manage events on a Google Calendar

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Project thu thập điểm chuẩn đại học 2014 - 2018 và phân tích dữ liệu

💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!

Python爬虫，京东自动登录，在线抢购商品

A cloud-native Pipeline resource.

A utility package for automating lighthouse reporting

A dungeon crawler

A powerful dynamic crawler for web vulnerability scanners

Lightweight, Python library for fast and reproducible experimentation 🔬

一个简单的分布式爬虫框架

Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.

Example workflows for the drake R package

Mutlitenancy for dotnet applications

Making whole bacterial genome sequencing data analysis easy

feapder是一款支持分布式、批次采集、任务防丢、报警丰富的python爬虫框架