130 Open source projects that are alternatives of or similar to pyCreeper

Use JSON to list all elements (with css 3 and jquery selector) that you want to crawl.

Pre-Render Pages on the Fly in Laravel

Python Web Crawler with Selenium and PhantomJS

media(audio/image/qrcode/markdown/html/svg) support web service (多媒体编辑服务, 酷炫二维码, 音频, 图片, svg, markdown, html渲染服务支持)

Handy API for Headless Chromium

滑动验证码破解示例，仅供学习使用。

Framework for testing ExtJs applications

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)

It is a web crawler which crawls the stackoverfolw website (http://stackoverflow.com/) and finds the most popular technologies at current point of time by getting the tags info of the newest questions asked on the website.

A high-level browser automation library.

今日校园/体温签到/海南大学/QQ机器人

A maven plugin for installing the phantomjs binary on your system automatically.

[MOVED] Grunt task for running mocha specs in a headless browser (PhantomJS)

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

PhantomJS integration module for NodeJS

Universal scrapping tool, which allows you to extract data using multiple environments

Convert HTML to an image, PDF or string

扫描“微信读书”已购图书并下载本地PDF的爬虫

Go client for PhantomJS.

Website style analyzer for designers. This is the service part that does the analyzation etc.

Run QUnit unit tests in a headless PhantomJS instance without using Grunt

Install Selenium-compatible Chrome/Firefox/Opera/PhantomJS/Edge webdrivers automatically.

A Java wrapper around the PhantomJS binaries including a packaged HTML to PDF render script

✅ js微信聊天机器人（使用个人账号，非公众号） Wechat chat robot write by js.

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

Raspagem de dados para iniciante usando Scrapy e outras libs básicas

A set of improvements to Minitest/Capybara/Poltergeist/PhantomJS test stack that reduces the occurrence of transient failures.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Go-like features for Python and Cython. (mirror of https://lab.nexedi.com/kirr/pygolang)

A simple distributed crawler for zhihu && data analysis

Phantomjs for raspberry pi (armv6/armv7) and aarch64/arm64

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

Automation Tools for PHP

A simple tool for fetching usable proxies from several websites.

🕶️ A curated list of resources around the topic: visual regression testing

简单易用的Python爬虫框架，QQ交流群：597510560

Automate Javascript/Ajax websites with Perl and PhantomJS

A simple but powerful web crawler library for .NET

PhantomJS/Node.js web scraper for AWS Lambda

开源矢量地理数据获取与预处理工具(POI/AOI/行政区/路网/土地利用)

pip install funboost，python全功能分布式函数调度框架,。支持python所有类型的并发模式和全球一切知名消息队列中间件，python函数加速器，框架包罗万象，一统编程思维，兼容50% python编程业务场景，适用范围广。只需要一行代码即可分布式执行python一切函数。旧名字是function_scheduling_distributed_framework

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.

👻 Create screenshots or previews of web pages

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

When you solve the problem of Baekjoon Online Judge, it automatically commits and pushes to the remote repository.

快速登录大连理工大学统一身份认证系统（SSO）的Python模块，可轻松实现成绩提醒、抢课、玉兰卡信息、个人信息查询等功能。

Commandline tool for automated downloads of echo360 videos hosted by university

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

（已不適用新版臺鐵訂票系統，且不再更新）台灣鐵路訂票應用程式（臺鐵 / 台鐵 / 訂單程票 / 訂來回票），基於 Selenium + PyQt4。

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!

ACHE is a web crawler for domain-specific search.

Repl.it Discord security bot, ready to srop crime!

Concurrent browser tests with elixir

A tutorial-based introduction to web scraping with Python.

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Distributed task redisqueue(最简单python分布式函数调度框架)

Taking a screenshot of a webpage.

常用javaweb技术，不定期更新，欢迎讨论。