738 Open source projects that are alternatives of or similar to Nosmoke

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Next-gen browser and mobile automation test framework for Node.js

A lite distributed Java spider framework :-)

Gospider - Fast web spider written in Go

Java 網路資料爬蟲包

A tool for pixiv.net. 人人可用的P站爬虫

News, full-text, and article metadata extraction in Python 3. Advanced docs:

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

抖音 SDK，数据采集，爬虫抓取不是梦

Green is a clean, colorful, fast python test runner.

疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道

基金投资管理回测引擎

Cucumber JVM with Selenium Java

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Declarative HTTP Testing for Python and anything else

Selenium Python Bindings Documentation

Amazon商品引流的 python 爬虫

京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

A high-level API for programmatically interacting with web pages through WebDriver.

JSON based visual regression testing library. Liberate creativity, minimize repeated works ✌️

Pure Clojure Webdriver protocol implementation

All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers

一个超级轻量的百度图片爬虫

Crawl a website and run it through Google lighthouse

Simple but useful Python web scraping tutorial code.

Fuzzinator Random Testing Framework

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

A search application to explore, discover and share online files

MM131网站图片爬取 🚨

Fess is very powerful and easily deployable Enterprise Search Server.

利用文本挖掘技术进行新闻热点关注问题分析

基于搜狗微信搜索的微信公众号爬虫接口

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API

Carina automation framework: Web, Mobile, API, DB

爬虫代理IP池服务，可供其他爬虫程序通过restapi获取

Remote control interface that enables introspection and control of user agents.

AutoLink是一个开源Web IDE自动化测试集成解决方案

Redis-based components for Scrapy.

🤖 Repeat tests. Repeat tests. Repeat tests.

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

🕷️ A node crawler for github trending.

带你了解一下Golang的市场行情

新浪微博相册大图多线程爬虫。

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

As you can see, a kuaishou crawler

WebDriver Screenshot utility. Take screenshots, crop, prettify, compare

Automatically detect and generate missing assertions for Junit test cases (also known as test amplification)

CoCrawler is a versatile web crawler built using modern tools and concurrency.

GraphQuery is a query language and execution engine tied to any backend service.

An updated EdgeDriver implementation for Selenium 3 with newly-added support for Microsoft Edge (Chromium).

Discover hidden deepweb pages

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

Rust Web Crawler saving pages on Redis

Quickly spin up a temporary PostgreSQL test databases

Automated end-to-end tests for WordPress.com

🔍 简单的搜索引擎, django 框架

A python3 crawler for crawling Pixiv ranking top and any illustrator all artworks