769 Open source projects that are alternatives of or similar to Appcrawler

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

GraphQuery is a query language and execution engine tied to any backend service.

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

Android 本地网络小说爬虫，基于jsoup及xpath

SirixDB is a temporal, evolutionary database system, which uses an accumulate only approach. It keeps the full history of each resource. Every commit stores a space-efficient snapshot through structural sharing. It is log-structured and never overwrites data. SirixDB uses a novel page-level versioning approach called sliding snapshot.

一些爬虫开发的技巧和案例

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

A simple and beautiful text diff viewer component made with Diff and React.

京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取

✌️ Python3 BitTorrent DHT crawler

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

A diff for DOM elements, as client-side JavaScript code. Gets all modifications, insertions and removals between two DOM fragments.

React component for rendering the diff of two React elements

A tool which helps you diff xcodeproj files.

Appium template for android testing training

Diffuse is library that aims to simplify the diffing of two collections

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

http://defiantjs.com

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

💾 A library for backporting UITableView/UICollectionViewDiffableDataSource.

[Unmaintained] A simple and clean video/music/image downloader 👾

一个超级轻量的百度图片爬虫

Simple but useful Python web scraping tutorial code.

Library to diff and merge Java objects with ease

HTML scraping for Objective-C.

基金投资管理回测引擎

Diff Utils library is an OpenSource library for performing the comparison / diff operations between texts or some kind of data: computing diffs, applying patches, generating unified diffs or parsing them, generating diff output for easy future displaying (like side-by-side view) and so on.

Crawls a website, gets PageSpeed Insights data for each page, and exports an HTML report.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

UFO source file diff application

Dark Web OSINT Tool

Opinion mining of Mobile reviews on Amazon platform

A high performance web crawler in Elixir.

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

HTTP API for Scrapy spiders

vim script to diff two directories like BeyondCompare by using `diff`

A multi-thread crawler framework with many builtin image crawlers provided.

Life is short, *****.

🚅 - Hyper fast diffing algorithm for real DOM nodes

A Tumblr Blog Backup Application

🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。

Minimal edits from one collection to another

align and compare tables

Elasticsearch File System Crawler (FS Crawler)

Free Web Scraping Tool with Java

Gospider - Fast web spider written in Go

API of DouYin for Humans used to Crawl Popular Videos and Musics

Merge and diff documents online

A search application to explore, discover and share online files

A tool for pixiv.net. 人人可用的P站爬虫

Automatically track websites changes on Android in background.

Fess is very powerful and easily deployable Enterprise Search Server.

A fast & lightweight XML & HTML parser in Swift with XPath & CSS support

A tool for running android and iOS appium tests in parallel across devices... U like it STAR it !

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

基于搜狗微信搜索的微信公众号爬虫接口