1382 Open source projects that are alternatives of or similar to Scrapy Azuresearch Crawler Samples

A search application to explore, discover and share online files

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

crawler framework, distributed crawler extractor

Open-source Enterprise Grade Search Engine Software

A Facebook crawler

爬虫, http代理, 模拟登陆!

scrapy专利爬虫（停止维护）

Official Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler

swiss army knife for hackers

码云仓库链接:AJay13/ECommerceCrawlers Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers 项目展示平台链接:http://wechat.doonsec.com

Simple but useful Python web scraping tutorial code.

Dynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具

Github 仓库及用户分析爬虫

Crawlera middleware for Scrapy

Scrapy-based Crawlers for news of Taiwan

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

An Open Source Search Engine

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

PTT 網路版爬蟲

基于搜狗微信搜索的微信公众号爬虫接口

Fess is very powerful and easily deployable Enterprise Search Server.

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

HTTP API for Scrapy spiders

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

根据关键词抓取微博数据，再生成词云

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

A search engine that doesn't track you.

Source code for the IndieWeb search engine.

Redis-based components for Scrapy.

基于Scrapy的QQ音乐爬虫(QQ Music Spider)，爬取歌曲信息、歌词、精彩评论等，并且分享了QQ音乐中排名前6400名的内地和港台歌手的49万+的音乐语料

Some scrapy and web.py exmaples

Parser and database to index the terpene profile of different strains of Cannabis from online databases

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

A framework for creating semi-automatic web content extractors

A multi-thread crawler framework with many builtin image crawlers provided.

Life is short, *****.

Scraping scripts for various websites.

Simple CORPORA list crawler

Azure Logic Apps extension for VS Code

This project contains instructions to deploy the Global Azure Bootcamp 2019 Science Lab

Open-source productivity booster with a brain

Distributed Image Search Engine Crawler

Offensive Terraform Website

Discover what visitors are searching for on your WordPress site.

Moqui Tool Component for ElasticSearch useful for scalable faceted text search, and analytics and reporting using aggregations and other great features

基于appium的app自动遍历工具

🔍 A set of UI components to build a fully customized search!

Open Sketch Search Engine－ 3D object retrieval based on sketch image as input

AI-powered search engine

This is a end to end Personalisation business scenario

A Tumblr Blog Backup Application

Preview Unicode characters and emoji in Alfred

Better Search WordPress plugin

fast easy interval overlapping for nim-lang

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.