298 Open source projects that are alternatives of or similar to Pdf_downloader

HTTP API for Scrapy spiders

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Scrapy middleware to handle javascript pages using selenium

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A Scrapy extension to log items coverage when the spider shuts down

Simple but useful Python web scraping tutorial code.

Build and deploy a machine learning app from scratch 🚀

swiss army knife for hackers

🎊 Design and implement of lightweight crawler framework.

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

Distributed crawler powered by Headless Chrome

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

Python + Scrapy + MongoDB . 5 million data per day !!!💥 The world's largest website.

Redis-based components for Scrapy.

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Crawlera middleware for Scrapy

A set of spiders and scrapers to extract location information from places that post their location on the internet.

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Random User-Agent middleware based on fake-useragent

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

百度贴吧爬虫(基于scrapy和mysql)

Declarative web scraping

TripAdvisor scraper

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

基于搜狗微信搜索的微信公众号爬虫接口

🚀电商网站爬虫合集，淘宝京东亚马逊等

TweetScraper is a simple crawler/spider for Twitter Search without using API

a reliable high-level web crawling & scraping framework for Node.js.

python爬虫

[Unmaintained] A simple and clean video/music/image downloader 👾

Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.

Scrapy Extension for monitoring spiders execution.

A Facebook crawler

一个灵活、友好的爬虫框架

Faster requests on Python 3

Stop stalking and start StopStalking 😉

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Seeker - another job board aggregator.

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Extract structured data from web sites. Web sites scraping.

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

use multiple proxies with Scrapy

The simple, easy to use command line web crawler.

A multi-thread crawler framework with many builtin image crawlers provided.

Uno Crawler por https://douban.com

A framework for creating semi-automatic web content extractors

python-Reptile

Possibly the best practice of Scrapy 🕷 and renting a house 🏡

Crawly, a high-level web crawling & scraping framework for Elixir.

a simple & tiny scrapy clustering solution, considered a drop-in replacement for scrapyd

No description or website provided.

Python入门网络爬虫之精华版

ISP Data Pollution to Protect Private Browsing History with Obfuscation

今日头条科技新闻接口爬虫

[OUTDATED] scrapy spiders to crawl the financial text data 📚 📜 pertinent to train word vectors 🚀

Life is short, *****.