ScrapedinLinkedIn Scraper (currently working 2020)
Stars: ✭ 453 (-71.79%)
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (-72.6%)
PypergrabberFetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.
Stars: ✭ 14 (-99.13%)
bots-zooNo description or website provided.
Stars: ✭ 59 (-96.33%)
ToapiEvery web site provides APIs.
Stars: ✭ 3,209 (+99.81%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+153.86%)
ScrapyrtHTTP API for Scrapy spiders
Stars: ✭ 637 (-60.34%)
ScrapitScraping scripts for various websites.
Stars: ✭ 25 (-98.44%)
Annie👾 Fast and simple video download library and CLI tool written in Go
Stars: ✭ 16,369 (+919.24%)
PymarketcapPython3 API wrapper and web scraper for https://coinmarketcap.com
Stars: ✭ 73 (-95.45%)
Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (-85.62%)
dijnet-botAz összes számlád még egy helyen :)
Stars: ✭ 17 (-98.94%)
Weixin Spider微信公众号爬虫,公众号历史文章,文章评论,文章阅读及在看数据,可视化web页面,可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现,高效微信爬虫,微信公众号爬虫,历史文章,文章评论,数据更新。
Stars: ✭ 287 (-82.13%)
Hquery.phpAn extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.
Stars: ✭ 295 (-81.63%)
GosintOSINT Swiss Army Knife
Stars: ✭ 401 (-75.03%)
lightnovel epub🍭 epub generator for (light)novels (轻) 小说 epub 生成器,支持站点:轻之国度、轻小说文库
Stars: ✭ 89 (-94.46%)
JikanUnofficial MyAnimeList PHP+REST API which provides functions other than the official API
Stars: ✭ 531 (-66.94%)
FbcrawlA Facebook crawler
Stars: ✭ 536 (-66.63%)
DuckduckgoAn unofficial DuckDuckGo search API.
Stars: ✭ 6 (-99.63%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (-59.15%)
Social ScraperTổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Stars: ✭ 47 (-97.07%)
Jd AutobuyPython爬虫,京东自动登录,在线抢购商品
Stars: ✭ 1,174 (-26.9%)
Ruiji.netcrawler framework, distributed crawler extractor
Stars: ✭ 220 (-86.3%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (-22.42%)
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (-86.86%)
PoliteBe nice on the web
Stars: ✭ 253 (-84.25%)
Media ScraperScrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
Stars: ✭ 206 (-87.17%)
arachnodHigh performance crawler for Nodejs
Stars: ✭ 17 (-98.94%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-99.07%)
Tianyanchapip安装的天眼查爬虫API,指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.
Stars: ✭ 206 (-87.17%)
Crawlertutorial爬蟲極簡教學(fetch, parse, search, multiprocessing, API)- PTT 為例
Stars: ✭ 282 (-82.44%)
RcrawlerAn R web crawler and scraper
Stars: ✭ 274 (-82.94%)
Weibo terminator workflowUpdate Version of weibo_terminator, This is Workflow Version aim at Get Job Done!
Stars: ✭ 259 (-83.87%)
Freshonions TorscraperFresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
Stars: ✭ 348 (-78.33%)
Xcrawler快速、简洁且强大的PHP爬虫框架
Stars: ✭ 344 (-78.58%)
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+867.31%)
GoogledictionaryapiGoogle does not provide Google Dictionary API so I created one.
Stars: ✭ 528 (-67.12%)
FinvizUnofficial API for finviz.com
Stars: ✭ 493 (-69.3%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+198.44%)
Lulu[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (-50.87%)
CrawlerA high performance web crawler in Elixir.
Stars: ✭ 781 (-51.37%)
Voyages Sncf ApiA scrapy spider that scraps times and prices from Voyages Sncf. It uses scrapyrt to provide an API interface.
Stars: ✭ 7 (-99.56%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+201.18%)
PixevalA Strong, Fast and Flexible Pixiv Client based on .NET Core and WPF
Stars: ✭ 1,031 (-35.8%)
GoscraperGolang pkg to quickly return a preview of a webpage (title/description/images)
Stars: ✭ 72 (-95.52%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+406.41%)
JvppeteerHeadless Chrome For Java (Java 爬虫)
Stars: ✭ 193 (-87.98%)
Querylist🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+48.94%)
AnitopAnitop is an unofficial simple API from https://anitrendz.net/ site
Stars: ✭ 30 (-98.13%)
WombatLightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Stars: ✭ 1,220 (-24.03%)
ScrapoxyScrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (-17.68%)