Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (-47.45%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+1387.96%)
PoliteBe nice on the web
Stars: ✭ 253 (-7.66%)
Weibo terminator workflowUpdate Version of weibo_terminator, This is Workflow Version aim at Get Job Done!
Stars: ✭ 259 (-5.47%)
bots-zooNo description or website provided.
Stars: ✭ 59 (-78.47%)
Lulu[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (+187.96%)
DotnetcrawlerDotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
Stars: ✭ 100 (-63.5%)
ScrapoxyScrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (+382.48%)
Php CrawlerA php crawler that finds emails on the internets
Stars: ✭ 119 (-56.57%)
OnegramThis repository is no longer maintained.
Stars: ✭ 137 (-50%)
Querylist🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+772.99%)
dijnet-botAz összes számlád még egy helyen :)
Stars: ✭ 17 (-93.8%)
ScrapyrtHTTP API for Scrapy spiders
Stars: ✭ 637 (+132.48%)
GoscraperGolang pkg to quickly return a preview of a webpage (title/description/images)
Stars: ✭ 72 (-73.72%)
WombatLightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Stars: ✭ 1,220 (+345.26%)
Instagram-Scraper-2021Scrape Instagram content and stories anonymously, using a new technique based on the har file (No Token + No public API).
Stars: ✭ 57 (-79.2%)
PypergrabberFetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.
Stars: ✭ 14 (-94.89%)
Instagram CrawlerCrawl instagram photos, posts and videos for download.
Stars: ✭ 178 (-35.04%)
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-30.66%)
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+5569.71%)
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (-22.99%)
Ruiji.netcrawler framework, distributed crawler extractor
Stars: ✭ 220 (-19.71%)
Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (-15.69%)
newspaperjsNews extraction and scraping. Article Parsing
Stars: ✭ 59 (-78.47%)
FbcrawlA Facebook crawler
Stars: ✭ 536 (+95.62%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+1649.27%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+1665.33%)
CrawlerA high performance web crawler in Elixir.
Stars: ✭ 781 (+185.04%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+139.42%)
ScrapitScraping scripts for various websites.
Stars: ✭ 25 (-90.88%)
Jd AutobuyPython爬虫,京东自动登录,在线抢购商品
Stars: ✭ 1,174 (+328.47%)
Social ScraperTổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Stars: ✭ 47 (-82.85%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+354.74%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+2868.25%)
NewspaperNews, full-text, and article metadata extraction in Python 3. Advanced docs:
Stars: ✭ 11,545 (+4113.5%)
ScrapedinLinkedIn Scraper (currently working 2020)
Stars: ✭ 453 (+65.33%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-37.59%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (-41.61%)
JvppeteerHeadless Chrome For Java (Java 爬虫)
Stars: ✭ 193 (-29.56%)
Instagram Scraperscrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
Stars: ✭ 2,209 (+706.2%)
Media ScraperScrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
Stars: ✭ 206 (-24.82%)
Tianyanchapip安装的天眼查爬虫API,指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.
Stars: ✭ 206 (-24.82%)
Annie👾 Fast and simple video download library and CLI tool written in Go
Stars: ✭ 16,369 (+5874.09%)
newsembleAPI for fetching data from news websites.
Stars: ✭ 42 (-84.67%)
bing-ip2hostsbingip2hosts is a Bing.com web scraper that discovers websites by IP address
Stars: ✭ 99 (-63.87%)
Mimo-CrawlerA web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.
Stars: ✭ 22 (-91.97%)
robotstxtrobots.txt file parsing and checking for R
Stars: ✭ 65 (-76.28%)
metacritic apiPHP Metacritic API - Mirrored by my GitLab
Stars: ✭ 31 (-88.69%)
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+60.58%)
Google Play ScraperGoogle play scraper for Python inspired by <facundoolano/google-play-scraper>
Stars: ✭ 143 (-47.81%)
arachnodHigh performance crawler for Nodejs
Stars: ✭ 17 (-93.8%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-94.53%)