WombatLightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Stars: ✭ 1,220 (-44.77%)
Is GoogleVerify that a request is from Google crawlers using Google's DNS verification steps
Stars: ✭ 82 (-96.29%)
PoliteBe nice on the web
Stars: ✭ 253 (-88.55%)
Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (-89.54%)
JvppeteerHeadless Chrome For Java (Java 爬虫)
Stars: ✭ 193 (-91.26%)
scrapersscrapers for building your own image databases
Stars: ✭ 46 (-97.92%)
stweetAdvanced python library to scrap Twitter (tweets, users) from unofficial API
Stars: ✭ 287 (-87.01%)
Mygf InstagramLike posts of my girlfriend's Instagram using web api, don't need any api key and access tokens just login and password
Stars: ✭ 140 (-93.66%)
Laravel Crawler DetectA Laravel wrapper for CrawlerDetect - the web crawler detection library
Stars: ✭ 227 (-89.72%)
instagram-get-imagesInstagram get images 🌄 (hashtags, account, locations) with puppeteer
Stars: ✭ 69 (-96.88%)
instagram-hashtag-scraperNodeJS application for scraping recent top posts from Instagram by hashtag without API access.
Stars: ✭ 17 (-99.23%)
botCompletely free and open-source human-like Instagram bot. Powered by UIAutomator2 and compatible with basically any Android device 5.0+ that can run Instagram - real or emulated.
Stars: ✭ 321 (-85.47%)
arachnodHigh performance crawler for Nodejs
Stars: ✭ 17 (-99.23%)
bots-zooNo description or website provided.
Stars: ✭ 59 (-97.33%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-99.32%)
lightnovel epub🍭 epub generator for (light)novels (轻) 小说 epub 生成器,支持站点:轻之国度、轻小说文库
Stars: ✭ 89 (-95.97%)
LinkedinLinkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (-86.01%)
Hquery.phpAn extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.
Stars: ✭ 295 (-86.65%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (-43.59%)
Instagram Profilecrawl💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!
Stars: ✭ 110 (-95.02%)
InstagramcrawlerA non API python program to crawl public photos, posts or followers
Stars: ✭ 349 (-84.2%)
Freshonions TorscraperFresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
Stars: ✭ 348 (-84.25%)
Moodle Downloader 2A Moodle downloader that downloads course content fast from Moodle (eg. lecture pdfs)
Stars: ✭ 118 (-94.66%)
ScrapedinLinkedIn Scraper (currently working 2020)
Stars: ✭ 453 (-79.49%)
Instagram ScraperScrapes an instagram user's photos and videos
Stars: ✭ 5,664 (+156.41%)
ScrapyrtHTTP API for Scrapy spiders
Stars: ✭ 637 (-71.16%)
Instagram4j📷 Instagram private API in Java
Stars: ✭ 629 (-71.53%)
Lulu[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (-64.28%)
Instagram Python ScraperA instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!
Stars: ✭ 115 (-94.79%)
PypergrabberFetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.
Stars: ✭ 14 (-99.37%)
CrawlerA high performance web crawler in Elixir.
Stars: ✭ 781 (-64.64%)
AvbookAV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+268.18%)
InstautoSimple to use wrapper around the private Instagram API, written in Python.
Stars: ✭ 52 (-97.65%)
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (-80.08%)
Instascrape🚀 A fast and lightweight utility and Python library for downloading posts, stories, and highlights from Instagram.
Stars: ✭ 76 (-96.56%)
GoscraperGolang pkg to quickly return a preview of a webpage (title/description/images)
Stars: ✭ 72 (-96.74%)
Jd AutobuyPython爬虫,京东自动登录,在线抢购商品
Stars: ✭ 1,174 (-46.85%)
Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (-93.48%)
ScrapoxyScrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (-40.15%)
InstaloctrackAn Instagram OSINT tool to collect all the geotagged locations available on an Instagram profile in order to plot them on a map, and dump them in a JSON.
Stars: ✭ 85 (-96.15%)
SkraperKotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, IFunny, VK, Pikabu)
Stars: ✭ 72 (-96.74%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (-92.76%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-92.26%)
Igbot🐙 Free scripts, bots and Python API wrapper. Get free followers with our auto like, auto follow and other scripts!
Stars: ✭ 4,094 (+85.33%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (-96.92%)
Instalike Instagram Botautomate instagram activities using instagram bot - like follow/unfollow comment; python requests lib
Stars: ✭ 128 (-94.21%)
Instabot.rbAn instagram bot works without instagram api, only needs your username and password. written in ruby
Stars: ✭ 149 (-93.25%)