1244 Open source projects that are alternatives of or similar to Awesome Java Crawler
Instagramcrawler
A non API python program to crawl public photos, posts or followers
Stars: ✭ 349 (+53.07%)
Singlefile
Web Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a single HTML file
Stars: ✭ 4,417 (+1837.28%)
Chromeless
🖥 Chrome automation made simple. Runs locally or headless on AWS Lambda.
Stars: ✭ 13,254 (+5713.16%)
Price Monitor
京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取
Stars: ✭ 634 (+178.07%)
Dockselpy
Dockerized Selenium and Python with support for Chrome, Firefox and PhantomJS
Stars: ✭ 237 (+3.95%)
Undetected Chromedriver
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Stars: ✭ 365 (+60.09%)
Tianyancha
pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.
Stars: ✭ 206 (-9.65%)
Python3 Spider
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
Stars: ✭ 2,129 (+833.77%)
Docker Python Xvfb Selenium Chrome Firefox
Dockerfiles for Python 3.6/2.7 & Selenium in a headless Chrome or Firefox environment
Stars: ✭ 82 (-64.04%)
Instagram Profilecrawl
📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.
Stars: ✭ 816 (+257.89%)
Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stars: ✭ 125 (-45.18%)
Skrape.it
A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (+1.32%)
Instagram Profilecrawl
💻 Quickly crawl the information (e.g. followers, tags, etc...) of an instagram profile. No login required!
Stars: ✭ 110 (-51.75%)
Examples Of Web Crawlers
一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )
Stars: ✭ 10,724 (+4603.51%)
Selenium Python Helium
Selenium-python but lighter: Helium is the best Python library for web automation.
Stars: ✭ 2,732 (+1098.25%)
Infospider
INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。
Stars: ✭ 5,984 (+2524.56%)
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-56.14%)
Arsenic
Async WebDriver implementation for asyncio and asyncio-compatible frameworks
Stars: ✭ 209 (-8.33%)
Algoliasearch Netlify
Official Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler
Stars: ✭ 208 (-8.77%)
Zalenium
A flexible and scalable container based Selenium Grid with video recording, live preview, basic auth & dashboard.
Stars: ✭ 2,381 (+944.3%)
Media Scraper
Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
Stars: ✭ 206 (-9.65%)
Arachnid
Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites
Stars: ✭ 224 (-1.75%)
Simple Headless Chrome
Simple abstraction to use Chrome as a Headless Browser with Node JS
Stars: ✭ 216 (-5.26%)