ImdbpyIMDbPY is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies
Stars: ✭ 792 (+3068%)
Python Spider豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章
Stars: ✭ 615 (+2360%)
Php GooseReadability / Html Content / Article Extractor & Web Scrapping library written in PHP
Stars: ✭ 392 (+1468%)
Indonesia News ScraperA news scraper for nodejs that help to scrap news from Indonesian news portal.
Stars: ✭ 18 (-28%)
Docker Python ChromedriverDockerfile for running Python Selenium in headless Chrome (Python 2.7 / 3.6 / 3.7 / 3.8 / Alpine based Python / Chromedriver / Selenium / Xvfb included in different versions)
Stars: ✭ 385 (+1440%)
Osi.igInformation Gathering Instagram.
Stars: ✭ 377 (+1408%)
Foxr🦊 Node.js API to control Firefox
Stars: ✭ 783 (+3032%)
NetdiscoveryNetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。
Stars: ✭ 573 (+2192%)
Undetected ChromedriverCustom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Stars: ✭ 365 (+1360%)
ScrapersA list of scrapers from around the web.
Stars: ✭ 366 (+1364%)
Scrapy SeleniumScrapy middleware to handle javascript pages using selenium
Stars: ✭ 550 (+2100%)
AtataC#/.NET test automation framework for web
Stars: ✭ 362 (+1348%)
FluentleniumFluentLenium is a website & mobile automation framework which extends Selenium to write reliable and resilient UI functional tests. This framework is React ready. Written and maintained by people who are automating browser-based tests on a daily basis.
Stars: ✭ 766 (+2964%)
InstagramcrawlerA non API python program to crawl public photos, posts or followers
Stars: ✭ 349 (+1296%)
CheerioFast, flexible, and lean implementation of core jQuery designed specifically for the server.
Stars: ✭ 24,616 (+98364%)
Serenity JsA next generation, full-stack acceptance testing framework optimised for collaboration, speed and scale!
Stars: ✭ 346 (+1284%)
Gifhub📈 Create GIFs from user's GitHub activity graph
Stars: ✭ 17 (-32%)
XidelCommand line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Stars: ✭ 335 (+1240%)
FbcrawlA Facebook crawler
Stars: ✭ 536 (+2044%)
JavgoJavGo是一个集合影片管理,影片刮削,视频处理,资源搜索等综合一体的全功能影音软件,支持爬取javbus,jav321,javdb,javlibrary进行刮削,支持db,bus的磁力搜索,支持获取library的影片评论。
Stars: ✭ 338 (+1252%)
OnlyfansScrape all the media from an OnlyFans account - Updated regularly
Stars: ✭ 731 (+2824%)
Docker AndroidAndroid in docker solution with noVNC supported and video recording
Stars: ✭ 4,042 (+16068%)
Docker SeleniumDocker images for the Selenium Grid Server
Stars: ✭ 5,476 (+21804%)
Proxy ScraperProxy-Scraper is simple Perl script for scraping proxies from multiple websites.
Stars: ✭ 24 (-4%)
CryptocmdCryptocurrency historical price data library in Python. Data from https://coinmarketcap.com.
Stars: ✭ 299 (+1096%)
GoogledictionaryapiGoogle does not provide Google Dictionary API so I created one.
Stars: ✭ 528 (+2012%)
Socialmanagertools Gui🤖 👻 Desktop application for Instagram Bot, Twitter Bot and Facebook Bot
Stars: ✭ 293 (+1072%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+2524%)
WebinspectorRuby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.
Stars: ✭ 288 (+1052%)
Operative Frameworkoperative framework is a OSINT investigation framework, you can interact with multiple targets, execute multiple modules, create links with target, export rapport to PDF file, add note to target or results, interact with RESTFul API, write your own modules.
Stars: ✭ 511 (+1944%)
Java Spider一个基于webmagic框架二次开发的java爬虫框架实战,已实现能爬取腾讯,搜狐,今日头条(单独集成功能)等资讯内容,配合elasticsearch框架用法,实现了自动爬虫,已投入线上生产使用。
Stars: ✭ 276 (+1004%)
ReginaFetch new releases from http://www.juno.co.uk/.
Stars: ✭ 6 (-76%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+19072%)
Playwright GoPlaywright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
Stars: ✭ 272 (+988%)
Instagram CrawlerGet Instagram posts/profile/hashtag data without using Instagram API
Stars: ✭ 643 (+2472%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+19248%)
The EndorserAn OSINT tool that allows you to draw out relationships between people on LinkedIn via endorsements/skills.
Stars: ✭ 269 (+976%)
WebdriversKeep your Selenium WebDrivers updated automatically
Stars: ✭ 466 (+1764%)
GgrA lightweight load balancer used to create big Selenium clusters
Stars: ✭ 268 (+972%)
PoiumSelenium/appium-based Page Objects test library
Stars: ✭ 261 (+944%)
Find Cheapest FlightsUse Google Flights API and scrape Expedia to find the cheapest/shortest flights!
Stars: ✭ 18 (-28%)
DuckduckgoAn unofficial DuckDuckGo search API.
Stars: ✭ 6 (-76%)
Practical.cleanarchitectureAsp.Net Core 5 Clean Architecture (Microservices, Modular Monolith, Monolith) samples (+Blazor, Angular 11, React 17, Vue 2.6), Domain-Driven Design, CQRS, Event Sourcing, SOLID, Asp.Net Core Identity Custom Storage, Identity Server 4 Admin UI, Entity Framework Core, Selenium E2E Testing, SignalR Notification, Hangfire Tasks Scheduling, Health Checks, Security Headers, ...
Stars: ✭ 639 (+2456%)
Weibo terminator workflowUpdate Version of weibo_terminator, This is Workflow Version aim at Get Job Done!
Stars: ✭ 259 (+936%)
PywhatsappPython Automation using selenium & Scheduling of messages and media
Stars: ✭ 257 (+928%)
HtmlelementsHtml Elements is a Java framework providing easy-to-use way of interaction with web-page elements in web-page tests.
Stars: ✭ 258 (+932%)
DorknetSelenium powered Python script to automate searching for vulnerable web apps.
Stars: ✭ 256 (+924%)
Selenium PythonSelenium Python Bindings Documentation
Stars: ✭ 640 (+2460%)
Php WebdriverPHP client for Selenium/WebDriver protocol. Previously facebook/php-webdriver
Stars: ✭ 4,477 (+17808%)