DaftlistingsA library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.
Stars: ✭ 86 (-30.65%)
WrpWeb Rendering Proxy: Use vintage, historical, legacy browsers on modern web
Stars: ✭ 503 (+305.65%)
Project TauroA Router WiFi key recovery/cracking tool with a twist.
Stars: ✭ 52 (-58.06%)
User AgentsA JavaScript library for generating random user agents with data that's updated daily.
Stars: ✭ 485 (+291.13%)
Scrapyd Cluster On HerokuSet up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉
Stars: ✭ 106 (-14.52%)
PychromeA Python Package for the Google Chrome Dev Protocol [threading base]
Stars: ✭ 469 (+278.23%)
Puppeteer DeepPuppeteer, Headless Chrome;爬取《es6标准入门》、自动推文到掘金、站点性能分析;高级爬虫、自动化UI测试、性能分析;
Stars: ✭ 1,033 (+733.06%)
Pptraas.comPuppeteer as a service
Stars: ✭ 433 (+249.19%)
Detect CmsPHP Library for detecting CMS
Stars: ✭ 78 (-37.1%)
Docker Python ChromedriverDockerfile for running Python Selenium in headless Chrome (Python 2.7 / 3.6 / 3.7 / 3.8 / Alpine based Python / Chromedriver / Selenium / Xvfb included in different versions)
Stars: ✭ 385 (+210.48%)
FerrumHeadless Chrome Ruby API
Stars: ✭ 1,009 (+713.71%)
NightmareA high-level browser automation library.
Stars: ✭ 19,067 (+15276.61%)
actor-scraperHouse of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.
Stars: ✭ 83 (-33.06%)
Proxy ChainNode.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.
Stars: ✭ 374 (+201.61%)
Chrome PoolHeadless chrome tabs manage pool
Stars: ✭ 40 (-67.74%)
Webstera reliable high-level web crawling & scraping framework for Node.js.
Stars: ✭ 364 (+193.55%)
Mochify.js☕️ TDD with Browserify, Mocha, Headless Chrome and WebDriver
Stars: ✭ 338 (+172.58%)
PulsarTurn large Web sites into tables and charts using simple SQLs.
Stars: ✭ 100 (-19.35%)
Ionic Boilerplate✨ An Ionic Starter kit featuring Tests, E2E, Karma, Protractor, Jasmine, Istanbul, Gitlab CI, Automatic IPA and APK, TypeScript 2, TsLint, Codelyzer, Typedoc, Yarn, Rollup, and Webpack 2
Stars: ✭ 309 (+149.19%)
Actor Google Search ScraperApify actor that crawls Google Search result pages (SERPs) and extracts a list of organic results, ads, related queries and more. It supports selection of custom country, language and location.
Stars: ✭ 38 (-69.35%)
Gopa[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+123.39%)
Api StoreContains all the public APIs listed in Phantombuster's API store. Pull requests welcome!
Stars: ✭ 69 (-44.35%)
RendertronA Headless Chrome rendering solution
Stars: ✭ 5,593 (+4410.48%)
NavaliaA bullet-proof, fast, and reliable headless browser API
Stars: ✭ 950 (+666.13%)
CriType safe go bindings to interact with chrome remote interface.
Stars: ✭ 119 (-4.03%)
WebmiddleNode.js framework for modular web scraping and data extraction
Stars: ✭ 13 (-89.52%)
hawk-eye前端监控:定时监控站点渲染情况,记录异常并保存截图: puppeteer, thinkjs,mongodb,headless-chrome,vuejs
Stars: ✭ 88 (-29.03%)
CascadiaGo cascadia package command line CSS selector
Stars: ✭ 67 (-45.97%)
Text-AnalysisExplaining textual analysis tools in Python. Including Preprocessing, Skip Gram (word2vec), and Topic Modelling.
Stars: ✭ 48 (-61.29%)
Letterboxd recommendationsScraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username
Stars: ✭ 23 (-81.45%)
raspagem-de-dados-fatec📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí
Stars: ✭ 22 (-82.26%)
Splashr💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R
Stars: ✭ 93 (-25%)
Youtube tutorialsCollection of scripts corresponding to LucidProgramming YouTube tutorials
Stars: ✭ 769 (+520.16%)
Page2image📷 page2image is a npm package for taking screenshots which also provides CLI command
Stars: ✭ 66 (-46.77%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-87.9%)
BrowserlessA browser driver on top of puppeteer, ready for production scenarios.
Stars: ✭ 664 (+435.48%)
Save For OfflineAndroid app for saving webpages for offline reading.
Stars: ✭ 114 (-8.06%)
sp-subway-scraper🚆This web scraper builds a dataset for São Paulo subway operation status
Stars: ✭ 24 (-80.65%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+429.03%)
headless-chromeImplementation of the new headless chrome with chromedriver and selenium.
Stars: ✭ 34 (-72.58%)
halfstaff🇺🇸 Is the US flag at half-staff?
Stars: ✭ 22 (-82.26%)
investigation-amazon-brandsMaterials to reproduce our findings in our stories, "Amazon Puts Its Own 'Brands' First Above Better-Rated Products" and "When Amazon Takes the Buy Box, it Doesn’t Give it up"
Stars: ✭ 56 (-54.84%)
Puppeteer DartA Dart library to automate the Chrome browser over the DevTools Protocol. This is a port of the Puppeteer API
Stars: ✭ 92 (-25.81%)
ChromyChromy is a library for operating headless chrome. 🍺🍺🍺
Stars: ✭ 593 (+378.23%)
CoolqlcoolNextjs server to query websites with GraphQL
Stars: ✭ 623 (+402.42%)
Awesome PuppeteerA curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (+1293.55%)
30 Days Of PythonLearn Python for the next 30 (or so) Days.
Stars: ✭ 1,748 (+1309.68%)
HumanoidNode.js package to bypass CloudFlare's anti-bot JavaScript challenges
Stars: ✭ 88 (-29.03%)
CrawlergoA powerful dynamic crawler for web vulnerability scanners
Stars: ✭ 1,088 (+777.42%)