ulixee / Secret Agent
Licence: mit
The web browser that's built for scraping.
Stars: ✭ 151
Programming Languages
typescript
32286 projects
Projects that are alternatives of or similar to Secret Agent
mitm-play
Man in the middle using Playwright
Stars: ✭ 13 (-91.39%)
Mutual labels: mitm, chromium, mitmproxy, puppeteer
Marionette
Selenium alternative for Crystal. Browser manipulation without the Java overhead.
Stars: ✭ 119 (-21.19%)
Mutual labels: devtools, browser, puppeteer
LInkedIn-Reverese-Lookup
🔎Search LinkedIn profile by email address📧
Stars: ✭ 20 (-86.75%)
Mutual labels: scraping, chromium, puppeteer
Flaresolverr
Proxy server to bypass Cloudflare protection
Stars: ✭ 241 (+59.6%)
Mutual labels: proxy, puppeteer, chromium
Proxify
Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go.
Stars: ✭ 1,153 (+663.58%)
Mutual labels: proxy, mitm, mitmproxy
Wendigo
A proper monster for front-end automated testing
Stars: ✭ 121 (-19.87%)
Mutual labels: browser, puppeteer, chromium
Ssh Mitm
ssh mitm server for security audits supporting public key authentication, session hijacking and file manipulation
Stars: ✭ 335 (+121.85%)
Mutual labels: proxy, mitm, mitmproxy
Headless Chrome Crawler
Distributed crawler powered by Headless Chrome
Stars: ✭ 5,129 (+3296.69%)
Mutual labels: scraping, puppeteer, chromium
Proxy.py
⚡⚡⚡Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on Network monitoring, controls & Application development, testing, debugging
Stars: ✭ 1,291 (+754.97%)
Mutual labels: proxy, mitm, mitmproxy
Hotel
🏩 A simple process manager for developers. Start apps from your browser and access them using local domains
Stars: ✭ 9,736 (+6347.68%)
Mutual labels: proxy, devtools
Puppeteer Webperf
Automating Web Performance testing with Puppeteer 🎪
Stars: ✭ 1,392 (+821.85%)
Mutual labels: devtools, puppeteer
Bettercap
The Swiss Army knife for 802.11, BLE, IPv4 and IPv6 networks reconnaissance and MITM attacks.
Stars: ✭ 10,735 (+7009.27%)
Mutual labels: proxy, mitm
Grawler
Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.
Stars: ✭ 98 (-35.1%)
Mutual labels: scraping, proxy
Mitm Scripts
🔄 A collection of mitmproxy inline scripts
Stars: ✭ 109 (-27.81%)
Mutual labels: proxy, mitmproxy
Sushi Browser
Sushi Browser is the next generation browser which mounts the multi-panel and the video support function and so on. Its goal is to be as fantastic as sushi. 🍣
Stars: ✭ 116 (-23.18%)
Mutual labels: browser, puppeteer
Kdeconnect Chrome Extension
A browser extension to send pages and content from your browser to connected KDE Connect devices.
Stars: ✭ 124 (-17.88%)
Mutual labels: browser, chromium
Chrome Protocol Proxy
Chrome DevTools Protocol Proxy - intelligent proxy for debugging purposes
Stars: ✭ 94 (-37.75%)
Mutual labels: proxy, chromium
Awesome Puppeteer
A curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (+1044.37%)
Mutual labels: scraping, puppeteer
Vscode Browse Lite
🚀 An embedded browser in VS Code
Stars: ✭ 134 (-11.26%)
Mutual labels: devtools, browser
SecretAgent
SecretAgent is a web browser that's built for scraping.
- [x] Built for scraping - it's the first modern headless browsers designed specifically for scraping instead of just automated testing.
- [x] Designed for web developers - We've recreated a fully compliant DOM directly in NodeJS allowing you bypass the headaches of previous scraper tools.
- [x] Powered by Chrome - The powerful Chrome engine sits under the hood, allowing for lightning fast rendering.
- [x] Emulates any modern browser - BrowserEmulators make it easy to disguise your script as practically any browser.
- [x] Avoids detection along the entire stack - Don't be blocked because of TLS fingerprints in your networking stack.
Check out our website for more details.
Installation
npm i --save secret-agent
or
yarn add secret-agent
Usage
SecretAgent provides access to the W3C DOM specification without the need for Puppeteer's complicated evaluate callbacks and multi-context switching:
const agent = require('secret-agent');
(async () => {
await agent.goto('https://example.org');
const title = await agent.document.title;
const intro = await agent.document.querySelector('p').textContent;
await agent.close();
})();
Browse the full API docs.
Contributing
We'd love your help in making SecretAgent a better tool. Please don't hesitate to send a pull request.
License
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].