All Projects → ayakashi-io → Ayakashi

ayakashi-io / Ayakashi

Licence: other
⚡️ Ayakashi.io - The next generation web scraping framework

Programming Languages

typescript
32286 projects

Projects that are alternatives of or similar to Ayakashi

Apify Js
Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Stars: ✭ 3,154 (+2595.73%)
Mutual labels:  automation, web-scraping, headless-chrome
Rpa
UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Stars: ✭ 477 (+307.69%)
Mutual labels:  automation, web-scraping
Autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+3384.62%)
Mutual labels:  automation, web-scraping
Ferrum
Headless Chrome Ruby API
Stars: ✭ 1,009 (+762.39%)
Mutual labels:  automation, headless-chrome
iww
AI based web-wrapper for web-content-extraction
Stars: ✭ 61 (-47.86%)
Mutual labels:  data-mining, web-scraping
Playwright Go
Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
Stars: ✭ 272 (+132.48%)
Mutual labels:  automation, headless-chrome
Navalia
A bullet-proof, fast, and reliable headless browser API
Stars: ✭ 950 (+711.97%)
Mutual labels:  automation, headless-chrome
Chirp
Interface to manage and centralize Google Alert information
Stars: ✭ 227 (+94.02%)
Mutual labels:  automation, data-mining
Golang Chrome Automation
Automate Chrome tasks with Golang and ChromeDP
Stars: ✭ 58 (-50.43%)
Mutual labels:  automation, headless-chrome
Decapitated
Headless 'Chrome' Orchestration in R
Stars: ✭ 65 (-44.44%)
Mutual labels:  web-scraping, headless-chrome
Api Store
Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!
Stars: ✭ 69 (-41.03%)
Mutual labels:  automation, headless-chrome
codepen-puppeteer
Use Puppeteer to download pens from Codepen.io as single html pages
Stars: ✭ 22 (-81.2%)
Mutual labels:  web-scraping, headless-chrome
Puppeteer
Headless Chrome Node.js API
Stars: ✭ 75,197 (+64170.94%)
Mutual labels:  automation, headless-chrome
Taiko
A node.js library for testing modern web applications
Stars: ✭ 2,964 (+2433.33%)
Mutual labels:  automation, headless-chrome
Chrome Devtools Protocol
Chrome Devtools Protocol client for PHP
Stars: ✭ 112 (-4.27%)
Mutual labels:  automation, headless-chrome
Nickjs
Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
Stars: ✭ 494 (+322.22%)
Mutual labels:  automation, headless-chrome
Phantomas
Headless Chromium-based web performance metrics collector and monitoring tool
Stars: ✭ 2,191 (+1772.65%)
Mutual labels:  automation, headless-chrome
Puppeteer Extra
💯 Teach puppeteer new tricks through plugins.
Stars: ✭ 3,397 (+2803.42%)
Mutual labels:  automation, headless-chrome
Puphpeteer
A Puppeteer bridge for PHP, supporting the entire API.
Stars: ✭ 1,014 (+766.67%)
Mutual labels:  automation, headless-chrome
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-14.53%)
Mutual labels:  automation, web-scraping

Ayakashi

Get Started
npm license
test suite


The next generation web scraping framework

The web has changed. Gone are the days that raw html parsing scripts were the proper tool for the job.
Javascript and single page applications are now the norm.
Demand for data scraping and automation is higher than ever, from business needs to data science and machine learning.
Our tools need to evolve.

Ayakashi helps you build scraping and automation systems that are

  • easy to build
  • simple or sophisticated
  • highly performant
  • maintainable and built for change

Powerful querying and data models

Ayakashi's way of finding things in the page and using them is done with props and domQL.
Directly inspired by the relational database world (and SQL), domQL makes DOM access easy and readable no matter how obscure the page's structure is.
Props are the way to package domQL expressions as re-usable structures which can then be passed around to actions or to be used as models for data extraction.

domql

High level builtin actions

Ready made actions so you can focus on what matters.
Easily handle infinite scrolling, single page navigation, events and more.
Plus, you can always build your own actions, either from scratch or by composing other actions.

Preload code on pages

Need to include a bunch of code, a library you made or a 3rd party module and make it available on a page?
Preloaders have you covered.

Control how you save your data

Automatically save your extracted data to all major SQL engines, JSON and CSV.
Need something more exotic or the ability to control exactly how the data is persisted?
Package and plug your custom logic as a script.

Manage the flow with pipelines

Scraping the data is only one part of the deal.
How about something like this:

pipelines

Need it to also be clean, readable and performant?
If so, pipelines can help.

Utilize all your cores

Ayakashi can utilize available cores as needed. Especially useful for projects that need to run multiple operations in parallel.

Extend it as you like

All APIs used to build the builtin functionality are properly exposed.
All core entities are composable and extensible.

Use the language of the web

Many argue about javascript and its quirkiness as a language but the truth is:
If you want to scrape the web, you should speak its language.

Great editor support

Ayakashi comes bundled with a fully documented public API that you can explore directly in your editor.
Autocomplete any method, check signatures and examples or follow links to more documentation.

editor support

Sounds cool?
Just head over to the getting started guide!


Documentation
Roadmap
Changelog
Twitter

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].