9356 Open source projects that are alternatives of or similar to Ferret

Distributed crawler powered by Headless Chrome

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Crawly, a high-level web crawling & scraping framework for Elixir.

ProxyCrawl Python library for scraping and crawling

No description or website provided.

Scrapy, a fast high-level web crawling & scraping framework for Python.

[Unmaintained] A simple and clean video/music/image downloader 👾

Elegant Scraper and Crawler Framework for Golang

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

A cat(1) clone with wings.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

🔰 Instant PHP quality checks from your console

News, full-text, and article metadata extraction in Python 3. Advanced docs:

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Simple but useful Python web scraping tutorial code.

Universal scrapping tool, which allows you to extract data using multiple environments

a library to define a continuous delivery pipeline in code

A Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.

Find files with SQL-like queries

Lightweight package to query popular search engines and scrape for result titles, links and descriptions

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

Use SQL on various data sources

At present contains scraped data from around 1500 problems present on the site. More to follow....

List any node_modules directories in your system, as well as the space they take up. You can then select which ones you want to erase to free up space.

manipulate JSON files

A python module to manipulate XCode projects

**[ARCHIVED]** website changes tracker 🔍

🚀 A fast and lightweight utility and Python library for downloading posts, stories, and highlights from Instagram.

Command line program to download documents from web portals.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

HTTP API for Scrapy spiders

Headless Chrome For Java （Java 爬虫）

Scrapy Extension for monitoring spiders execution.

Hacktoberfest 2021 | World's largest Contributor driven code dataset | Algorithms that run our universe | Your personal library of every algorithm and data structure code that you will ever encounter |

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

一个灵活、友好的爬虫框架

Extract structured data from web sites. Web sites scraping.

A tool to develop and improve a student’s programming skills by introducing the earliest lessons of coding.

The KIImagePager is inspired by foursquare's ImageSlideshow, the user may scroll through images loaded from the Web

An open-source game launcher for your games

Graphback - Out of the box GraphQL server and client

A simple, fast and user-friendly alternative to 'find'

ack is a grep-like search tool optimized for source code.

To-do list & time tracker for programmers and other digital workers with Jira, Github, and Gitlab integration

快速、简洁且强大的PHP爬虫框架

Better than landingzones!

AskQL is a query language that can express any data request

Horusec is an open source tool that improves identification of vulnerabilities in your project with just one command.

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

A Python Tool For google Hacking

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

A JSON Query Language CLI tool