271 Open source projects that are alternatives of or similar to ksoup

Python (asyncio) wrapper for hackernews api

Elegant Scraper and Crawler Framework for Golang

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Transistor, a Python web scraping framework for intelligent use cases.

🦀 Scraping & crawling websites using Rust, and ReasonML

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

Image Dataset Tool (idt) is a cli tool designed to make the otherwise repetitive and slow task of creating image datasets into a fast and intuitive process.

A website dedicated to making recipes from any website easy to read.

simple multi-level scraper json input/output for Cheerio

🤖 A Versatile Telegram Bot

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

youtube & tiktok analysis + youchoose recommendation custmizer. backend, extensions, and tooling

Linkedin Learning videos downloader

Pillaging the seven seas for torrents, pieces of eight and other bounty.

The web browser that's built for scraping.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

Tool to scrape linkedin

PHP Scraper - an highly opinionated web-interface for PHP

The simplest demo of chrome automation by python and selenium in AWS Lambda

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.

The 4CAT Capture and Analysis Toolkit provides modular data capture & analysis for a variety of social media platforms.

Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!

⚽ Instantly find 🏆EURO 2016 live-streams & highlights, now a Web App!

scan for webcams on the internet

extract data from html table

Distributed crawler, database and web frontend for public directories indexing

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

Simple scriptes for Level UP your scraping Skills, and source code for Level UP playlist on Youtube

A Selenium-driven tool for automated website interaction and scraping.

Scrapy, a fast high-level web crawling & scraping framework for Python.

Pahe.ph [Pahe.in] Movies Website Scraper

Laravel package to collect your bank statements history. Currently support for parsing statements history from BCA, Mandiri, BNI, and MUAMALAT e-banking websites.

Provides a virtual web browser (a.k.a. "headless browser") appearing as a node.

Python scraper for Language Pods such as Japanesepod101.com 👹 🗾 🍣 Compatible with Japanese, Chinese, French, German, Italian, Korean, Portuguese, Russian, Spanish and many more! ✨

Tiny little ruby on rails website that crawls though your public github repos to find out what your favourite languages are.

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Lightweight parallel Java tasks

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

Generate a network graph of connections from your WhatsApp groups data

Example end to end data engineering project.

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

A Telegram bot combined with python to serve some basic functions like weather, music charts, cricket score and much more.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

👨‍👩‍👦 Social account detection and extraction in Python, e.g. for crawling/scraping.

ASP.NET View State Decoder

Use SQL on various data sources

Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!

Overwatch API library for player details and career stats

scrapers for building your own image databases

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

🕵‍♂ Bot detection tests for Puppeteer. Hide and seek!

Easily crawl news portals or blog sites using Storm Crawler.

This app download all the book available on (http://libgen.io/)

📊 Python tool to scrape real-time information about ETFs from the web and mixing them together by proportionally distributing their assets allocation

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.