243 Open source projects that are alternatives of or similar to pickall

Lightweight package to query popular search engines and scrape for result titles, links and descriptions

Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

OCaml functional web scraping library

Use SQL on various data sources

Code for the collection and analysis of the MTNT dataset

PHP Scraper - an highly opinionated web-interface for PHP

Creating Scrapy scrapers via the Django admin interface

Elegant Scraper and Crawler Framework for Golang

Public, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

This class can retrieve search results from Google.

HTML scraping for Objective-C.

📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.

✂️ High performance, multi-threaded image scraper

Transistor, a Python web scraping framework for intelligent use cases.

Free Web Scraping Tool with Java

Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!

Tabula is a tool for liberating data tables trapped inside PDF files

📄 Python tool to turn Notion.so pages into lightweight, customizable static websites

🥫 The simple, fast, and modern web scraping library

scan for webcams on the internet

Analyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, contacts, friends added statistics and more

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)

Distributed crawler, database and web frontend for public directories indexing

Declarative web scraping

TV Series is a tool that scrapes Episode Synopsis' of popular TV Series' from websites like Wikipedia / IMDb and show in one place with a user-friendly navigation UI.

Extract structured data from web sites. Web sites scraping.

Simple scriptes for Level UP your scraping Skills, and source code for Level UP playlist on Youtube

Jekyll-based static site for The Programming Historian

Image Dataset Tool (idt) is a cli tool designed to make the otherwise repetitive and slow task of creating image datasets into a fast and intuitive process.

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Scrapy, a fast high-level web crawling & scraping framework for Python.

COVID-19 Coronavirus data scraped from government and curated data sources.

Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.

Laravel package to collect your bank statements history. Currently support for parsing statements history from BCA, Mandiri, BNI, and MUAMALAT e-banking websites.

Social media scraping / data collection library for Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

simple multi-level scraper json input/output for Cheerio

🧶 Extract data from any website without code, just clicks.

Python scraper for Language Pods such as Japanesepod101.com 👹 🗾 🍣 Compatible with Japanese, Chinese, French, German, Italian, Korean, Portuguese, Russian, Spanish and many more! ✨

Scrapy Extension for monitoring spiders execution.

A Telegram bot combined with python to serve some basic functions like weather, music charts, cricket score and much more.

Scrape any website, article or RSS/Atom Feed with ease!

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

一个灵活、友好的爬虫框架

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

Crawlera middleware for Scrapy

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

reborn of https://bitbucket.org/rflechner/scrapysharp

Example end to end data engineering project.

Tiny little ruby on rails website that crawls though your public github repos to find out what your favourite languages are.

Generate a network graph of connections from your WhatsApp groups data

List of major web + mobile browser user agent strings. +1 Bonus script to scrape :)

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Pythonic HTML Parsing for Humans™

Data extraction of Google's COVID-19 Mobility Reports