194 Open source projects that are alternatives of or similar to crawlzone

Scrape, standardize and share public meetings from local government websites

The Python Code Tutorials

MetaData html scraper and parser for Node.js (supports Promises and callback style)

A JavaScript library for generating random user agents with data that's updated daily.

Building a Concurrent Web Scraper with Python and Selenium

A framework for creating semi-automatic web content extractors

Learn Python for the next 30 (or so) Days.

Python binding to Modest engine (fast HTML5 parser with CSS selectors).

Python scripts for building 'Short Jokes' dataset, featured on Kaggle

ACHE is a web crawler for domain-specific search.

General Assembly's 2015 Data Science course in Washington, DC

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Kattis demo solutions with unit testing

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

[Python] Scraps comics and manga from various websites and creates cbz files from them

Tutorial: Web scraping in Python with Beautiful Soup

Explaining textual analysis tools in Python. Including Preprocessing, Skip Gram (word2vec), and Topic Modelling.

Turn large Web sites into tables and charts using simple SQLs.

⏪ Tools to Work with the Various Internet Archive Wayback Machine APIs

Node.js framework for modular web scraping and data extraction

📖 Using deep learning and scraping to analyze/summarize articles! Just drop in any URL!

💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R

A web scraping tool to systematically extract the text of scientific papers and corresponding metadata from university accessible journals.

Twitter Intelligence OSINT project performs tracking and analysis of the Twitter

A Docker tutorial using a link extraction application example

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

🇺🇸 Is the US flag at half-staff?

Crawling and scraping the Web for fun and profit

Materials to reproduce our findings in our stories, "Amazon Puts Its Own 'Brands' First Above Better-Rated Products" and "When Amazon Takes the Buy Box, it Doesn’t Give it up"

Simple web scraping for R

House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.

Web scrapping and related analytics using Python tools

Heroshi – open source web crawler.

Extract clean(er), readable text from web pages via Mercury Web Parser.

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz

An unofficial API for Quora.

Text Mining for Indonesian Online News Articles About Corona

Powerful web scraping framework for Crystal

WhatsApps related deaths News Articles along with other articles across India during that period

Scrapy Training companion code

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Headless 'Chrome' Orchestration in R

This repository contains very useful restful API's and functionalities in node-js containing many important tutorial code for mastering node-js, all tutorials have been published on medium.com, tutorials link is given below

A Programming language for Web Scraping

(DISCONTINUED) Save WhatsApp Web Sessions as files and open them everywhere!

Download/access photos, videos, stories, story highlights, postlives, following and followers of Instagram

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, SHFE and news data crawlers on BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

AI based web-wrapper for web-content-extraction

A Router WiFi key recovery/cracking tool with a twist.

🕸🧰☕️Tools to Scrape Dynamic Web Content via the 'HtmlUnit' Java Library

A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

📈 Regression and Classification with UC Davis student quiz data and exam data

Exercises, data, and more for our 2017 summer workshop (funded by the Estes Fund and in partnership with Project Jupyter and Berkeley's D-Lab)

C# command line tool for running tests on Android / iOS / tvOS devices and simulators

Powerful Framework for Backend Automation Testing on Java (Rest, Soap, WebSocket)

Công cụ quét và phân tích từ khoá các trang báo mạng Việt Nam

PHP Scraper - an highly opinionated web-interface for PHP

Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username