134 Open source projects that are alternatives of or similar to Docbao

Headless 'Chrome' Orchestration in R

Node.js framework for modular web scraping and data extraction

General Assembly's 2015 Data Science course in Washington, DC

Extract clean(er), readable text from web pages via Mercury Web Parser.

A JavaScript library for generating random user agents with data that's updated daily.

MetaData html scraper and parser for Node.js (supports Promises and callback style)

A Router WiFi key recovery/cracking tool with a twist.

Scrapy Training companion code

Faster requests on Python 3

Turn large Web sites into tables and charts using simple SQLs.

Simple web scraping for R

Python binding to Modest engine (fast HTML5 parser with CSS selectors).

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

Powerful web scraping framework for Crystal

Web scrapping and related analytics using Python tools

Download/access photos, videos, stories, story highlights, postlives, following and followers of Instagram

Learn Python for the next 30 (or so) Days.

📈 Regression and Classification with UC Davis student quiz data and exam data

Tutorial: Web scraping in Python with Beautiful Soup

Collection of scripts corresponding to LucidProgramming YouTube tutorials

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

The Python Code Tutorials

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, SHFE and news data crawlers on BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

A framework for creating semi-automatic web content extractors

💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

PHP Scraper - an highly opinionated web-interface for PHP

PHP Library for detecting CMS

Web Scraping Framework

Receive an email or Telegram message as soon as Migros Sanalmarket is available for delivery in your neighborhood.

Zillow Scraper for Python using Selenium

Go cascadia package command line CSS selector

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

Fetch user's data across social media

Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSON-LD metadata, analyzes AJAX requests, etc.

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

This repository provides everything you need to get started with Python for (social science) research.

Apify actor that crawls Google Search result pages (SERPs) and extracts a list of organic results, ads, related queries and more. It supports selection of custom country, language and location.

⚡️ Ayakashi.io - The next generation web scraping framework

Snoop — инструмент разведки на основе открытых данных (OSINT world)

Selenium-python but lighter: Helium is the best Python library for web automation.

Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username

Android app for saving webpages for offline reading.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Netflix like full-stack application with SPA client and backend implemented in service oriented architecture

Nextjs server to query websites with GraphQL

A Devtools driver for web automation and scraping

Random User-Agent middleware based on fake-useragent

Machine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)

UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++

Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

List of libraries, tools and APIs for web scraping and data processing.

A Chrome extension for writing custom web scraping programs and web automation programs. Just demonstrate how to collect the first row of data, then let the extension write the program for collecting all rows.

Python Package for scraping NHL Play-by-Play and Shift data

Scrape, standardize and share public meetings from local government websites

Python scripts for building 'Short Jokes' dataset, featured on Kaggle

Twitter Intelligence OSINT project performs tracking and analysis of the Twitter

Scrapy crawler to collect data on the back catalog of songs listed for sale.

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges