435 Open source projects that are alternatives of or similar to memes-api

Twitter bot powering @arichduvet

Docker microservice & Crawler by scrapy

Scraping the status of WhatsApp contacts

Web crawling and scraping framework for Golang

🕸 List of mini projects that involve web scraping 🕸

免费 IP 代理池。Scrapy 爬虫框架插件

A collection of pipelines for Scrapy

🎊 Design and implement of lightweight crawler framework.

Easily download the main content of a web page in html, markdown, and/or epub format from command line.

A middleware of cookies persistence for Scrapy

Parse a human name string into salutation, first name, middle name, last name, suffix.

一些简单有趣的Python小Demo

HTML parsing and processing tool for PowerShell.

Main API Flight Git Repository

Scape top GitHub repositories and users based on keywords

Scrape and take screenshots of dynamic and static webpages

Archive web pages with all relevant assets or save as a single file HTML

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

一个基于Scrapy的数据采集爬虫代码库

A PHP Scraping Class

知名網站avgle下載器

A Web Crawler that scrapes using YAML and python code.

基于Scrapy的爬虫demo

Scrapy项目（mysql+mongodb豆瓣top250电影）

秀人网爬虫 55156爬虫

ProxyCrawl Python library for scraping and crawling

Crawl sites for RSS, Atom, and JSON feeds.

A web search engine built with Python which uses TF-IDF and PageRank to sort search results.

A simple API for scraping Overwatch stats

爬虫学习仓库，适合零基础的人学习，对新手比较友好

scrapy 常用爬网必备工具包

Using Scrapy to crawl Autohome, storage into MonogDB, simple analysis and NLP coming soon

No description or website provided.

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

Transform unstructured document collections to structured Linked Data

Kotlin Wrapper for Jsoup

人生苦短 我用Python

extract data from html table

Personal Data Engineering Projects

🕵‍♂ Bot detection tests for Puppeteer. Hide and seek!

youtube & tiktok analysis + youchoose recommendation custmizer. backend, extensions, and tooling

🌍 Dataset that shows the Internet affordability by country (a shocking reality!)

🔎Search LinkedIn profile by email address📧

No description or website provided.

Python分布式爬虫打造搜索引擎

PTT 網路版爬蟲

爬虫管理平台

A GUI for Instaloader to scrape users and hashtags with on Instagram

Scraping Ebay's products using Scrapy Web Crawling Framework

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

Easily crawl news portals or blog sites using Storm Crawler.

🦀 Scraping & crawling websites using Rust, and ReasonML

A POSIX shell script to parse HTML

Bangumi小工具

👍 京东爬虫（大量注释，对刚入门爬虫者极度友好）

Download images from reddit

古汉语(文言文)字典-爬取文言文字典网,制作Kindle字典.

The simplest demo of chrome automation by python and selenium in AWS Lambda