682 Open source projects that are alternatives of or similar to Yispider

那些年，我爬过的北科。一个由浅入深的定向爬虫教程。

As you can see, a kuaishou crawler

File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

Web Crawler written in C#

🐨实时监控1000家中国企业的新闻动态

Golang pkg to quickly return a preview of a webpage (title/description/images)

Declarative web scraping

Sample of using proxies to crawl baidu search results.

A framework for creating semi-automatic web content extractors

Some scrapy and web.py exmaples

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

news-please - an integrated web crawler and information extractor for news that just works.

大数据生态解决方案数据平台：基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。

抖音爬虫. 通过手机代理爬取用户的作品和用户的喜欢

java framework for prerender

An easy to use, powerful crawler implemented in PHP. Can execute Javascript.

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

🔥 Shadowsocks 账号爬虫

feapder是一款支持分布式、批次采集、任务防丢、报警丰富的python爬虫框架

The Resources for "Natural Language to Logical Form" ; "自然语言转逻辑形式"研究资料收集。

Crawler for leboncoin.fr

대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.

爬取菜鸟教程网站并转PDF__python_crawer_by_chrome

A crawler for automated functional testing of a web application

Script that crawls meta data from ICLR OpenReview webpage. Tutorials on installing and using Selenium and ChromeDriver on Ubuntu.

Python爬虫和Flask实现小说网站

今日热榜，一个获取各大热门网站热门头条的聚合网站，使用Go语言编写，多协程异步快速抓取信息，预览:https://mo.fish

优雅地玩知乎

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说、招标网、采购网、小红书》

👩 美女写真套图爬虫（一）

🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi

扒网站工具，看好哪个网站，指定好URL，自动扒下来做成模版。所见网站，皆可为我所用！

小说精品屋-plus是一个多端（PC、WAP）阅读、功能完善的原创文学CMS系统，由前台门户系统、作家后台管理系统、平台后台管理系统、爬虫管理系统等多个子系统构成，支持多模版、会员充值、订阅模式、新闻发布和实时统计报表等功能，新书自动入库，老书自动更新。

This repository is no longer maintained.

🐞简单轻便的Java爬虫框架，只要会一点简单的正则表达式和简单的css选择器就能轻松的采集数据。

Easily download all the photos/videos from tumblr blogs. 下载指定的 Tumblr 博客中的图片，视频

Python爬虫，返回一定格式的信息，下载，使用flask提供简易api。抖音无水印、皮皮虾、快手、网易云音乐、qq音乐、咪咕音乐、荔枝FM音频、知乎视频、最右语音、视频、微博......

整合第三方网站的视频和弹幕资源, 为白嫖党提供最佳看番追剧体验

Google, Naver multiprocess image web crawler (Selenium)

Python多线程下载 草榴(t66y.com) 网站【新時代的我們】和【達蓋爾的旗幟】两个板块帖子内的图片

Dynamic meta tags in your AngularJS single page application

用严肃的数据来回答“什么样的企业会到什么样的大学招聘”？

Multithreading download all HD photos / pictures from someone's Sina Weibo album.

A dungeon crawler

头条号爬虫案例

Testing Using Python Demo. 使用Python测试脚本demo。

抖音API、抖音数据、抖音直播数据、抖音直播Api、抖音视频Api、抖音爬虫、抖音去水印、抖音视频下载、抖音视频解析、抖音直播监控、抖音数据采集

Crawler (Bot) searching for credential leaks on different paste sites.

Your web font utility belt. It can subset web fonts. It can find unicode-ranges for you automatically. It makes julienne fries.

A configurable web spider with a easy-to-use web console

一个简单的分布式爬虫框架

JSpider会每周更新至少一个网站的JS解密方式，欢迎 Star，交流微信：13298307816

Crawler used to crawl papers

一个浏览器端数据爬虫，做每个人的数据助手

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Scrapy as a Web Crawler for Azure Search Samples

A Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.

Tor website crawler (specific for Alphabay at the time)