545 Open source projects that are alternatives of or similar to Scrapy Training

📈 Regression and Classification with UC Davis student quiz data and exam data

Scrapy project for scraping data from IMDB with Movie Dataset including 58,623 movies' data.

Random User-Agent middleware based on fake-useragent

Scraping Ebay's products using Scrapy Web Crawling Framework

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Faster requests on Python 3

Build a Flask web application to help users retrieve key restaurant information and feature-based reviews (generated by applying market-basket model – Apriori algorithm and NLP on user reviews).

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

A framework for creating semi-automatic web content extractors

Scrape, standardize and share public meetings from local government websites

Netflix like full-stack application with SPA client and backend implemented in service oriented architecture

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

Scrapy crawler to collect data on the back catalog of songs listed for sale.

Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSON-LD metadata, analyzes AJAX requests, etc.

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

Container, Monitoring & Logging, Cloud & DevOps Tutorials and Labs

🏃 Some of the python tutorial - 《Python学习笔记》

Samples associated with Pluralsight design patterns in c# courses.

This is the repository for my course, Building a Website with Node.js and Express.js on LinkedIn Learning and Lynda.com.

可视化爬虫自动采集平台

MongoDB for Python developers course handouts from Talk Python Training

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

Damn Vulnerable Web Application Docker container

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

A collection of awesome web scaper, crawler.

INGInious is a secure and automated exercises assessment platform using your own tests, also providing a pluggable interface with your existing LMS.

⚡️ Ayakashi.io - The next generation web scraping framework

react + flask + scrapy 构建的单页应用漫画网站

This repository provides you with a easy to use labeling tool for State-of-the-art Deep Learning training purposes.

[cvpr19] Demo, training and evaluation code for generating dense hand+object reconstructions from single rgb images

基于微博的数据挖掘与社交舆情分析

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

scrapy框架爬取51job(scrapy.Spider)，智联招聘(扒接口)，拉勾网(CrawlSpider)

基于Scrapy的QQ音乐爬虫(QQ Music Spider)，爬取歌曲信息、歌词、精彩评论等，并且分享了QQ音乐中排名前6400名的内地和港台歌手的49万+的音乐语料

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

Learn Python for the next 30 (or so) Days.

Repository for my tutorial course: Bootstrap 4 Essential Training on LinkedIn Learning and Lynda.com.

Scala for Statistical Computing and Data Science Short Course

PHP Scraper - an highly opinionated web-interface for PHP

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Zillow Scraper for Python using Selenium

用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, SHFE and news data crawlers on BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

a spider for cnki patent content, just for study and commucation, no use for business.

pigat ( Passive Intelligence Gathering Aggregation Tool ) 被动信息收集聚合工具

Backend para coleta e disponibilização dos dados 📜

General Assembly's 2015 Data Science course in Washington, DC

SciPy 2017 Pandas Tutorial

Learning Management System OpenOlat

Android app for saving webpages for offline reading.

scrapy专利爬虫（停止维护）

Kubernetes course on Udemy from @BretFisher and @jpetazzo

微博爬虫：每天定时爬取微博热搜榜的内容，留下互联网人的记忆。

Unmaintained 🐳 ☕️ 🕷 Scala crawler(spider) framework, inspired by scrapy, created by @gaocegege

A Chrome extension for writing custom web scraping programs and web automation programs. Just demonstrate how to collect the first row of data, then let the extension write the program for collecting all rows.

Train machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.

Data Science in the tidyverse, a two-day workshop @ rstudio:conf(2018)

This is the repository for my course, Building a Web Interface with React.js on LinkedIn Learning and Lynda.com.