holgerd77 / Django Dynamic Scraper
Licence: bsd-3-clause
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Django Dynamic Scraper
scrapy facebooker
Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-97.85%)
Mutual labels: scraper, spider, scraping, scrapy
Mailinglistscraper
A python web scraper for public email lists.
Stars: ✭ 19 (-98.14%)
Mutual labels: spider, scraper, scrapy, webscraping
Funpyspidersearchengine
Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索
Stars: ✭ 782 (-23.63%)
Mutual labels: spider, scrapy, django
OpenScraper
An open source webapp for scraping: towards a public service for webscraping
Stars: ✭ 80 (-92.19%)
Mutual labels: scraper, spider, scrapy
Seeker
Seeker - another job board aggregator.
Stars: ✭ 16 (-98.44%)
Mutual labels: spider, scrapy, django
crawler-chrome-extensions
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (-94.82%)
Mutual labels: scraper, spider, scraping
robotstxt
robots.txt file parsing and checking for R
Stars: ✭ 65 (-93.65%)
Mutual labels: scraper, spider, webscraping
scrapy-distributed
A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
Stars: ✭ 38 (-96.29%)
Mutual labels: spider, scraping, scrapy
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-81.45%)
Mutual labels: spider, scraper, scrapy
Linkedin
Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (-69.82%)
Mutual labels: scraper, scrapy, scraping
ARGUS
ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9
Stars: ✭ 68 (-93.36%)
Mutual labels: scraping, scrapy, webscraping
Autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+298.14%)
Mutual labels: scraper, scraping, webscraping
Gerapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
Stars: ✭ 2,601 (+154%)
Mutual labels: spider, scrapy, django
Py Elasticsearch Django
基于python语言开发的千万级别搜索引擎
Stars: ✭ 207 (-79.79%)
Mutual labels: spider, scrapy, django
wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Stars: ✭ 52 (-94.92%)
Mutual labels: scraper, spider, scraping
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+1417.09%)
Mutual labels: spider, scraper, scraping
Copybook
用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站
Stars: ✭ 117 (-88.57%)
Mutual labels: spider, scrapy, django
Linkedin Profile Scraper
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-83.3%)
Mutual labels: spider, scraper, scraping
Crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (-57.03%)
Mutual labels: spider, scraper, scraping
django-dynamic-scraper
Django Dynamic Scraper (DDS) is an app for Django which builds on top of the scraping framework Scrapy and lets you create and manage Scrapy spiders via the Django admin interface. It was originally developed for german webtv program site http://fernsehsuche.de.
Documentation
Read more about DDS in the ReadTheDocs documentation:
Getting Help/Updates
There is a mailing list on Google Groups, feel free to ask questions or make suggestions:
Infos about new releases and updates on Twitter:
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].