GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → gsh199449 → Spider

gsh199449 / Spider

Licence: gpl-3.0
A configurable web spider with a easy-to-use web console

Programming Languages

java
68154 projects - #9 most used programming language

Labels

spidertext-mining

Projects that are alternatives of or similar to Spider

TwEater
A Python Bot for Scraping Conversations from Twitter
Stars: ✭ 16 (-98.32%)
Mutual labels:  text-mining, spider
Douban spider
一个简单的豆瓣信息爬虫😄
Stars: ✭ 8 (-99.16%)
Mutual labels:  spider
Rake Nltk
Python implementation of the Rapid Automatic Keyword Extraction algorithm using NLTK.
Stars: ✭ 793 (-16.88%)
Mutual labels:  text-mining
Text Mining
Text Mining in Python
Stars: ✭ 18 (-98.11%)
Mutual labels:  text-mining
Anti Anti Spider
越来越多的网站具有反爬虫特性，有的用图片隐藏关键数据，有的使用反人类的验证码，建立反反爬虫的代码仓库，通过与不同特性的网站做斗争（无恶意）提高技术。（欢迎提交难以采集的网站）（因工作原因，项目暂停）
Stars: ✭ 6,907 (+624%)
Mutual labels:  spider
Mailinglistscraper
A python web scraper for public email lists.
Stars: ✭ 19 (-98.01%)
Mutual labels:  spider
Gospider
Gospider - Fast web spider written in Go
Stars: ✭ 785 (-17.71%)
Mutual labels:  spider
Blackwidow
A Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.
Stars: ✭ 887 (-7.02%)
Mutual labels:  spider
Easylogin
A python3 package for writing spider more easily.
Stars: ✭ 26 (-97.27%)
Mutual labels:  spider
Bagofconcepts
Python implementation of bag-of-concepts
Stars: ✭ 18 (-98.11%)
Mutual labels:  text-mining
Javlibrary
Javlibrary spider
Stars: ✭ 17 (-98.22%)
Mutual labels:  spider
Autophrase
AutoPhrase: Automated Phrase Mining from Massive Text Corpora
Stars: ✭ 835 (-12.47%)
Mutual labels:  text-mining
Scrapit
Scraping scripts for various websites.
Stars: ✭ 25 (-97.38%)
Mutual labels:  spider
Torbot
Dark Web OSINT Tool
Stars: ✭ 821 (-13.94%)
Mutual labels:  spider
Pholcus
Pholcus is a distributed high-concurrency crawler software written in pure golang
Stars: ✭ 6,990 (+632.7%)
Mutual labels:  spider
Nlp In Practice
Starter code to solve real world text data problems. Includes: Gensim Word2Vec, phrase embeddings, Text Classification with Logistic Regression, word count with pyspark, simple text preprocessing, pre-trained embeddings and more.
Stars: ✭ 790 (-17.19%)
Mutual labels:  text-mining
Zhihu Crawler
zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目
Stars: ✭ 890 (-6.71%)
Mutual labels:  spider
Baiduyunspider
百度云网盘搜索引擎，包含爬虫 & 网站
Stars: ✭ 903 (-5.35%)
Mutual labels:  spider
Jspider
JSpider会每周更新至少一个网站的JS解密方式，欢迎 Star，交流微信：13298307816
Stars: ✭ 914 (-4.19%)
Mutual labels:  spider
Go Demo
Go语言实例教程从入门到进阶，包括基础库使用、设计模式、面试易错点、工具类、对接第三方等
Stars: ✭ 881 (-7.65%)
Mutual labels:  spider
View All Similar Projects ➔

欢迎使用 Gather Platform 数据采集与分析平台

Readme in English

欢迎加入 Gather Platform交流 QQ群 : 206264662

详细使用方法请参考 在线文档

Build Status

Gather Platform 数据抓取平台是一套基于Webmagic内核的,具有Web任务配置和任务管理界面的数据采集与搜索平台.具有以下功能

  • 根据配置的模板进行数据采集，支持Ajax网页采集
  • 在不配置采集模板的情况下自动检测网页正文,自动抽取文章发布时间
  • 动态字段抽取与静态字段植入
  • 已抓取数据的管理,包括:搜索,增删改查,按照新的数据模板重新抽取数据
  • 对采集的数据进行NLP处理,包括:抽取关键词,抽取摘要,抽取实体词
  • 含有相关文章推荐，文章中人物、地点之间的关联关系分析

5分钟即可部署完毕,半分钟即可完成一个爬虫,开始数据采集. 不需要进行任何编码就可以完成一个功能强大的爬虫.

show

Windows/Mac/Linux 全平台支持

本系统需要如下依赖:

  • JDK 8 及以上
  • Tomcat 8.3 及以上

可选依赖组件:

 - Elasticsearch 5.0

部署、使用方法、二次开发手册、常见问题等全部迁移至在线文档

联系我

邮箱: [email protected]

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].