All Projects → adanvillarreal → Spydan

adanvillarreal / Spydan

Licence: MIT license
A web spider for shodan.io without using the Developer API.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Spydan

Gosint
OSINT Swiss Army Knife
Stars: ✭ 401 (+1236.67%)
Mutual labels:  scraper, shodan, spider
Linkedin Profile Scraper
🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+470%)
Mutual labels:  scraper, spider
Not Your Average Web Crawler
A web crawler (for bug hunting) that gathers more than you can imagine.
Stars: ✭ 107 (+256.67%)
Mutual labels:  scraper, spider
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+51683.33%)
Mutual labels:  scraper, spider
aliexscrape
Get Aliexpress product details in JSON
Stars: ✭ 80 (+166.67%)
Mutual labels:  scraper, spider
Geziyor
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+4053.33%)
Mutual labels:  scraper, spider
Querylist
🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+7873.33%)
Mutual labels:  scraper, spider
Mailinglistscraper
A python web scraper for public email lists.
Stars: ✭ 19 (-36.67%)
Mutual labels:  scraper, spider
ant
A web crawler for Go
Stars: ✭ 264 (+780%)
Mutual labels:  scraper, spider
blinkist-m4a-downloader
Grabs all of the audio files from all of the Blinkist books
Stars: ✭ 100 (+233.33%)
Mutual labels:  scraper, spider
scraper
图片爬取下载工具,极速爬取下载 站酷https://www.zcool.com.cn/, CNU 视觉 http://www.cnu.cc/ 设计师/用户 上传的 图片/照片/插画。
Stars: ✭ 64 (+113.33%)
Mutual labels:  scraper, spider
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+3313.33%)
Mutual labels:  scraper, spider
Avbook
AV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+27010%)
Mutual labels:  scraper, spider
OpenScraper
An open source webapp for scraping: towards a public service for webscraping
Stars: ✭ 80 (+166.67%)
Mutual labels:  scraper, spider
Scrapit
Scraping scripts for various websites.
Stars: ✭ 25 (-16.67%)
Mutual labels:  scraper, spider
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (+533.33%)
Mutual labels:  scraper, spider
robotstxt
robots.txt file parsing and checking for R
Stars: ✭ 65 (+116.67%)
Mutual labels:  scraper, spider
Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+2086.67%)
Mutual labels:  scraper, spider
Crawler
A high performance web crawler in Elixir.
Stars: ✭ 781 (+2503.33%)
Mutual labels:  scraper, spider
crawler-chrome-extensions
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (+76.67%)
Mutual labels:  scraper, spider

Spydan

A web spider for shodan.io

What can you do?

Spydan allows you to retrieve information from shodan.io without an API Key and export it as CSV, JSON or XML by creating queries with given parameters.

Due to the five page limit Shodan has for standard users, the results are limited to those shown in these first pages.

Getting Started

Prerequisites

Before you start crawling, you need to have Scrapy

pip install scrapy

Usage

You only need to worry about the file spydan.py. You will need to add some arguments when running Spydan. The required ones are username, password, and networks. Other arguments are ports, services, and products.

A full example is:

python spydan.py adanvillarreal p455w0rd 131.178.0.0/16 148.241.128.0/18 --ports 80 8080 8084 8484 --fname prueba --type json

You can get more information about arguments by executing

python spydan.py -h

Tweaking

There is a delay between requests of 3 seconds and a limit for 2 concurrent requests due to 503 codes that I received with higher values, which are located in settings.py under the directory spydan.

Author

Adan Villarreal

License

This project is licensed under the MIT License - see the LICENSE file for details

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].