All Projects → bttown → torrent-spider

bttown / torrent-spider

Licence: MIT license
基于DHT的p2p网络资源爬虫

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to torrent-spider

Bt
BitTorrent library and client with DHT, magnet links, encryption and more
Stars: ✭ 2,011 (+2993.85%)
Mutual labels:  torrent, bittorrent, p2p, dht, magnet-link, magnet
Aria2.sh
Aria2 一键安装管理脚本 增强版
Stars: ✭ 1,276 (+1863.08%)
Mutual labels:  torrent, bittorrent, download, magnet-link, magnet
Snail
基于Java、JavaFX开发的下载工具,支持下载协议:BT(BitTorrent、磁力链接、种子文件)、HLS(M3U8)、FTP、HTTP。人家才不要你的⭐⭐呢,哼
Stars: ✭ 102 (+56.92%)
Mutual labels:  torrent, bittorrent, p2p, dht, magnet
Torrent Discovery
Discover BitTorrent and WebTorrent peers
Stars: ✭ 177 (+172.31%)
Mutual labels:  torrent, bittorrent, p2p, dht
bthello
Python3 DHT 磁力种子爬虫 种子解析 种子搜索 演示地址
Stars: ✭ 43 (-33.85%)
Mutual labels:  torrent, bittorrent, dht, magnet
Bt Btt
磁力網站U3C3介紹以及域名更新
Stars: ✭ 261 (+301.54%)
Mutual labels:  bittorrent, download, magnet-link, magnet
Torrent
Full-featured BitTorrent client package and utilities
Stars: ✭ 4,138 (+6266.15%)
Mutual labels:  torrent, bittorrent, p2p, magnet-link
Rdcli
The simple way to download and unrestrict DDL files, torrents and magnets
Stars: ✭ 75 (+15.38%)
Mutual labels:  torrent, download, p2p, magnet
Bittorrent Dht
🕸 Simple, robust, BitTorrent DHT implementation
Stars: ✭ 1,004 (+1444.62%)
Mutual labels:  torrent, bittorrent, p2p, dht
Aria2 Pro Docker
Aria2 Pro | A perfect Aria2 Docker image | 更好用的 Aria2 Docker 容器镜像
Stars: ✭ 802 (+1133.85%)
Mutual labels:  bittorrent, download, magnet-link, magnet
Magnetx
资源搜索型软件 macOS OSX magnet
Stars: ✭ 1,819 (+2698.46%)
Mutual labels:  torrent, download, magnet-link, magnet
Dottorrent Gui
An advanced GUI torrent file creator with batch functionality, powered by PyQt and dottorrent
Stars: ✭ 175 (+169.23%)
Mutual labels:  torrent, bittorrent, p2p
Magnet Uri
Parse a magnet URI and return an object of keys/values
Stars: ✭ 183 (+181.54%)
Mutual labels:  torrent, bittorrent, magnet-link
Gmdb
GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)
Stars: ✭ 189 (+190.77%)
Mutual labels:  torrent, magnet-link, magnet
Dht
dht is used by anacrolix/torrent, and is intended for use as a library in other projects both torrent related and otherwise
Stars: ✭ 184 (+183.08%)
Mutual labels:  torrent, bittorrent, dht
Put.io Adder
OS X put.io client that acts as handler for magnet: links and .torrent files, and adds them to your put.io download queue
Stars: ✭ 172 (+164.62%)
Mutual labels:  torrent, bittorrent, magnet
Magnetissimo
Web application that indexes all popular torrent sites, and saves it to the local database.
Stars: ✭ 2,551 (+3824.62%)
Mutual labels:  torrent, bittorrent, p2p
torrent-hound
Search torrents from multiple websites via the CLI
Stars: ✭ 28 (-56.92%)
Mutual labels:  torrent, bittorrent, magnet-link
Transgui
🧲 A feature rich cross platform Transmission BitTorrent client. Faster and has more functionality than the built-in web GUI.
Stars: ✭ 2,488 (+3727.69%)
Mutual labels:  torrent, bittorrent, p2p
torrent-webseed-creator
Webseeded torrent creator using GitHub Actions
Stars: ✭ 54 (-16.92%)
Mutual labels:  torrent, bittorrent, download

magnetlink spider

a magnet-link spider in p2p. 一个磁力链接收集器,让你简单快速地收集DHT网络中其他节点下载资源的信息.

Install && Usage

go get -u github.com/bttown/torrent-spider
go build github.com/bttown/torrent-spider
./torrent-spider

Notice

  1. 需要运行在公网服务器上, 否则收集到种子的可能性很小(正好形成UDP打洞)
  2. 收集器刚启动的时候需要较长的时间(作者使用阿里云1核1G的机器测试大概需要1到2天:()来和DHT网络中的其他节点通信,当我们的节点被大量其他节点收录时,大量资源就会不请自来了 http://bttown.net/

snapshot

Code

package main

import (
	"fmt"
	"github.com/bttown/dht"
	"github.com/bttown/metadata"
	"log"
)

var (
	collectorQueriesBufferSize = 5000
	collectorMaxPendingQueries = 2000
)

var (
	// DHT 节点
	node = dht.NewNode(dht.OptionAddress("0.0.0.0:8662"))
	// 种子信息获取器
	collector = metadata.NewCollector(metadata.Options{
		QueriesBufferSize: collectorQueriesBufferSize,
		MaxPendingQueries: collectorMaxPendingQueries,
	})
)

func main() {
	// 新获取种子时调用的Hook
	collector.OnFinish(func(req metadata.Request, torrent metadata.Torrent) {
		magnetLink := fmt.Sprintf("magnet:?xt=urn:btih:%s", req.HashInfo)
		log.Println("[Metadata]", magnetLink, torrent.Info.Name)
	})
	defer collector.Close()

	// 当发现DHT网络中有人下载资源时,告知收集器去获取种子详细信息
	node.PeerHandler = func(ip string, port int, hashInfo, peerID string) {
		if err := collector.Get(&metadata.Request{
			IP:       ip,
			Port:     port,
			HashInfo: hashInfo,
			PeerID:   peerID,
		}); err != nil {
			panic(err)
		}

	}
	node.Serve()
}
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].