htrgouvea / uranus

Harpoon

An ecosystem of crawlers for detecting: leaks, sensitive data exposure and attempts exfiltration of data

Summary

⚠️ Warning: Harpoon is currently in development, you've been warned :) and please consider contributing!

This project is summarized in several crawlers that constitute a single ecosystem, that monitor certain channels such as: Github, Bing, Pastebin and iHaveBeenPwned? in order to perform data leak detection, exposed sensitive files and data exfiltration attempts.

How it works

• You can click here to see a diagram of the database and also a catalog on them.

Download and setup

  # Download
  $ git clone https://github.com/GouveaHeitor/uranus && cd uranus

  # Building and starting MariaDB Database
  $ docker build -t uranus-database ./rest-server/database/
  $ docker run -d -p 3306:3306 --name database -e MARIADB_ROOT_PASSWORD=mypassword uranus-database

  # Building and starting the REST API
  $ docker build -t uranus-rest-server ./rest-server.
  $ docker run -d -p 80:80 --name rest-server uranus-rest-server

  # Building all crawlers/workers containers
  $ docker build -t bing-crawler ./crawlers/bing/
  $ docker build -t email-notify ./workers/email-notify

  # Running all crawlers/workers containers
  $ docker run -d --name bing bing-crawler
  $ docker run -d --name email-notify email-notify

Contribution

• Your contributions and suggestions are heartily ♥ welcome. See here the contribution guidelines. Please, report bugs via issues page and for security issues, see here the security policy. (✿ ◕‿◕) This project follows the best practices defined by this style guide.

• If you are interested in providing financial support to this project, please visit: heitorgouvea.me/donate

License

• This work is licensed under MIT License.