All Projects → nielsing → Yar

nielsing / Yar

Licence: gpl-3.0
Yar is a tool for plunderin' organizations, users and/or repositories.

Programming Languages

go
31211 projects - #10 most used programming language
golang
3204 projects

Projects that are alternatives of or similar to Yar

Spiderfoot
SpiderFoot automates OSINT for threat intelligence and mapping your attack surface.
Stars: ✭ 6,882 (+3855.17%)
Mutual labels:  osint, infosec, reconnaissance
Rengine
reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface. reNgine makes it easy for penetration testers to gather reconnaissance with…
Stars: ✭ 3,439 (+1876.44%)
Mutual labels:  osint, reconnaissance, infosec
Spaces Finder
A tool to hunt for publicly accessible DigitalOcean Spaces
Stars: ✭ 122 (-29.89%)
Mutual labels:  osint, infosec, reconnaissance
Argos
This script will automatically set up an OSINT workstation starting from a Ubuntu OS.
Stars: ✭ 73 (-58.05%)
Mutual labels:  osint, infosec, reconnaissance
Hosthunter
HostHunter a recon tool for discovering hostnames using OSINT techniques.
Stars: ✭ 427 (+145.4%)
Mutual labels:  osint, reconnaissance, tool
aquatone
A Tool for Domain Flyovers
Stars: ✭ 43 (-75.29%)
Mutual labels:  osint, infosec, reconnaissance
flydns
Related subdomains finder
Stars: ✭ 29 (-83.33%)
Mutual labels:  osint, infosec, reconnaissance
Git Hound
Reconnaissance tool for GitHub code search. Finds exposed API keys using pattern matching, commit history searching, and a unique result scoring system.
Stars: ✭ 602 (+245.98%)
Mutual labels:  osint, reconnaissance, secrets
Awesome Asset Discovery
List of Awesome Asset Discovery Resources
Stars: ✭ 1,017 (+484.48%)
Mutual labels:  osint, infosec, reconnaissance
Sarenka
OSINT tool - gets data from services like shodan, censys etc. in one app
Stars: ✭ 120 (-31.03%)
Mutual labels:  osint, reconnaissance
Search4
Search people on the Internet.
Stars: ✭ 124 (-28.74%)
Mutual labels:  osint, tool
Chatter
internet monitoring osint telegram bot for windows
Stars: ✭ 123 (-29.31%)
Mutual labels:  osint, infosec
Analyst Arsenal
A toolkit for Security Researchers
Stars: ✭ 112 (-35.63%)
Mutual labels:  osint, infosec
D4n155
OWASP D4N155 - Intelligent and dynamic wordlist using OSINT
Stars: ✭ 105 (-39.66%)
Mutual labels:  osint, tool
Tidos Framework
The Offensive Manual Web Application Penetration Testing Framework.
Stars: ✭ 1,290 (+641.38%)
Mutual labels:  osint, reconnaissance
Autosetup
Auto setup is a bash script compatible with Debian based distributions to install and setup necessary programs.
Stars: ✭ 140 (-19.54%)
Mutual labels:  infosec, reconnaissance
Breach.tw
A service that can track data breaches like "Have I Been Pwned", but it is specific for Taiwan.
Stars: ✭ 144 (-17.24%)
Mutual labels:  osint, infosec
Geo Recon
An OSINT CLI tool desgined to fast track IP Reputation and Geo-locaton look up for Security Analysts.
Stars: ✭ 82 (-52.87%)
Mutual labels:  osint, reconnaissance
Asnip
ASN target organization IP range attack surface mapping for reconnaissance, fast and lightweight
Stars: ✭ 126 (-27.59%)
Mutual labels:  osint, reconnaissance
Raccoon
A high performance offensive security tool for reconnaissance and vulnerability scanning
Stars: ✭ 2,312 (+1228.74%)
Mutual labels:  osint, reconnaissance

(Y)et (A)nother (R)obber: Sail ye seas of git for booty is to be found

Yar the pirate gopher

Sail ho! Yar is a tool for plunderin' organizations, users and/or repositories...

In all seriousness though, yar is an OSINT tool for reconnaissance of repositories/users/organizations on Github. Yar clones repositories of users/organizations given to it and goes through the whole commit history in order of commit time, in search for secrets/tokens/passwords, essentially anything that shouldn't be there. Whenever yar finds a secret, it will print it out for you to further assess.

Yar searches for secrets either by regex, entropy or both, the choice is yours! Inspired by other git secret grabbers.

Installation

  1. Make sure you have the GOPATH environment variable set in your preferred shell rc and that the $GOPATH/bin directory is in your PATH. More info here.
  2. You can install this by running go get github.com/nielsing/yar
  3. Or you can download the latest release of Yar for your operating system here. Just make sure you have the yarconfig.json file as well in $GOPATH/src/github.com/nielsing/yar/config.

Usage

Want to search for secrets within an organization?

yar -o orgname

You can also include the members of the organization with:

yar -o orgname --include-members

Want to search for secrets within a users repositories?

yar -u username

Want to search for secrets within a single repository?

yar -r https://github.com/User/Repo

or if you have already cloned the repository

yar -r /path/to/.git/folder

Want to search for secrets within an organization, a user and a repository?

yar -o orgname -u username -r https://github.com/User/Repo

Getting too much/not enough noise?

All rules are marked with a noise level from 0 to 9. Noise levels from 0 to 4 are considered secrets while noise levels from 5 to 9 are considered reconnaissance info (emails, IPs, etc...). You can decide which noise levels yar searches for, the default is to (and including 3).

Search for all secrets with noise level 4 or less

yar -r https://github.com/User/Repo -n -4

Search for all secrets with noise level 6 or more

yar -r https://github.com/User/Repo -n 6-

Search for all secrets from 1 to (and including) 3.

yar -r https://github.com/User/Repo -n 1-3

Search for all secrets with noise level exactly 7

yar -r https://github.com/User/Repo -n 7

Search for all secrets with any noise level

yar -r https://github.com/User/Repo -n -

Have your own predefined rules?

Rules are stored in a JSON file with the following format:

{
    "Rules": [
        {
            "Reason": "The reason for the match",
            "Rule": "The regex rule",
            "Noise": 3
        },
        {
            "Reason": "Super secret token",
            "Rule": "^Token: .*$",
            "Noise": 2
        }
    ]
    "FileBlacklist": [
        "Regex rule here"
        "^.*\\.lock"
    ]
}

You can then load your own rule set with the following command:

yar -u username --rules PATH_TO_JSON_FILE

If you already have a truffleHog config and want to port it over to a yar config there is a script in the config folder that does it for you. Simply run python3 trufflestoconfig.py PATH_TO_TRUFFLEHOG_CONFIG and the script will give you a file named yarconfig.json.

Don't like regex?

yar -u username --entropy

Want the best of both worlds?

yar -u username --both

Want to search as an authenticated user?

Add your github token to your environment variables.

export YAR_GITHUB_TOKEN=YOUR_TOKEN_HERE

Want to save your findings to a JSON file for later analysis?

yar -o orgname --save

Don't like the default colors and want to add your own color settings?

It is possible to customize the colors of the output for Yar through environment variables. The possible colors to choose from are the following:

black
blue
cyan
green
magenta
red
white
yellow
hiBlack
hiBlue
hiCyan
hiGreen
hiMagenta
hiRed
hiWhite
hiYellow

Each color can then be suffixed with bold, i.e. blue bold to make the letters bold.

This is done through the following env variables:

YAR_COLOR_VERBOSE -> Color of verbose lines.
YAR_COLOR_SECRET  -> Color of the highlighted secret.
YAR_COLOR_INFO    -> Color of info, that is, simple strings that tell you something.
YAR_COLOR_DATA    -> Color of data, i.e. commit message, reason, etc.
YAR_COLOR_SUCC    -> Color of succesful messages.
YAR_COLOR_WARN    -> Color of warnings.
YAR_COLOR_FAIL    -> Color of fatal warnings.

Like so export YAR_COLOR_SECRET="hiRed bold".

Extra Knowledge

There are some design decisions which might be good to know about. Yar saves all cloned github repos in a folder named yar within the temp directory. Yar then tries to load github repos from this cache by default, if you don't want to load from cache then you can add the --no-cache flag.

Yar also clones bare repos by default, if you want to get all files within a repo and not just the metadata then you can add the --no-bare flag.

If you want to remove repos from cache then you can use the --cleanup flag. This flag either removes the whole cache if no folder was specified or just removes the specified folder. The folder structure within the cache folder is like so:

/yar
|--- /User1
|  |--- /Repo1
|  |--- /Repo2
|
|--- /User2
|  |--- /Repo1
|  |--- /Repo2

So you can run --cleanup User1 to remove the cache of User1 or --cleanup User1/Repo1 to clean up Repo1 of User1. You can think of the flag as a wrapper around rm -r /tmp/yar/{USER_INPUT}.

Finally yar goes 10000 commits deep by default and goes through them in order of time (oldest to newest). This depth is configurable so if you ever want to cover more or fewer commits simply add the --depth flag with the depth you want.

Help

usage: yar [-h|--help] [-o|--org "<value>"] [-u|--user "<value>"] [-r|--repo
           "<value>"] [-c|--context <integer>] [-e|--entropy] [-b|--both]
           [-f|--forks] [-n|--noise "<value>"] [-d|--depth <integer>]
           [-C|--config <file>] [--no-bare] [--no-cache] [--no-context]
           [--include-members] [--skip-duplicates] [--cleanup "<value>"]
           [-s|--save "<value>"]

           Sail ye seas of git for booty is to be found

Arguments:

  -h  --help             Print help information
  -o  --org              Organization to plunder
  -u  --user             User to plunder
  -r  --repo             Repository to plunder
  -c  --context          Show N number of lines for context. Default: 2
  -e  --entropy          Search for secrets using entropy analysis. Default:
                         false
  -b  --both             Search by using both regex and entropy analysis.
                         Overrides entropy flag. Default: false
  -f  --forks            Specifies whether forked repos are included or not.
                         Default: false
  -n  --noise            Specify the range of the noise for rules. Can be
                         specified as up to (and including) a certain value
                         (-4), from a certain value (5-), between two values
                         (3-5), just a single value (4) or the whole range (-).
                         Default: -3
  -d  --depth            Specify the depth limit of commits fetched when
                         cloning. Default: 10000
  -C  --config           JSON file containing yar config.
      --no-bare          Clone the whole repository. Default: false
      --no-cache         Don't load from cache. Default: false
      --no-context       Only show the secret itself, similar to trufflehog's
                         regex output. Overrides context flag. Default: false
      --include-members  Include an organization's members for plunderin'.
                         Default: false
      --skip-duplicates  Skip duplicate secrets within repositories. Default:
                         false
      --cleanup          Remove specified cloned directory within yar cache
                         folder. Leave blank to remove the cache folder
                         completely.
  -s  --save             Yar will save all findings to a specified file.
                         Default: findings.json

Acknowledgements

It is important to point out that this idea is inspired by the infamous truffleHog tool and the code used for entropy searching is in fact borrowed from the truffleHog repository which in turn is borrowed from this blog post.

This project wouldn't have been possible without the following libraries:

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].