GShark

Scan for sensitive information easily and effectively.

GShark

The project is based on golang with AdminLTE to build a management system to manage the Github search results. Github API is utilized to crawl the related results according to key words and some rules. It proves to be a proper way to detect the information related to your company.🚀🚀🚀 For a detailed introduction, you can refer here.

Features

• Support multi platform, including Gitlab, Github, Searchcode
• Support search keyword in Huawei app store
• Flexible rules
• Utilize gobuster to brute force subdomain
• Batch insert rules

Quick start

git clone https://github.com/madneal/gshark

go get ./...

go build main.go

# check the config
mv app-template.ini app.ini 

# start web service
./main web 

# start crawler
./main scan

Config

The configuration can be set according to app-template.ini. You should rename it to app.ini to config the project.

HTTP_HOST = 127.0.0.1
HTTP_PORT = 8000
MAX_INDEXERS = 2
DEBUG_MODE = true
REPO_PATH = repos
MAX_Concurrency_REPOS = 5

; server酱配置口令
SCKEY =
; gobuster file path
gobuster_path =
; gobuster subdomain wordlist file path
subdomain_wordlist_file =

[database]
;support sqlite3, mysql, postgres
DB_TYPE = sqlite
HOST = 127.0.0.1
PORT = 3306
NAME = misec
USER = root
PASSWD = 
SSL_MODE = disable
;the path to store the database file of sqlite3
PATH = 

Before Running

• Make sure you have installed dependencies, suggest to use go mod
• Make sure the app.ini in config folder, you can rename app-template.ini to app.ini
• Make sure that you have config and set database correctly, make sure create the corresponding database when using mysqp or postgresql
• Make sure that you have config corresponding tokens for Github or Gitlab

Run

You should build the main.go file firstly with the command go build main.go.

USAGE:
   main [global options] command [command options] [arguments...]

COMMANDS:
     web      Startup a web Service
     scan     Start to scan github leak info
     help, h  Show a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d             Debug Mode
   --host value, -H value  web listen address (default: "0.0.0.0")
   --port value, -p value  web listen port (default: 8000)
   --time value, -t value  scan interval(second) (default: 900)
   --help, -h              show help
   --version, -v           print the version

Add Token

To execute main scan, you need to add a Github token for crawl information in github. You can generate a token in tokens. Most access scopes are enough. For Gitlab search, remember to add token too.

Docker support(not suggested)

Make sure rename app-docker.ini to app.ini.

Build

 docker build -t gshark-docker .      

Run web

sqlite_database_folder is the folder for the sqlite database folder, make sure create gshark.db file inside the folder.

docker run -e OPTION=web -p 8000:8000 -v sqlite_database_folder:/data/gshark gshark-docker

Run Scan

docker run -e OPTION=scan -v sqlite_database_folder:/data/gshark gshark-docker

Add notification

Now support notification by server 酱. Set the config of SCKEY in app.ini file.

FAQ

1. Access web service 403 forbidden

Access to http://127.0.0.1/admin/login

2. Default username and password

gshark/gshark

3. # github.com/mattn/go-sqlite3 exec: "gcc": executable file not found in %PATH%

https://github.com/mattn/go-sqlite3/issues/435#issuecomment-314247676

4. go get ./... connection error

It's suggested to enable goproxy(refer this article for golang upgrade):

go env -w GOPROXY=https://goproxy.cn,direct
go env -w GO111MODULE=on

Reference

• x-patrol
• authz
• macaron

Wechat

If you would like to join wechat group, you can add my wechat mmadneal with the message gshark.

License

Apache License 2.0

404StarLink 2.0 - Galaxy

GShark 是 404Team 星链计划2.0中的一环，如果对 GShark 有任何疑问又或是想要找小伙伴交流，可以参考星链计划的加群方式。

• https://github.com/knownsec/404StarLink2.0-Galaxy#community