All Projects → csirtgadgets → cif-v5

csirtgadgets / cif-v5

Licence: MPL-2.0 license
The FASTEST way to consume threat intel.

Programming Languages

python
139335 projects - #7 most used programming language
shell
77523 projects
Dockerfile
14818 projects
Makefile
30231 projects

Projects that are alternatives of or similar to cif-v5

MurMurHash
This little tool is to calculate a MurmurHash value of a favicon to hunt phishing websites on the Shodan platform.
Stars: ✭ 79 (+49.06%)
Mutual labels:  threatintel, threathunting
ioc-fanger
Fang and defang indicators of compromise. You can test this project in a GUI here: http://ioc-fanger.hightower.space .
Stars: ✭ 47 (-11.32%)
Mutual labels:  threatintel
Threatbus
🚌 The missing link to connect open-source threat intelligence tools.
Stars: ✭ 139 (+162.26%)
Mutual labels:  threatintel
Intrec Pack
Intelligence and Reconnaissance Package/Bundle installer.
Stars: ✭ 177 (+233.96%)
Mutual labels:  threatintel
Misp Dashboard
A dashboard for a real-time overview of threat intelligence from MISP instances
Stars: ✭ 142 (+167.92%)
Mutual labels:  threatintel
commerce-cif-magento-graphql
Magento GraphQL data models and query builders for AEM
Stars: ✭ 20 (-62.26%)
Mutual labels:  cif
Chatter
internet monitoring osint telegram bot for windows
Stars: ✭ 123 (+132.08%)
Mutual labels:  threatintel
csirtg-smrt-v1
the fastest way to consume threat intelligence.
Stars: ✭ 27 (-49.06%)
Mutual labels:  threatintel
Public-Intelligence-Feeds
Standard-Format Threat Intelligence Feeds
Stars: ✭ 60 (+13.21%)
Mutual labels:  threatintel
Sweetie Data
This repo contains logstash of various honeypots
Stars: ✭ 163 (+207.55%)
Mutual labels:  threatintel
Bearded Avenger
CIF v3 -- the fastest way to consume threat intelligence
Stars: ✭ 152 (+186.79%)
Mutual labels:  threatintel
Intelowl
Intel Owl: analyze files, domains, IPs in multiple ways from a single API at scale
Stars: ✭ 2,114 (+3888.68%)
Mutual labels:  threatintel
aem-cif-project-archetype
Maven template to create new CIF Project AEM projects that follow best practices
Stars: ✭ 20 (-62.26%)
Mutual labels:  cif
Forager
Multithreaded threat Intelligence gathering built with Python3
Stars: ✭ 140 (+164.15%)
Mutual labels:  threatintel
Malware-Zoo
Hashes of infamous malware
Stars: ✭ 18 (-66.04%)
Mutual labels:  threatintel
Graylog Plugin Threatintel
Graylog Processing Pipeline functions to enrich log messages with IoC information from threat intelligence databases
Stars: ✭ 132 (+149.06%)
Mutual labels:  threatintel
Hippocampe
Threat Feed Aggregation, Made Easy
Stars: ✭ 149 (+181.13%)
Mutual labels:  threatintel
Misp Training
MISP trainings, threat intel and information sharing training materials with source code
Stars: ✭ 185 (+249.06%)
Mutual labels:  threatintel
ThreatIntelligence
Tracking APT IOCs
Stars: ✭ 23 (-56.6%)
Mutual labels:  threatintel
commerce-cif-connector
AEM Commerce connector for Magento and GraphQL
Stars: ✭ 42 (-20.75%)
Mutual labels:  cif

Getting Started

CIF is a model, .. it's a toy. While it's used in a number of large scale environments, it's meant to be a teaching tool. It enables you as an operator to learn from years of operational, institutional knowledge. The default, out of the box configuration is geared 100% towards getting you up and running as quickly as possible (5min or less) so you can LEARN from our experience.

The default, CIF/Docker configuration is NOT meant to be deployed in large scale operations. That's your job. Taking what you've learned and the components we've given you, then creating your own master piece with them. Better yet- because of them.

python 3.6+ is required (eg: Ubuntu 18 LTS or higher)

Need More Help?

Docker

Up and Running

$ export CSIRTG_TOKEN=1234  # sign up at csirtg.io
$ export MAXMIND_USERID=1234  # sign up at maxmind.com to leverage geo location data
$ export MAXMIND_LIC=1234

$ git clone https://github.com/csirtgadgets/cif-v5.git
$ mkdir data  # shared data directory for containers
$ cp cif-v5/docker-compose.yml ./
$ mkdir -p data/rules  # copy any local custom rules here
$ # cp my_custom_rule.yml data/rules/
$ chmod 755 data/rules
$ docker-compose pull
$ docker-compose up -d

Status

$ docker-compose logs -f
Attaching to cif-router, cif-httpd, cif-enrichers, csirtg-fm
cif-httpd       | [2020-03-14 15:05:29 +0000] [7] [INFO] Starting gunicorn 19.10.0
cif-httpd       | [2020-03-14 15:05:29 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7)
cif-httpd       | [2020-03-14 15:05:29 +0000] [7] [INFO] Using worker: gevent
cif-httpd       | [2020-03-14 15:05:29 +0000] [10] [INFO] Booting worker with pid: 10
cif-httpd       | [2020-03-14 15:05:29 +0000] [11] [INFO] Booting worker with pid: 11
cif-httpd       | [2020-03-14 15:05:29 +0000] [12] [INFO] Booting worker with pid: 12
cif-httpd       | [2020-03-14 15:05:29 +0000] [13] [INFO] Booting worker with pid: 13
csirtg-fm       | 2020-03-14 15:05:30,107 - INFO - csirtg_fm.cli[316] - random delay is 1.0
csirtg-fm       | 2020-03-14 15:05:30,107 - INFO - csirtg_fm.cli[317] - running every 60 after that  # <--- data will start coming in after this 60 delay
csirtg-fm       | 2020-03-14 15:06:30,184 - INFO - csirtg_fm.cli[85] - starting run...
csirtg-fm       | 2020-03-14 15:06:30,261 - INFO - csirtg_fm.cli[157] - processing: openphish.yml - urls
csirtg-fm       | 2020-03-14 15:06:38,681 - INFO - csirtg_fm.cli[157] - processing: abuse_ch.yml - urlhaus
csirtg-fm       | 2020-03-14 15:07:01,407 - INFO - csirtg_fm[125] - sending: 6
csirtg-fm       | 2020-03-14 15:07:01,598 - INFO - csirtg_fm.cli[157] - processing: abuse_ch.yml - feodo_malware
csirtg-fm       | 2020-03-14 15:07:20,339 - INFO - csirtg_fm.cli[157] - processing: abuse_ch.yml - feodo_botnet
csirtg-fm       | 2020-03-14 15:07:32,947 - INFO - csirtg_fm[125] - sending: 31
csirtg-fm       | 2020-03-14 15:07:33,745 - INFO - csirtg_fm.cli[157] - processing: alexa.yml - top-1000
csirtg-fm       | 2020-03-14 15:07:36,896 - INFO - csirtg_fm[125] - sending: 500
csirtg-fm       | 2020-03-14 15:07:45,937 - INFO - csirtg_fm[125] - sending: 500

Testing

# this requires python3.6 or higher (eg: ubuntu 18 LTS..)

$ pip3 install geoip2 'cifsdk>=5.0b1,<6.0'

$ export CIF_REMOTE='http://localhost:5000'
$ cif -nq example.com
$ cif --itype url --tags malware  # may have to wait 5-10 after starting as data flows in

Where Next?

What's Changed?

  • Less. Stuff.
  • Abstracted a lot of the technical pieces into separate libraries (eg: csirtg-geo, csirtg-peers, etc)
  • Codebase has been significantly simplified
  • NO TOKENS, YOU ARE RESPONSIBLE FOR PROTECTING YOUR NODES!!!
  • Pipelines and better Plugin support for "external enrichment"
  • Docker Compose is the first class citizen, feel free to customize from there
  • Docker containers are now split up, not running as supervisord anymore
  • cif-httpd is significantly simpler to interact with, improved REST doc (openapi)
  • New threat intel rules
  • Python 3.7+ support

Building Locally

$ git clone https://github.com/csirtgadgets/cif-v5.git
$ cd cif-v5/
$ make docker
$ make docker-tag
$ docker-compose up -d
$ docker-compose logs -f

Architecture

                                                              cif-enricher
                                                               ^        +
                                                               |        |
                                                               +        v
csirtg-fm +--> cifsdk  +--------->  cif-httpd +------------> cif-router +-----> cif-store +-----> sqlite
                                                               +
                                           ^                   |        ^
                                           |                   |        |
                                           |                   v        +
                                           |                   cif-hunter
                                           +

                                        cifsdk

Getting Involved

There are many ways to get involved with the project. If you have a new and exciting feature, or even a simple bugfix, simply fork the repo, create some simple test cases, generate a pull-request and give yourself credit!

If you've never worked on a GitHub project, this is a good piece for getting started.

paypal

COPYRIGHT AND LICENSE

Copyright (C) 2020 the CSIRT Gadgets

Free use of this software is granted under the terms of the Mozilla Public License (MPLv2).

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].