All Projects → mitre → stix2patterns_translator

mitre / stix2patterns_translator

Licence: Apache-2.0 license
Translate STIX 2 Patterning Queries

Programming Languages

python
139335 projects - #7 most used programming language
ANTLR
299 projects

Labels

Projects that are alternatives of or similar to stix2patterns translator

server
A cyber threat intelligence server based on TAXII 2 and written in Golang
Stars: ✭ 28 (+12%)
Mutual labels:  stix
cti-stix-generator
OASIS Cyber Threat Intelligence (CTI) TC: A tool for generating STIX content for prototyping and testing. https://github.com/oasis-open/cti-stix-generator
Stars: ✭ 27 (+8%)
Mutual labels:  stix
Misp
MISP (core software) - Open Source Threat Intelligence and Sharing Platform
Stars: ✭ 3,485 (+13840%)
Mutual labels:  stix
stix-icons
stix-icons is a collection of colourful and clean icons for use in software, training and marketing material to visualize cyber threats according to the STIX language for intelligence exchange, defined by OASIS Cyber Threat Intelligence (CTI) TC
Stars: ✭ 14 (-44%)
Mutual labels:  stix
cti-stix-elevator
OASIS Cyber Threat Intelligence (CTI) TC Open Repository: Convert STIX 1.2 XML to STIX 2.x JSON
Stars: ✭ 42 (+68%)
Mutual labels:  stix
cti-pattern-validator
OASIS TC Open Repository: Validate patterns used to express cyber observable content in STIX Indicators
Stars: ✭ 18 (-28%)
Mutual labels:  stix
cti-stix2-json-schemas
OASIS TC Open Repository: Non-normative schemas and examples for STIX 2
Stars: ✭ 75 (+200%)
Mutual labels:  stix
libstix2
APIs for generating STIX 2.1 and TAXII 2.1 messages with Go (Golang)
Stars: ✭ 38 (+52%)
Mutual labels:  stix
stix2-graphics
Graphics, icons, and diagrams to support STIX 2
Stars: ✭ 32 (+28%)
Mutual labels:  stix
FlexTransform
Enables dynamic translation of structured data between formats
Stars: ✭ 14 (-44%)
Mutual labels:  stix
attack-stix-data
STIX data representing MITRE ATT&CK
Stars: ✭ 118 (+372%)
Mutual labels:  stix

stix-pattern-translator

This repository contains a prototype analytic translator that converts STIX2 Patterning queries into other query languages, currently ElasticSearch and Splunk. In addition to translating query syntax, the translator will also translate from STIX 2's Data Model to other target data models, currently MITRE's Cyber Analytic Repository (CAR) and Splunk's Common Information Model (CIM). Both the query language translation and the data model translation are implemented as loosely-coupled modules and other targets can be added simply by implementing new modules to generate them.

This functionality was originally developed for MITRE's CASCADE project. This repository offers CASCADE's translation capability as a standalone feature, and replaces CASCADE's own domain-specific language (DSL) with STIX 2.0 Patterning.

Requirements

Optionally, the Web API requires:

Installation

Install with pip:

$ pip install stix-pattern-translator

Optionally, to use the Web API, install with pip:

$ pip install stix-pattern-translator[web]

Note: if you are doing development, make sure to install the development dependencies with:

$ pip install stix-pattern-translator[dev]

Usage

The STIX Analytic Translator provides an executable script (translate-pattern), a simple web API provided as a Flask application, and Python translation method (stix2patterns_translator.translate).

From Python Code

from stix2patterns_translator.translator import *

translate("[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4']", SearchPlatforms.ELASTIC, DataModels.CAR)

Command Line

Use the command translate-stix-pattern, passing in the desired output language, data taxonomy, and STIX2 pattern:

$ translate-stix-pattern --output-language=elastic --output-data-model=car "[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4']"

Web API

The web API is provided as a Flask server that accepts the STIX pattern as input and returns the translated pattern. The path determines the output search platform and data taxonomy. To run the web server, install the 'web' extras (flask).

$ pip install stix-pattern-translator[web]

The server can be run directly from the source code, via a command line script, or by importing the code into an existing application. The easiest way to test it is via the command line script:

$ pattern-translator-server

With the server running, use a command-line to send and receive from Flask instance, either locally or across a network. As the below example shows, using cURL to send a POST with JSON generates results back to the calling shell:

$ curl -X POST -H "Content-Type: text/plain" -d "[process:pid <= 5]" http://localhost:5000/car-elastic

That yields:

data_model.object:"process" AND data_model.action:"*" AND (
    data_model.fields.pid:<=5
)

Testing

Pytest integration tests are auto generated from input patterns pulled from files in test/input_files. Input files are named by test type, e.g. "md5_hash.json" and contain json with input pattern (key = stix-input) and expected results, where the key is <datamodel>-<platform> and the value is the expected result:

{
  "stix-input":"[file:hashes.MD5 ='79054025255fb1a26e4bc422aef54eb4']",
  "car-elastic":"data_model.object:\"file\" AND data_model.action:\"*\" AND (data_model.fields.md5_hash:\"79054025255fb1a26e4bc422aef54eb4\")",
  "car-splunk":"match(tag, \"dm-file-.*\") md5_hash=\"79054025255fb1a26e4bc422aef54eb4\"",
  "cim-elastic": null,
  "cim-splunk":null
}

Expected result can either be a string, which tests for success and a match of that string, or null, which tests for error. As usual, the tests can be run by running pytest:

pytest

Tests can also be run that push events to Splunk/Elastic and then making sure the correct events match. They use the same input files, but require a few additional keys. The "matches" key should contain a dictionary with a set of keys for each data model (currently just "CAR"). That dictionary then contains a list of events that should match the pattern. The "nonmatches" key is identical, but obviously will be tested to make sure they don't match. Running the live integration tests requires starting docker:

docker-compose -f test.docker-compose.yml up

They can then be run via pytest, by calling the test directly:

pytest test/integration_tests.py
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].