GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → rambasnet → DeepLearning-IDS

rambasnet / DeepLearning-IDS

Licence: MIT license
Network Intrusion Detection System using Deep Learning Techniques

Programming Languages

python
139335 projects - #7 most used programming language

Labels

theanodeep-learningtensorflowneural-networksintrusion-detectionfastai

Projects that are alternatives of or similar to DeepLearning-IDS

symbolic-pymc
Tools for the symbolic manipulation of PyMC models, Theano, and TensorFlow graphs.
Stars: ✭ 58 (-23.68%)
Mutual labels:  theano
sandfly-setup
Sandfly Security Agentless Compromise and Intrusion Detection System For Linux
Stars: ✭ 45 (-40.79%)
Mutual labels:  intrusion-detection
fastai-docker
Fast.AI course complete docker container for Paperspace and Gradient
Stars: ✭ 52 (-31.58%)
Mutual labels:  fastai
autofocus
Deep learning computer vision for classifying wildlife in camera trap images
Stars: ✭ 48 (-36.84%)
Mutual labels:  fastai
Final-year-project-deep-learning-models
Deep learning for freehand sketch object recognition
Stars: ✭ 22 (-71.05%)
Mutual labels:  theano
Plant Disease Detection
Plant Disease Detector Web Application
Stars: ✭ 181 (+138.16%)
Mutual labels:  fastai
rnn benchmarks
RNN benchmarks of pytorch, tensorflow and theano
Stars: ✭ 85 (+11.84%)
Mutual labels:  theano
UnityImageSynthesisTutorial1
Use Unity to generate synthetic images for deep learning image segmentation in PyTorch and fastai
Stars: ✭ 93 (+22.37%)
Mutual labels:  fastai
fastdownload
Easily download, verify, and extract archives
Stars: ✭ 35 (-53.95%)
Mutual labels:  fastai
FAI-notes
Some notes, tutorials, and some experimentation with the fast.ai library (https://github.com/fastai/fastai)
Stars: ✭ 58 (-23.68%)
Mutual labels:  fastai
fastblog
My fastai blog
Stars: ✭ 20 (-73.68%)
Mutual labels:  fastai
cudnn rnn theano benchmarks
No description or website provided.
Stars: ✭ 22 (-71.05%)
Mutual labels:  theano
BURN
[WIP] Anti-Forensics ToolKit to clear post-intrusion sensible logfiles 🔥 (For Research Only)
Stars: ✭ 13 (-82.89%)
Mutual labels:  intrusion-detection
NetLearner
Deep Learning Tool for Off-line Network Intrusion Detection System
Stars: ✭ 24 (-68.42%)
Mutual labels:  intrusion-detection
bihm
Bidirectional Helmholtz Machines
Stars: ✭ 40 (-47.37%)
Mutual labels:  theano
ulm-basenet
Implementation of ULMFit algorithm for text classification via transfer learning
Stars: ✭ 94 (+23.68%)
Mutual labels:  fastai
encrypted-skin-cancer-detection
Detecting skin cancer in encrypted images with TensorFlow
Stars: ✭ 27 (-64.47%)
Mutual labels:  fastai
wazuh-packages
Wazuh - Tools for packages creation
Stars: ✭ 54 (-28.95%)
Mutual labels:  intrusion-detection
fasterai1
FasterAI: A repository for making smaller and faster models with the FastAI library.
Stars: ✭ 34 (-55.26%)
Mutual labels:  fastai
fastai sequence tagging
sequence tagging for NER for ULMFiT
Stars: ✭ 21 (-72.37%)
Mutual labels:  fastai
View All Similar Projects ➔

Deep Learning - IDS

Towards Developing a Network Intrusion Detection System using Deep Learning Techniques

Introduction

In this project, we aim to explore the capabilities of various deep-learning frameworks in detecting and classifying network intursion traffic with an eye towards designing a ML-based intrusion detection system.

Dataset

  • Downloaded from: https://www.unb.ca/cic/datasets/ids-2018.html
  • contains: 7 csv preprocessed and labelled files, top feature selected files, original traffic data in pcap format and logs
  • used csv preprocessed and labelled files for this research project

Data Cleanup

  • dropped rows with Infinitiy values
  • some files had repeated headers; dropped those
  • converted timestamp value that was date time format: 15-2-2018 to UNIX epoch since 1/1/1970
  • separated data based on attack types for each data file
  • ~20K rows were removed as a part of data cleanup
  • see data_cleanup.py script for this phase
  • # Samples in table below are total samples left in each dataset after dropping # Dropped rows/samples

Dataset Summary

File Name Traffic Type # Samples # Dropped
02-14-2018.csv Benign 663,808 3818
FTP-BruteForce 193,354 6
SSH-Bruteforce 187,589 0
02-15-2018.csv Benign 988,050 8027
DOS-GoldenEye 41,508 0
DOS-Slowloris 10,990 0
02-16-2018.csv Benign 446,772 0
Dos-SlowHTTPTest 139,890 0
DoS-Hulk 461,912 0
02-22-2018.csv Benign 1,042,603 5610
BruteForce-Web 249 0
BruteForce-XSS 79 0
SQL-Injection 34 0
02-23-2018.csv Benign 1,042,301 5708
BruteForce-Web 362 0
BruteForce-XSS 151 0
SQL-Injection 53 0
03-01-2018.csv Benign 235,778 2259
Infiltration 92,403 660
03-02-2018.csv Benign 758,334 4050
BotAttack 286,191 0
Traffic Type # Samples
Benign 5,177,646
FTP-BruteForce 193,354
SSH-BruteForce 187,589
DOS-GoldenEye 41,508
Dos-Slowloris 10,990
Dos-SlowHTTPTest 139,890
Dos-Hulk 461,912
BruteForce-Web 611
BruteForce-XSS 230
SQL-Injection 87
Infiltration 92,403
BotAttack 286,191
Total Attack 1,414,765

Deep Learning Frameworks

  • perfomance results using various deep learning frameworks are compared
  • 10-fold cross-validation techniques was used to validate the model

FastAI

Keras

Experiment Results

Using Salamander.ai

Dataset Framework Accuracy (%) Std-Dev GPU Time (~mins)
02-14-2018 FastAI 99.85 0.07 *
Keras-TensorFlow 98.80 * *
Keras-Theano * * *
02-15-2018 FastAI 99.98 0.01 25
Keras-Tensorfflow 99.32 * *
Keras-Theano * * *
02-16-2018 FastAI 100.00 0.00 16
Keras-TensorFlow 99.84 * *
Keras-Theano * * *
02-22-2018 FastAI 99.87 0.15 110
Keras-TensorFlow 99.97 * *
Keras-Theano * * *
02-23-2018 FastAI 99.92 0.00 120
Keras-TensorFlow 99.94 * *
Keras-Theano * * *
03-01-2018 FastAI 87.00 0.00 5
Keras-TensorFlow 72.20 * *
Keras-Theano * * *
03-02-2018 FastAI 99.97 .01 75
Keras-TensorFlow 98.12 * *
Keras-Theano * * *
=== === === === ===
Multiclass Keras-TensorFlow 94.73 * *
Keras-Theano * * *
FastAI * * *
Binaryclass Keras-TensorFlow 94.40 * *
Keras-Theano * * *
FastAI * * *

FastAI Results

Summary Results

Data File Accuracy Loss
02-14-2018.csv 99.99% 0.00212
02-15-2018.csv 99.86% 0.02500
02-16-2018.csv 99.97% 324160
02-22-2018.csv 99.97% 0.00221
02-23-2018.csv 99.82% 0.06295
03-01-2018.csv 87.14% 0.37611
03-02-2018.csv 99.72% 0.85127
IDS-2018-binaryclass.csv* 98.68% 0.37692
IDS-2018-multiclass.csv* 98.31% 7.06169

* Trained on VMgpu

Confusion Matrices

02-14-2018 02-15-2018 02-16-2018
02-22-2018 02-23-2018 03-01-2018
03-02-2018 IDS-2018-binaryclass IDS-2018-multiclass

Attack Sample Distribution and Detection Accuracy

Data File % of Attack Samples % Attacks Flagged Correctly % Benign Flagged Incorrectly
02-14-2018 36.46 100.00 0.00*
02-15-2018 5.04 99.85 0.00*
02-16-2018 57.39 100.00 0.00*
02-22-2018 0.00* 0.02 0.00
02-23-2018 0.00* 61.61 0.00*
03-01-2018 28.16 73.19 10.16
03-02-2018 27.40 99.85 0.00*
Binary-Class 21.50 94.60 0.21
Multi-Class 21.50 93.9 0.48

* Small, non-zero values

Using VMgpu

Dataset Framework Accuracy (%) Std-Dev GPU Time (~mins)
02-14-2018 FastAI 99.54 0.32 100.36
Keras-TensorFlow 99.14 * 100.29
Keras-Theano 98.58 * *
02-15-2018 FastAI 99.98 0.01 103.16
Keras-TensorFlow 99.33 * 96.34
Keras-Theano 99.17 * *
02-16-2018 FastAI 99.66 0.25 104.51
Keras-TensorFlow 99.66 * 99.59
Keras-Theano 99.41 * *
02-22-2018 FastAI 99.90 0.09 102.83
Keras-TensorFlow 99.97 * 96.71
Keras-Theano 99.97 * *
02-23-2018 FastAI 99.88 0.08 104.43
Keras-TensorFlow 95.95 * 100.79
Keras-Theano 99.95 * *
03-01-2018 FastAI 86.47 0.78 33.23
Keras-TensorFlow 72.16 * 33.15
Keras-Theano 72.04 * *
03-02-2018 FastAI 99.94 0.04 104.34
Keras-TensorFlow 98.47 * 105.95
Keras-Theano 93.95 * *
=== === === === ===
Multiclass FastAI 98.60 0.16 683.12
Keras-TensorFlow 92.09 * 652.89
Keras-Theano * * *
Binaryclass FastAI 98.75 0.05 675.98
Keras-TensorFlow 95.14 * 632.36
Keras-Theano * * *

fastai CPU vs GPU training time on vmGPU

Dataset Hardware Accuracy (%) Time (~mins)
02-14-2018
CPU 99.86 1193.84
GPU 99.54 100.36
02-15-2018
CPU 99.93 1299.55
GPU 99.89 103.16
02-16-2018
CPU 99.88 433.63
GPU 99.66 104.51
02-22-2018
CPU 99.83 3091.34
GPU 99.90 102.83
02-23-2018
CPU 99.83 1938.74
GPU 99.88 104.43
03-01-2018
CPU 85.39 80.07
GPU 86.47 33.23
03-02-2018
CPU 99.76 1503.18
GPU 99.94 104.34
=== === === ===
Multiclass
CPU 96.63 19361.95
GPU 98.60 683.12
Binaryclass
CPU 96.66 19441.55
GPU 98.75 632.36

References

  1. Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani, “Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization”, 4th International Conference on Information Systems Security and Privacy (ICISSP), Portugal, January 2018
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].