All Projects → MIC-DKFZ → Medicaldetectiontoolkit

MIC-DKFZ / Medicaldetectiontoolkit

Licence: apache-2.0
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

Programming Languages

139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Medicaldetectiontoolkit

Kiu Net Pytorch
Official Pytorch Code of KiU-Net for Image Segmentation - MICCAI 2020 (Oral)
Stars: ✭ 134 (-85.39%)
Mutual labels:  deep-neural-networks, segmentation, semantic-segmentation, medical-imaging
Jacinto Ai Devkit
Training & Quantization of embedded friendly Deep Learning / Machine Learning / Computer Vision models
Stars: ✭ 49 (-94.66%)
Mutual labels:  object-detection, segmentation, semantic-segmentation, detection
Cascaded Fcn
Source code for the MICCAI 2016 Paper "Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional NeuralNetworks and 3D Conditional Random Fields"
Stars: ✭ 296 (-67.72%)
Mutual labels:  deep-neural-networks, segmentation, semantic-segmentation, medical-imaging
Cnn Paper2
🎨 🎨 深度学习 卷积神经网络教程 :图像识别,目标检测,语义分割,实例分割,人脸识别,神经风格转换,GAN等🎨🎨
Stars: ✭ 77 (-91.6%)
Mutual labels:  object-detection, deep-neural-networks, segmentation, detection
Keras Unet
Helper package with multiple U-Net implementations in Keras as well as useful utility tools helpful when working with image semantic segmentation tasks. This library and underlying tools come from multiple projects I performed working on semantic segmentation tasks
Stars: ✭ 196 (-78.63%)
Mutual labels:  deep-neural-networks, segmentation, semantic-segmentation
PaddlePaddle End-to-End Development Toolkit(『飞桨』深度学习全流程开发工具)
Stars: ✭ 3,399 (+270.67%)
Mutual labels:  deep-neural-networks, segmentation, detection
Fast image augmentation library and an easy-to-use wrapper around other libraries. Documentation: Paper about the library:
Stars: ✭ 9,353 (+919.96%)
Mutual labels:  object-detection, segmentation, detection
[unmaintained] An open-source convolutional neural networks platform for research in medical image analysis and image-guided therapy
Stars: ✭ 1,276 (+39.15%)
Mutual labels:  deep-neural-networks, segmentation, medical-imaging
Kaggle Rsna
Deep Learning for Automatic Pneumonia Detection, RSNA challenge
Stars: ✭ 74 (-91.93%)
Mutual labels:  object-detection, medical-imaging, detection
All About The Gan
All About the GANs(Generative Adversarial Networks) - Summarized lists for GAN
Stars: ✭ 630 (-31.3%)
Mutual labels:  segmentation, medical-imaging, detection
Perception toolkit for sim2real training and validation
Stars: ✭ 208 (-77.32%)
Mutual labels:  object-detection, segmentation, detection
Vehicle counting tensorflow
🚘 "MORE THAN VEHICLE COUNTING!" This project provides prediction for speed, color and size of the vehicles with TensorFlow Object Counting API.
Stars: ✭ 582 (-36.53%)
Mutual labels:  object-detection, deep-neural-networks, detection
Deep Learning For Tracking And Detection
Collection of papers, datasets, code and other resources for object tracking and detection using deep learning
Stars: ✭ 1,920 (+109.38%)
Mutual labels:  object-detection, segmentation, detection
Resurces for MRI images processing and deep learning in 3D
Stars: ✭ 56 (-93.89%)
Mutual labels:  detection, medical-imaging, segmentation
Unet 3d
3D Unet Equipped with Advanced Deep Learning Methods
Stars: ✭ 57 (-93.78%)
Mutual labels:  3d-models, deep-neural-networks, segmentation
Crfasrnn pytorch
CRF-RNN PyTorch version
Stars: ✭ 102 (-88.88%)
Mutual labels:  deep-neural-networks, segmentation, semantic-segmentation
Awesome Gan For Medical Imaging
Awesome GAN for Medical Imaging
Stars: ✭ 1,814 (+97.82%)
Mutual labels:  segmentation, medical-imaging, detection
Ssd Pytorch
SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity
Stars: ✭ 107 (-88.33%)
Mutual labels:  object-detection, deep-neural-networks, detection
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation (ECCV2020)
Stars: ✭ 255 (-72.19%)
Mutual labels:  object-detection, segmentation, detection
Rectlabel Support
RectLabel - An image annotation tool to label images for bounding box object detection and segmentation.
Stars: ✭ 338 (-63.14%)
Mutual labels:  object-detection, segmentation, detection

Copyright © German Cancer Research Center (DKFZ), Division of Medical Image Computing (MIC). Please make sure that your usage of this code is in compliance with the code license.


This is a comprehensive framework for object detection featuring:

  • 2D + 3D implementations of prevalent object detectors: e.g. Mask R-CNN [1], Retina Net [2], Retina U-Net [3].
  • Modular and light-weight structure ensuring sharing of all processing steps (incl. backbone architecture) for comparability of models.
  • training with bounding box and/or pixel-wise annotations.
  • dynamic patching and tiling of 2D + 3D images (for training and inference).
  • weighted consolidation of box predictions across patch-overlaps, ensembles, and dimensions [3].
  • monitoring + evaluation simultaneously on object and patient level.
  • 2D + 3D output visualizations.
  • integration of COCO mean average precision metric [5].
  • integration of MIC-DKFZ batch generators for extensive data augmentation [6].
  • easy modification to evaluation of instance segmentation and/or semantic segmentation.

[1] He, Kaiming, et al. "Mask R-CNN" ICCV, 2017
[2] Lin, Tsung-Yi, et al. "Focal Loss for Dense Object Detection" TPAMI, 2018.
[3] Jaeger, Paul et al. "Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection" , 2018


How to cite this code

Please cite the original publication [3].


Setup package in a virtual environment:

git clone .
cd medicaldetectiontoolkit
virtualenv -p python3.6 venv
source venv/bin/activate
pip3 install -e .

We use two cuda functions: Non-Maximum Suppression (taken from pytorch-faster-rcnn and added adaption for 3D) and RoiAlign (taken from RoiAlign, fixed according to this bug report, and added adaption for 3D). In this framework, they come pre-compile for TitanX. If you have a different GPU you need to re-compile these functions:

GPU arch
TitanX sm_52
GTX 960M sm_50
GTX 1070 sm_61
GTX 1080 (Ti) sm_61
cd cuda_functions/nms_xD/src/cuda/
nvcc -c -o -x cu -Xcompiler -fPIC -arch=[arch]
cd ../../
cd ../

cd cuda_functions/roi_align_xD/roi_align/src/cuda/
nvcc -c -o -x cu -Xcompiler -fPIC -arch=[arch]
cd ../../
cd ../../

Prepare the Data

This framework is meant for you to be able to train models on your own data sets. Two example data loaders are provided in medicaldetectiontoolkit/experiments including thorough documentation to ensure a quick start for your own project. The way I load Data is to have a preprocessing script, which after preprocessing saves the Data of whatever data type into numpy arrays (this is just run once). During training / testing, the data loader then loads these numpy arrays dynamically. (Please note the Data Input side is meant to be customized by you according to your own needs and the provided Data loaders are merely examples: LIDC has a powerful Dataloader that handles 2D/3D inputs and is optimized for patch-based training and inference. Toy-Experiments have a lightweight Dataloader, only handling 2D without patching. The latter makes sense if you want to get familiar with the framework.).


  1. Set I/O paths, model and training specifics in the configs file: medicaldetectiontoolkit/experiments/your_experiment/

  2. Train the model:

    python --mode train --exp_source experiments/my_experiment --exp_dir path/to/experiment/directory       

    This copies snapshots of configs and model to the specified exp_dir, where all outputs will be saved. By default, the data is split into 60% training and 20% validation and 20% testing data to perform a 5-fold cross validation (can be changed to hold-out test set in configs) and all folds will be trained iteratively. In order to train a single fold, specify it using the folds arg:

    python --folds 0 1 2 .... # specify any combination of folds [0-4]
  3. Run inference:

    python --mode test --exp_dir path/to/experiment/directory 

    This runs the prediction pipeline and saves all results to exp_dir.


This framework features all models explored in [3] (implemented in 2D + 3D): The proposed Retina U-Net, a simple but effective Architecture fusing state-of-the-art semantic segmentation with object detection,

also implementations of prevalent object detectors, such as Mask R-CNN, Faster R-CNN+ (Faster R-CNN w\ RoIAlign), Retina Net, U-Faster R-CNN+ (the two stage counterpart of Retina U-Net: Faster R-CNN with auxiliary semantic segmentation), DetU-Net (a U-Net like segmentation architecture with heuristics for object detection.)

Training annotations

This framework features training with pixelwise and/or bounding box annotations. To overcome the issue of box coordinates in data augmentation, we feed the annotation masks through data augmentation (create a pseudo mask, if only bounding box annotations provided) and draw the boxes afterwards.

The framework further handles two types of pixel-wise annotations:

  1. A label map with individual ROIs identified by increasing label values, accompanied by a vector containing in each position the class target for the lesion with the corresponding label (for this mode set get_rois_from_seg_flag = False when calling ConvertSegToBoundingBoxCoordinates in your Data Loader).
  2. A binary label map. There is only one foreground class and single lesions are not identified. All lesions have the same class target (foreground). In this case the Dataloader runs a Connected Component Labelling algorithm to create processable lesion - class target pairs on the fly (for this mode set get_rois_from_seg_flag = True when calling ConvertSegToBoundingBoxCoordinates in your Data Loader).

Prediction pipeline

This framework provides an inference module, which automatically handles patching of inputs, and tiling, ensembling, and weighted consolidation of output predictions:

Consolidation of predictions (Weighted Box Clustering)

Multiple predictions of the same image (from test time augmentations, tested epochs and overlapping patches), result in a high amount of boxes (or cubes), which need to be consolidated. In semantic segmentation, the final output would typically be obtained by averaging every pixel over all predictions. As described in [3], weighted box clustering (WBC) does this for box predictions:

Visualization / Monitoring

By default, loss functions and performance metrics are monitored:

Histograms of matched output predictions for training/validation/testing are plotted per foreground class:

Input images + ground truth annotations + output predictions of a sampled validation abtch are plotted after each epoch (here 2D sampled slice with +-3 neighbouring context slices in channels):

Zoomed into the last two lines of the plot:


This framework is published under the Apache License Version 2.0.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].