Amazing-Semantic-Segmentation

Amazing Semantic Segmentation on Tensorflow && Keras (include FCN, UNet, SegNet, PSPNet, PAN, RefineNet, DeepLabV3, DeepLabV3+, DenseASPP, BiSegNet ...)

Models

The project supports these semantic segmentation models as follows:

1. FCN-8s/16s/32s - Fully Convolutional Networks for Semantic Segmentation
2. UNet - U-Net: Convolutional Networks for Biomedical Image Segmentation
3. SegNet - SegNet:A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
4. Bayesian-SegNet - Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
5. PSPNet - Pyramid Scene Parsing Network
6. RefineNet - RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
7. PAN - Pyramid Attention Network for Semantic Segmentation
8. DeepLabV3 - Rethinking Atrous Convolution for Semantic Image Segmentation
9. DeepLabV3Plus - Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
10. DenseASPP - DenseASPP for Semantic Segmentation in Street Scenes
11. BiSegNet - BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

Base Models

The project supports these backbone models as follows, and your can choose suitable base model according to your needs.

1. VGG16/19 - Very Deep Convolutional Networks for Large-Scale Image Recognition
2. ResNet50/101/152 - Deep Residual Learning for Image Recognition
3. DenseNet121/169/201/264 - Densely Connected Convolutional Networks
4. MobileNetV1 - MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
5. MobileNetV2 - MobileNetV2: Inverted Residuals and Linear Bottlenecks
6. Xception - Xception: Deep Learning with Depthwise Separable Convolutions
7. Xception-DeepLab - Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Losses

The project supports these loss functions:

1. Cross Entropy
2. Focal Loss
3. MIoU Loss
4. Self Balanced Focal Loss original
5. ...

Optimizers

The project supports these optimizers:

1. SGD
2. Adam
3. Nadam
4. AdamW
5. NadamW
6. SGDW

Learning Rate Scheduler

The project supports these learning rate schedule strategies:

1. step decay
2. poly decay
3. cosine decay
4. warm up

Dataset Setting

The folds of your dataset must satisfy the following structures:

|-- dataset
|  |-- train
|  |  |-- images
|  |  |-- labels
|  |-- valid
|  |  |-- images
|  |  |-- labels
|  |-- test
|  |  |-- images
|  |  |-- labels
|  |-- class_dict.csv
|  |-- evaluated_classes

Installation

• Numpy pip install numpy
• Pillow pip install pillow
• OpenCV pip install opencv-python
• Tensorflow pip install tensorflow-gpu

Note: The recommended version of tensorflow-gpu is 1.14 or 2.0. And if your tensorflow version is lower, you need to modify some API or upgrade your tensorflow.

Usage

Download

You can download the project through this command:
git clone [email protected]:luyanger1799/Amazing-Semantic-Segmentation.git

Training

The project contains complete codes for training, testing and predicting. And you can perform a simple command as this to build a model on your dataset:

python train.py --model FCN-8s --base_model ResNet50 --dataset "dataset_path" --num_classes "num_classes"

The detailed command line parameters are as follows:

usage: train.py [-h] --model MODEL [--base_model BASE_MODEL] --dataset DATASET
                [--loss {CE,Focal_Loss}] --num_classes NUM_CLASSES
                [--random_crop RANDOM_CROP] [--crop_height CROP_HEIGHT]
                [--crop_width CROP_WIDTH] [--batch_size BATCH_SIZE]
                [--valid_batch_size VALID_BATCH_SIZE]
                [--num_epochs NUM_EPOCHS] [--initial_epoch INITIAL_EPOCH]
                [--h_flip H_FLIP] [--v_flip V_FLIP]
                [--brightness BRIGHTNESS [BRIGHTNESS ...]]
                [--rotation ROTATION]
                [--zoom_range ZOOM_RANGE [ZOOM_RANGE ...]]
                [--channel_shift CHANNEL_SHIFT]
                [--data_aug_rate DATA_AUG_RATE]
                [--checkpoint_freq CHECKPOINT_FREQ]
                [--validation_freq VALIDATION_FREQ]
                [--num_valid_images NUM_VALID_IMAGES]
                [--data_shuffle DATA_SHUFFLE] [--random_seed RANDOM_SEED]
                [--weights WEIGHTS]

optional arguments:
  -h, --help            show this help message and exit
  --model MODEL         Choose the semantic segmentation methods.
  --base_model BASE_MODEL
                        Choose the backbone model.
  --dataset DATASET     The path of the dataset.
  --loss {CE,Focal_Loss}
                        The loss function for traing.
  --num_classes NUM_CLASSES
                        The number of classes to be segmented.
  --random_crop RANDOM_CROP
                        Whether to randomly crop the image.
  --crop_height CROP_HEIGHT
                        The height to crop the image.
  --crop_width CROP_WIDTH
                        The width to crop the image.
  --batch_size BATCH_SIZE
                        The training batch size.
  --valid_batch_size VALID_BATCH_SIZE
                        The validation batch size.
  --num_epochs NUM_EPOCHS
                        The number of epochs to train for.
  --initial_epoch INITIAL_EPOCH
                        The initial epoch of training.
  --h_flip H_FLIP       Whether to randomly flip the image horizontally.
  --v_flip V_FLIP       Whether to randomly flip the image vertically.
  --brightness BRIGHTNESS [BRIGHTNESS ...]
                        Randomly change the brightness (list).
  --rotation ROTATION   The angle to randomly rotate the image.
  --zoom_range ZOOM_RANGE [ZOOM_RANGE ...]
                        The times for zooming the image.
  --channel_shift CHANNEL_SHIFT
                        The channel shift range.
  --data_aug_rate DATA_AUG_RATE
                        The rate of data augmentation.
  --checkpoint_freq CHECKPOINT_FREQ
                        How often to save a checkpoint.
  --validation_freq VALIDATION_FREQ
                        How often to perform validation.
  --num_valid_images NUM_VALID_IMAGES
                        The number of images used for validation.
  --data_shuffle DATA_SHUFFLE
                        Whether to shuffle the data.
  --random_seed RANDOM_SEED
                        The random shuffle seed.
  --weights WEIGHTS     The path of weights to be loaded.

If you only want to use the model in your own training code, you can do as this:

from builders.model_builder import builder

model, base_model = builder(num_classes, input_size, model='SegNet', base_model=None)

Note: If you don't give the parameter "base_model", the default backbone will be used.

Testing

Similarly, you can evaluate the model on your own dataset:

python test.py --model FCN-8s --base_model ResNet50 --dataset "dataset_path" --num_classes "num_classes" --weights "weights_path"

Note: If the parameter "weights" is None, the weigths saved in default path will be loaded.

Predicting

You can get the prediction of a single RGB image as this:

python predict.py --model FCN-8s --base_model ResNet50 --num_classes "num_classes" --weights "weights_path" --image_path "image_path"

Evaluating

If you already have the predictions of all test images or you don't want to evaluate all classes, you can do as this:

python evaluate.py --dataset 'dataset_path' --predictions 'prediction_path'

Note: You must specify the class to be evaluated in dataset/evaluated_classes.txt.

PyPI

Alternatively, you can install the project through PyPI.

pip install semantic-segmentation

And you can use model_builders to build different models or directly call the class of semantic segmentation.

from semantic_segmentation import model_builders
net, base_net = model_builders(num_classes, input_size, model='SegNet', base_model=None)

or

from semantic_segmentation import models
net = models.FCN(num_classes, version='FCN-8s')(input_size=input_size)

Pre-trained

Due to my limited computing resources, there is no pre-training model yet. And maybe it will be added in the future.

Feedback

If you like this work, please give me a star! And if you find any errors or have any suggestions, please contact me.

Email: [email protected]