All Projects → sacmehta → Espnet

sacmehta / Espnet

Licence: mit
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

Programming Languages

139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Espnet

Espnetv2 Coreml
Semantic segmentation on iPhone using ESPNetv2
Stars: ✭ 66 (-86.05%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation, real-time
ESANet: Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis
Stars: ✭ 154 (-67.44%)
Mutual labels:  real-time, semantic-segmentation
Self Driving Golf Cart
Be Driven 🚘
Stars: ✭ 147 (-68.92%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
LEDNet: A Lightweight Encoder-Decoder Network for Real-time Semantic Segmentation
Stars: ✭ 450 (-4.86%)
Mutual labels:  semantic-segmentation, real-time
3dunet abdomen cascade
Stars: ✭ 91 (-80.76%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images
Stars: ✭ 100 (-78.86%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
AttaNet for real-time semantic segmentation.
Stars: ✭ 37 (-92.18%)
Mutual labels:  real-time, semantic-segmentation
Bcdu Net
BCDU-Net : Medical Image Segmentation
Stars: ✭ 314 (-33.62%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Geospatial Machine Learning
A curated list of resources focused on Machine Learning in Geospatial Data Science.
Stars: ✭ 289 (-38.9%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics.
Stars: ✭ 274 (-42.07%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Recursive Cnns
Implementation of my paper "Real-time Document Localization in Natural Images by Recursive Application of a CNN."
Stars: ✭ 80 (-83.09%)
Mutual labels:  convolutional-neural-networks, real-time
A light-weight, power efficient, and general purpose convolutional neural network
Stars: ✭ 377 (-20.3%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
A PyTorch-based library for working with 3D and 2D convolutional neural networks, with focus on semantic segmentation of volumetric biomedical image data
Stars: ✭ 78 (-83.51%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
An Embedded Computer Vision & Machine Learning Library (CPU Optimized & IoT Capable)
Stars: ✭ 1,460 (+208.67%)
Mutual labels:  convolutional-neural-networks, real-time
Chainer Pspnet
PSPNet in Chainer
Stars: ✭ 76 (-83.93%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
HyperSeg - Official PyTorch Implementation
Stars: ✭ 174 (-63.21%)
Mutual labels:  real-time, semantic-segmentation
[CVPR 2020] MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
Stars: ✭ 58 (-87.74%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
Stars: ✭ 1,110 (+134.67%)
Mutual labels:  convolutional-neural-networks, semantic-segmentation
Lightweight Segmentation
Lightweight models for real-time semantic segmentation(include mobilenetv1-v3, shufflenetv1-v2, igcv3, efficientnet).
Stars: ✭ 261 (-44.82%)
Mutual labels:  semantic-segmentation, real-time
Kimera Semantics
Real-Time 3D Semantic Reconstruction from 2D data
Stars: ✭ 368 (-22.2%)
Mutual labels:  semantic-segmentation, real-time

ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

This repository contains the source code of our paper, ESPNet (accepted for publication in ECCV'18).

Sample results

Check our project page for more qualitative results (videos).

Click on the below sample image to view the segmentation results on YouTube.

Structure of this repository

This repository is organized as:

  • train This directory contains the source code for trainig the ESPNet-C and ESPNet models.
  • test This directory contains the source code for evaluating our model on RGB Images.
  • pretrained This directory contains the pre-trained models on the CityScape dataset
    • encoder This directory contains the pretrained ESPNet-C models
    • decoder This directory contains the pretrained ESPNet models

Performance on the CityScape dataset

Our model ESPNet achives an class-wise mIOU of 60.336 and category-wise mIOU of 82.178 on the CityScapes test dataset and runs at

  • 112 fps on the NVIDIA TitanX (30 fps faster than ENet)
  • 9 FPS on TX2
  • With the same number of parameters as ENet, our model is 2% more accurate

Performance on the CamVid dataset

Our model achieves an mIOU of 55.64 on the CamVid test set. We used the dataset splits (train/val/test) provided here. We trained the models at a resolution of 480x360. For comparison with other models, see SegNet paper.

Note: We did not use the 3.5K dataset for training which was used in the SegNet paper.

Model mIOU Class avg.
ENet 51.3 68.3
SegNet 55.6 65.2
ESPNet 55.64 68.30


To run this code, you need to have following libraries:

  • OpenCV - We tested our code with version > 3.0.
  • PyTorch - We tested with v0.3.0
  • Python - We tested our code with Pythonv3. If you are using Python v2, please feel free to make necessary changes to the code.

We recommend to use Anaconda. We have tested our code on Ubuntu 16.04.


If ESPNet is useful for your research, then please cite our paper.

  title={ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation},
  author={Sachin Mehta, Mohammad Rastegari, Anat Caspi, Linda Shapiro, and Hannaneh Hajishirzi},


Assertion error with class labels (t >= 0 && t < n_classes).

If you are getting an assertion error with class labels, then please check the number of class labels defined in the label images. You can do this as:

import cv2
import numpy as np
labelImg = cv2.imread(<label_filename.png>, 0)
unique_val_arr = np.unique(labelImg)

The values inside unique_val_arr should be between 0 and total number of classes in the dataset. If this is not the case, then pre-process your label images. For example, if the label iamge contains 255 as a value, then you can ignore these values by mapping it to an undefined or background class as:

labelImg[labelImg == 255] = <undefined class id>
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].