All Projects → XifengGuo → Capsnet Keras

XifengGuo / Capsnet Keras

Licence: mit
A Keras implementation of CapsNet in NIPS2017 paper "Dynamic Routing Between Capsules". Now test error = 0.34%.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Capsnet Keras

Capsnet Tensorflow
A Tensorflow implementation of CapsNet(Capsules Net) in paper Dynamic Routing Between Capsules
Stars: ✭ 3,776 (+55.52%)
Mutual labels:  capsule-network, capsnet, capsule, dynamic-routing
CapsNet-tensorflow-jupyter
A simple tensorflow implementation of CapsNet (by Dr. G. Hinton), based on my understanding. This repository is built with an aim to simplify the concept, implement and understand it.
Stars: ✭ 16 (-99.34%)
Mutual labels:  capsule, dynamic-routing, capsnet, capsule-network
capsnet.mxnet
MXNet implementation of CapsNet
Stars: ✭ 30 (-98.76%)
Mutual labels:  capsule, dynamic-routing, capsnet
Capsule Net Pytorch
[NO MAINTENANCE INTENDED] A PyTorch implementation of CapsNet architecture in the NIPS 2017 paper "Dynamic Routing Between Capsules".
Stars: ✭ 158 (-93.49%)
Mutual labels:  capsule-network, capsnet
CapsNet-Fashion-MNIST
Capsule Network on Fashion MNIST dataset
Stars: ✭ 93 (-96.17%)
Mutual labels:  capsnet, capsnet-keras
me recognition
CapsuleNet for Micro-expression Recognition (IEEE FG 2019)
Stars: ✭ 56 (-97.69%)
Mutual labels:  capsule, capsule-network
capsules-tensorflow
Another implementation of Hinton's capsule networks in tensorflow.
Stars: ✭ 18 (-99.26%)
Mutual labels:  capsnet, capsule-network
CapsNet
Empirical studies on Capsule Network representation and improvements implemented with PyTorch.
Stars: ✭ 39 (-98.39%)
Mutual labels:  capsnet, capsule-network
Capslayer
CapsLayer: An advanced library for capsule theory
Stars: ✭ 351 (-85.54%)
Mutual labels:  capsule-network, capsnet
Variational Capsule Routing
Official Pytorch code for (AAAI 2020) paper "Capsule Routing via Variational Bayes", https://arxiv.org/pdf/1905.11455.pdf
Stars: ✭ 84 (-96.54%)
Mutual labels:  capsule-network, capsnet
Capsnet Visualization
🎆 A visualization of the CapsNet layers to better understand how it works
Stars: ✭ 371 (-84.72%)
Mutual labels:  capsule-network, capsnet
CapsuleLayer
PyTorch Capsule Layer
Stars: ✭ 28 (-98.85%)
Mutual labels:  capsule, capsnet
Capsnet Traffic Sign Classifier
A Tensorflow implementation of CapsNet(Capsules Net) apply on german traffic sign dataset
Stars: ✭ 166 (-93.16%)
Mutual labels:  capsule-network, capsnet
Capsule Gan
Code for my Master thesis on "Capsule Architecture as a Discriminator in Generative Adversarial Networks".
Stars: ✭ 120 (-95.06%)
Mutual labels:  capsule-network, capsnet
Capsnet Nlp
CapsNet for NLP
Stars: ✭ 66 (-97.28%)
Mutual labels:  capsnet
Capse
A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization (NAACL 2019)
Stars: ✭ 114 (-95.3%)
Mutual labels:  capsule-network
Nn encapsulation
Capsule research with our trivial contribution
Stars: ✭ 53 (-97.82%)
Mutual labels:  capsule-network
Lung Diseases Classifier
Diseases Detection from NIH Chest X-ray data
Stars: ✭ 52 (-97.86%)
Mutual labels:  capsule-network
Deep Learning With Csharp And Cntk
Deep Learning with C# and CNTK
Stars: ✭ 108 (-95.55%)
Mutual labels:  capsule-network
Capsgnn
A PyTorch implementation of "Capsule Graph Neural Network" (ICLR 2019).
Stars: ✭ 1,008 (-58.48%)
Mutual labels:  capsule-network

CapsNet-Keras

License

A Keras (branch tf2.2 supports TensorFlow 2) implementation of CapsNet in the paper:
Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. Dynamic Routing Between Capsules. NIPS 2017
The current average test error = 0.34% and best test error = 0.30%.

Differences with the paper:

  • We use the learning rate decay with decay factor = 0.9 and step = 1 epoch,
    while the paper did not give the detailed parameters (or they didn't use it?).
  • We only report the test errors after 50 epochs training.
    In the paper, I suppose they trained for 1250 epochs according to Figure A.1? Sounds crazy, maybe I misunderstood.
  • We use MSE (mean squared error) as the reconstruction loss and the coefficient for the loss is lam_recon=0.0005*784=0.392.
    This should be equivalent with using SSE (sum squared error) and lam_recon=0.0005 as in the paper.

Warnning

Please use Keras==2.0.7 with TensorFlow==1.2 backend, or the K.batch_dot function may not work correctly.

However, if you use Tensorflow>=2.0, then checkout branch tf2.2

Usage

Step 1. Clone this repository to local.

git clone https://github.com/XifengGuo/CapsNet-Keras.git capsnet-keras
cd capsnet-keras
git checkout tf2.2 # Only if use Tensorflow>=2.0

Step 2. Install Keras==2.0.7 with TensorFlow==1.2 backend.

pip install tensorflow-gpu==1.2
pip install keras==2.0.7

or install Tensorflow>=2.0

pip install tensorflow==2.2

Step 3. Train a CapsNet on MNIST

Training with default settings:

python capsulenet.py

More detailed usage run for help:

python capsulenet.py -h

Step 4. Test a pre-trained CapsNet model

Suppose you have trained a model using the above command, then the trained model will be saved to result/trained_model.h5. Now just launch the following command to get test results.

$ python capsulenet.py -t -w result/trained_model.h5

It will output the testing accuracy and show the reconstructed images. The testing data is same as the validation data. It will be easy to test on new data, just change the code as you want.

You can also just download a model I trained from https://pan.baidu.com/s/1sldqQo1 or https://drive.google.com/open?id=1A7pRxH7iWzYZekzr-O0nrwqdUUpUpkik

Step 5. Train on multi gpus

This requires Keras>=2.0.9. After updating Keras:

python capsulenet-multi-gpu.py --gpus 2

It will automatically train on multi gpus for 50 epochs and then output the performance on test dataset. But during training, no validation accuracy is reported.

Results

Test Errors

CapsNet classification test error on MNIST. Average and standard deviation results are reported by 3 trials. The results can be reproduced by launching the following commands.

python capsulenet.py --routings 1 --lam_recon 0.0    #CapsNet-v1   
python capsulenet.py --routings 1 --lam_recon 0.392  #CapsNet-v2
python capsulenet.py --routings 3 --lam_recon 0.0    #CapsNet-v3 
python capsulenet.py --routings 3 --lam_recon 0.392  #CapsNet-v4
Method Routing Reconstruction MNIST (%) Paper
Baseline -- -- -- 0.39
CapsNet-v1 1 no 0.39 (0.024) 0.34 (0.032)
CapsNet-v2 1 yes 0.36 (0.009) 0.29 (0.011)
CapsNet-v3 3 no 0.40 (0.016) 0.35 (0.036)
CapsNet-v4 3 yes 0.34 (0.016) 0.25 (0.005)

Losses and accuracies:

Training Speed

About 100s / epoch on a single GTX 1070 GPU.
About 80s / epoch on a single GTX 1080Ti GPU.
About 55s / epoch on two GTX 1080Ti GPU by using capsulenet-multi-gpu.py.

Reconstruction result

The result of CapsNet-v4 by launching

python capsulenet.py -t -w result/trained_model.h5

Digits at top 5 rows are real images from MNIST and digits at bottom are corresponding reconstructed images.

Manipulate latent code

python capsulenet.py -t --digit 5 -w result/trained_model.h5 

For each digit, the ith row corresponds to the ith dimension of the capsule, and columns from left to right correspond to adding [-0.25, -0.2, -0.15, -0.1, -0.05, 0, 0.05, 0.1, 0.15, 0.2, 0.25] to the value of one dimension of the capsule.

As we can see, each dimension has caught some characteristics of a digit. The same dimension of different digit capsules may represent different characteristics. This is because that different digits are reconstructed from different feature vectors (digit capsules). These vectors are mutually independent during reconstruction.

Other Implementations

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].