TropComplique / Knowledge Distillation Keras
A machine learning experiment
Stars: ✭ 160
Projects that are alternatives of or similar to Knowledge Distillation Keras
Computervision Recipes
Best Practices, code samples, and documentation for Computer Vision.
Stars: ✭ 8,214 (+5033.75%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Image classifier
CNN image classifier implemented in Keras Notebook 🖼️.
Stars: ✭ 139 (-13.12%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Keras transfer cifar10
Object classification with CIFAR-10 using transfer learning
Stars: ✭ 120 (-25%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Pytorch Image Classification
Tutorials on how to implement a few key architectures for image classification using PyTorch and TorchVision.
Stars: ✭ 272 (+70%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Pba
Efficient Learning of Augmentation Policy Schedules
Stars: ✭ 461 (+188.13%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Computer Vision
Programming Assignments and Lectures for Stanford's CS 231: Convolutional Neural Networks for Visual Recognition
Stars: ✭ 408 (+155%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Deep Learning With Python
Deep learning codes and projects using Python
Stars: ✭ 195 (+21.88%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Deep learning projects
Stars: ✭ 28 (-82.5%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Fakeimagedetector
Image Tampering Detection using ELA and CNN
Stars: ✭ 93 (-41.87%)
Mutual labels: jupyter-notebook, convolutional-neural-networks, image-classification
Protest Detection Violence Estimation
Implementation of the model used in the paper Protest Activity Detection and Perceived Violence Estimation from Social Media Images (ACM Multimedia 2017)
Stars: ✭ 114 (-28.75%)
Mutual labels: jupyter-notebook, image-classification
Pytorch Dc Tts
Text to Speech with PyTorch (English and Mongolian)
Stars: ✭ 122 (-23.75%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Simpsonrecognition
Detect and recognize The Simpsons characters using Keras and Faster R-CNN
Stars: ✭ 131 (-18.12%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Deep learning notes
a collection of my notes on deep learning
Stars: ✭ 112 (-30%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Sigver wiwd
Learned representation for Offline Handwritten Signature Verification. Models and code to extract features from signature images.
Stars: ✭ 112 (-30%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Cs231n Convolutional Neural Networks Solutions
Assignment solutions for the CS231n course taught by Stanford on visual recognition. Spring 2017 solutions are for both deep learning frameworks: TensorFlow and PyTorch.
Stars: ✭ 110 (-31.25%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Alexnet Experiments Keras
Code examples for training AlexNet using Keras and Theano
Stars: ✭ 109 (-31.87%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Tensorflow realtime multi Person pose estimation
Multi-Person Pose Estimation project for Tensorflow 2.0 with a small and fast model based on MobilenetV3
Stars: ✭ 129 (-19.37%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Reproduce Chexnet
Reproduce CheXNet
Stars: ✭ 136 (-15%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Deep Steganography
Hiding Images within other images using Deep Learning
Stars: ✭ 136 (-15%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Tfvos
Semi-Supervised Video Object Segmentation (VOS) with Tensorflow. Includes implementation of *MaskRNN: Instance Level Video Object Segmentation (NIPS 2017)* as part of the NIPS Paper Implementation Challenge.
Stars: ✭ 151 (-5.62%)
Mutual labels: jupyter-notebook, convolutional-neural-networks
Knowledge distillation with Keras
Keras implementation of Hinton's knowledge distillation (KD), a way of transferring knowledge from a large model into a smaller model.
Summary
- I use Caltech-256 dataset for a demonstration of the technique.
- I transfer knowledge from Xception to MobileNet-0.25 and SqueezeNet v1.1.
- Results:
| model | accuracy, % | top 5 accuracy, % | logloss |
|---|---|---|---|
| Xception | 82.3 | 94.7 | 0.705 |
| MobileNet-0.25 | 64.6 | 85.9 | 1.455 |
| MobileNet-0.25 with KD | 66.2 | 86.7 | 1.464 |
| SqueezeNet v1.1 | 67.2 | 86.5 | 1.555 |
| SqueezeNet v1.1 with KD | 68.9 | 87.4 | 1.297 |
Implementation details
- I use pretrained on ImageNet models.
- For validation I use 20 images from each category.
- For training I use 100 images from each category.
- I use random crops and color augmentation to balance the dataset.
- I resize all images to 299x299.
- In all models I train the last two layers.
Notes on flow_from_directory
I use three slightly different versions of Keras' ImageDataGenerator.flow_from_directory:
- original version for initial training of Xception and MobileNet.
- ver1 for getting logits from Xception. Now
DirectoryIterator.nextalso outputs image names. - ver2 for knowledge transfer. Here
DirectoryIterator.nextpacks logits with hard true targets. All three versions only differ inDirectoryIterator.nextfunction.
Requirements
- Python 3.5
- Keras 2.0.6
- torchvision, Pillow
- numpy, pandas, tqdm
References
[1] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, Distilling the Knowledge in a Neural Network
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].