luoyetx / Mx Lsoftmax
Licence: bsd-3-clause
mxnet version of Large-Margin Softmax Loss for Convolutional Neural Networks.
Stars: ✭ 175
Programming Languages
python
139335 projects - #7 most used programming language
Labels
Projects that are alternatives of or similar to Mx Lsoftmax
Insightface Just Works
Insightface face detection and recognition model that just works out of the box.
Stars: ✭ 127 (-27.43%)
Mutual labels: mxnet
Facerecognition
This is an implematation project of face detection and recognition. The face detection using MTCNN algorithm, and recognition using LightenenCNN algorithm.
Stars: ✭ 137 (-21.71%)
Mutual labels: mxnet
Pyeco
python implementation of efficient convolution operators for tracking
Stars: ✭ 150 (-14.29%)
Mutual labels: mxnet
Mxnet.sharp
.NET Standard bindings for Apache MxNet with Imperative, Symbolic and Gluon Interface for developing, training and deploying Machine Learning models in C#. https://mxnet.tech-quantum.com/
Stars: ✭ 134 (-23.43%)
Mutual labels: mxnet
Single Path One Shot Nas Mxnet
Single Path One-Shot NAS MXNet implementation with full training and searching pipeline. Support both Block and Channel Selection. Searched models better than the original paper are provided.
Stars: ✭ 136 (-22.29%)
Mutual labels: mxnet
Machine Learning Using K8s
Train and Deploy Machine Learning Models on Kubernetes using Amazon EKS
Stars: ✭ 145 (-17.14%)
Mutual labels: mxnet
Nlp Pretrained Model
A collection of Natural language processing pre-trained models.
Stars: ✭ 122 (-30.29%)
Mutual labels: mxnet
Crnn Mxnet Chinese Text Recognition
An implementation of CRNN (CNN+LSTM+warpCTC) on MxNet for chinese text recognition
Stars: ✭ 161 (-8%)
Mutual labels: mxnet
Deep Face Alignment
The MXNet Implementation of Stacked Hourglass and Stacked SAT for Robust 2D and 3D Face Alignment
Stars: ✭ 134 (-23.43%)
Mutual labels: mxnet
Horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
Stars: ✭ 11,943 (+6724.57%)
Mutual labels: mxnet
Ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
Stars: ✭ 13,376 (+7543.43%)
Mutual labels: mxnet
Aognet
Code for CVPR 2019 paper: " Learning Deep Compositional Grammatical Architectures for Visual Recognition"
Stars: ✭ 132 (-24.57%)
Mutual labels: mxnet
Deep Learning Containers
AWS Deep Learning Containers (DLCs) are a set of Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.
Stars: ✭ 152 (-13.14%)
Mutual labels: mxnet
Aws Machine Learning University Accelerated Nlp
Machine Learning University: Accelerated Natural Language Processing Class
Stars: ✭ 1,695 (+868.57%)
Mutual labels: mxnet
Coach
Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms
Stars: ✭ 2,085 (+1091.43%)
Mutual labels: mxnet
Mobulaop
A Simple & Flexible Cross Framework Operators Toolkit
Stars: ✭ 161 (-8%)
Mutual labels: mxnet
Djl
An Engine-Agnostic Deep Learning Framework in Java
Stars: ✭ 2,262 (+1192.57%)
Mutual labels: mxnet
mx-lsoftmax
mxnet version of Large-Margin Softmax Loss for Convolutional Neural Networks.
Derivatives
I put all formula I used to calculate the derivatives below. You can check it by yourself. If there's a mistake, please do tell me or open an issue.
The derivatives doesn't include lambda in the paper, but the code does. Instead of using lambda to weight the original f_i_yi = |w_yi||x_i|cos(t), the code uses beta because lambda is a keyword for Python, but beta plays exactly the same role as lambda.
Gradient Check
Gradient check can be failed with data type float32 but ok with data type float64. So don't afraid to see gradient check failed.
Operator Performance
I implement the operator both in Python and C++(CUDA). The performance below is training LeNet on a single GTX1070 with parameters margin = 4, beta = 1. Notice the C++ implement can only run on GPU context.
| Batch Size | traditional fully connected | lsoftmax in Python | lsoftmax in C++(CUDA) |
|---|---|---|---|
| 128 | ~45000 samples / sec | 2800 ~ 3300 samples / sec | ~40000 samples / sec |
| 256 | ~54000 samples / sec | 3500 ~ 4200 samples / sec | ~47000 samples / sec |
Visualization
original softmax (traditional fully connected)
lsoftmax with margin = 2, beta = 100 and scale = 0.99
lsoftmax with margin = 3, beta = 100 and scale = 0.99
lsoftmax with margin = 4, beta = 100 and scale = 0.99
lsoftmax with margin = 6, beta = 1000 and scale = 0.99
References
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].