wanglouis49 / Pytorch Weights_pruning
PyTorch Implementation of Weights Pruning
Stars: ✭ 158
Programming Languages
python
139335 projects - #7 most used programming language
Labels
Projects that are alternatives of or similar to Pytorch Weights pruning
Compress
Compressing Representations for Self-Supervised Learning
Stars: ✭ 43 (-72.78%)
Mutual labels: model-compression
Ghostnet
CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.
Stars: ✭ 1,744 (+1003.8%)
Mutual labels: model-compression
Condensa
Programmable Neural Network Compression
Stars: ✭ 129 (-18.35%)
Mutual labels: model-compression
Awesome Knowledge Distillation
Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。
Stars: ✭ 1,031 (+552.53%)
Mutual labels: model-compression
Neuronblocks
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
Stars: ✭ 1,356 (+758.23%)
Mutual labels: model-compression
Tf2
An Open Source Deep Learning Inference Engine Based on FPGA
Stars: ✭ 113 (-28.48%)
Mutual labels: model-compression
Knowledge Distillation Pytorch
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
Stars: ✭ 986 (+524.05%)
Mutual labels: model-compression
Ld Net
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Stars: ✭ 148 (-6.33%)
Mutual labels: model-compression
Nni
An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
Stars: ✭ 10,698 (+6670.89%)
Mutual labels: model-compression
Pretrained Language Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Stars: ✭ 2,033 (+1186.71%)
Mutual labels: model-compression
Keras model compression
Model Compression Based on Geoffery Hinton's Logit Regression Method in Keras applied to MNIST 16x compression over 0.95 percent accuracy.An Implementation of "Distilling the Knowledge in a Neural Network - Geoffery Hinton et. al"
Stars: ✭ 59 (-62.66%)
Mutual labels: model-compression
Micronet
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
Stars: ✭ 1,232 (+679.75%)
Mutual labels: model-compression
Awesome Model Compression
papers about model compression
Stars: ✭ 119 (-24.68%)
Mutual labels: model-compression
Awesome Pruning
A curated list of neural network pruning resources.
Stars: ✭ 1,017 (+543.67%)
Mutual labels: model-compression
Collaborative Distillation
PyTorch code for our CVPR'20 paper "Collaborative Distillation for Ultra-Resolution Universal Style Transfer"
Stars: ✭ 138 (-12.66%)
Mutual labels: model-compression
Model Optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
Stars: ✭ 992 (+527.85%)
Mutual labels: model-compression
Hawq
Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.
Stars: ✭ 108 (-31.65%)
Mutual labels: model-compression
Amc Models
[ECCV 2018] AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Stars: ✭ 154 (-2.53%)
Mutual labels: model-compression
Microexpnet
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images
Stars: ✭ 121 (-23.42%)
Mutual labels: model-compression
Neural Network Pruning PyTorch Implementation
Luyu Wang & Gavin Ding
Borealis AI
Motivation
Neural network pruning has become a trendy research topic, but we haven't found an easy to use PyTorch implementation. We want to take advantage of the power of PyTorch and build pruned networks to study their properties.
Note: this implementation is not aiming at obtaining computational efficiency but to offer convenience for studying properties of pruned networks. Discussions on how to have an efficient implementation is welcome. Thanks!
High-level idea
- We write wrappers on PyTorch Linear and Conv2d layers.
- For each layer, once a binary mask tensor is computed, it is multiplied with the actual weights tensor on the forward pass.
- Multiplying the mask is a differentiable operation and the backward pass is handed by automatic differentiation (no explicit coding needed).
Pruning methods
Weight pruning
Han et al propose to compress deep learning models via weights pruning Han et al, NIPS 2015. This repo is an implementation in PyTorch. The pruning method is replaced by the "class-blinded" method mentioned in See et al, CoNLL 2016, which is much easier to implement and has better performance as well.
Filter pruning
Pruning convolution filters has the advantage that it is more hardware friendly. We also implement the "minimum weight'' approach in Molchanov et al, ICLR 2017
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].