All Projects → Zhen-Dong → BitPack

Zhen-Dong / BitPack

Licence: MIT license
BitPack is a practical tool to efficiently save ultra-low precision/mixed-precision quantized models.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to BitPack

ATMC
[NeurIPS'2019] Shupeng Gui, Haotao Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, Ji Liu, “Model Compression with Adversarial Robustness: A Unified Optimization Framework”
Stars: ✭ 41 (+13.89%)
Mutual labels:  quantization, model-compression
Model Optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
Stars: ✭ 992 (+2655.56%)
Mutual labels:  quantization, model-compression
Pretrained Language Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Stars: ✭ 2,033 (+5547.22%)
Mutual labels:  quantization, model-compression
torch-model-compression
针对pytorch模型的自动化模型结构分析和修改工具集,包含自动分析模型结构的模型压缩算法库
Stars: ✭ 126 (+250%)
Mutual labels:  quantization, model-compression
Awesome Ml Model Compression
Awesome machine learning model compression research papers, tools, and learning material.
Stars: ✭ 166 (+361.11%)
Mutual labels:  quantization, model-compression
Paddleslim
PaddleSlim is an open-source library for deep model compression and architecture search.
Stars: ✭ 677 (+1780.56%)
Mutual labels:  quantization, model-compression
Awesome Automl And Lightweight Models
A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.
Stars: ✭ 691 (+1819.44%)
Mutual labels:  quantization, model-compression
Micronet
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
Stars: ✭ 1,232 (+3322.22%)
Mutual labels:  quantization, model-compression
Tf2
An Open Source Deep Learning Inference Engine Based on FPGA
Stars: ✭ 113 (+213.89%)
Mutual labels:  quantization, model-compression
Hawq
Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.
Stars: ✭ 108 (+200%)
Mutual labels:  quantization, model-compression
Awesome Ai Infrastructures
Infrastructures™ for Machine Learning Training/Inference in Production.
Stars: ✭ 223 (+519.44%)
Mutual labels:  quantization, model-compression
Kd lib
A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.
Stars: ✭ 173 (+380.56%)
Mutual labels:  quantization, model-compression
ZAQ-code
CVPR 2021 : Zero-shot Adversarial Quantization (ZAQ)
Stars: ✭ 59 (+63.89%)
Mutual labels:  quantization, model-compression
memory
Memory game 🎴
Stars: ✭ 24 (-33.33%)
Mutual labels:  memory
doc
Get usage and health data about your Node.js process.
Stars: ✭ 17 (-52.78%)
Mutual labels:  memory
slimarray
SlimArray compresses uint32 into several bits, by using a polynomial to describe overall trend of an array.
Stars: ✭ 39 (+8.33%)
Mutual labels:  memory
fastT5
⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
Stars: ✭ 421 (+1069.44%)
Mutual labels:  quantization
kvs
Lightweight key-value storage library for Browser, Node.js, and In-Memory.
Stars: ✭ 126 (+250%)
Mutual labels:  memory
nodejs
Node.js in-process collectors for Instana
Stars: ✭ 66 (+83.33%)
Mutual labels:  memory
libmem
Advanced Game Hacking Library for C/C++, Rust and Python (Windows/Linux/FreeBSD) (Process/Memory Hacking) (Hooking/Detouring) (Cross Platform) (x86/x64/ARM/ARM64) (DLL/SO Injection) (Internal/External)
Stars: ✭ 336 (+833.33%)
Mutual labels:  memory

BitPack

BitPack is a practical tool that can efficiently save quantized neural network models with mixed bitwidth.

Installation

  • PyTorch version >= 1.4.0
  • Python version >= 3.5
  • To install Bitpack simply run:
git clone https://github.com/Zhen-Dong/BitPack.git
cd BitPack

Usage

  • We can use BitPack pack.py to save integer checkpoints with various bitwidth, and use BitPack unpack.py to load the packed checkpoint, as shown in the demo.
  • To pack integer values that are saved in floating point format, add --force-pack-fp in the command.
  • To directly save packed checkpoint in PyTorch, please use save_quantized_state_dict() and load_quantized_state_dict() in pytorch_interface.py. If you don't want to operate jointly on state_dict, then codes inside the for loop of those two functions can be applied on every quantized tensor (ultra low-precision integer tensors) in various quantization frameworks.



Quick Start

BitPack is handy to use on various quantization frameworks. Here we show a demo that applying BitPack to save mixed-precision model generated by HAWQ.

export CUDA_VISIBLE_DEVICES=0
python pack.py --input-int-file quantized_checkpoint.pth.tar --force-pack-fp
python unpack.py --input-packed-file packed_quantized_checkpoint.pth.tar --original-int-file quantized_checkpoint.pth.tar

To get a better sense of how BitPack works, we provide a simple test that compares the original tensor, the packed tensor, and the unpacked tensor in details.

cd bitpack
python bitpack_utils.py

Results of BitPack on ResNet50

Original Precision Quantization Original Size(MB) Packed Size(MB) Compression Ratio
Floating Point Mixed-Precision(4bit/8bit) 102 13.8 7.4x
8-bit Mixed-Precision(2bit/8bit) 26 7.9 3.3x

Special Notes

  • unpack.py can be used for checking correctness. It loads and unpacks the packed model, and then compares it with the original model.

License

BitPack is released under the MIT license.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].