160 Open source projects that are alternatives of or similar to Pretrained Language Model

针对pytorch模型的自动化模型结构分析和修改工具集，包含自动分析模型结构的模型压缩算法库

PaddleSlim is an open-source library for deep model compression and architecture search.

BitPack is a practical tool to efficiently save ultra-low precision/mixed-precision quantized models.

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

CVPR 2021 : Zero-shot Adversarial Quantization (ZAQ)

Efficient-Computing

Infrastructures™ for Machine Learning Training/Inference in Production.

Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.

An Open Source Deep Learning Inference Engine Based on FPGA

Intel® Neural Compressor (formerly known as Intel® Low Precision Optimization Tool), targeting to provide unified APIs for network compression technologies, such as low precision quantization, sparsity, pruning, knowledge distillation, across different deep learning frameworks to pursue optimal inference performance.

Awesome machine learning model compression research papers, tools, and learning material.

[NeurIPS'2019] Shupeng Gui, Haotao Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, Ji Liu, “Model Compression with Adversarial Robustness: A Unified Optimization Framework”

A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.

Lightweight and Scalable framework that combines mainstream algorithms of Click-Through-Rate prediction based computational DAG, philosophy of Parameter Server and Ring-AllReduce collective communication.

Model Compression Based on Geoffery Hinton's Logit Regression Method in Keras applied to MNIST 16x compression over 0.95 percent accuracy.An Implementation of "Distilling the Knowledge in a Neural Network - Geoffery Hinton et. al"

A treasure chest for image classification powered by PaddlePaddle

Pytorch implementation of various Knowledge Distillation (KD) methods.

NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego

Training & Quantization of embedded friendly Deep Learning / Machine Learning / Computer Vision models

[CVPR2020] GhostNet: More Features from Cheap Operations

An Open-Source Package for Deep Learning to Hash (DeepHash)

A curated list of neural network pruning resources.

Data Efficient Model Compression

Palette quantization library that powers pngquant and other PNG optimizers

Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller

FrostNet: Towards Quantization-Aware Network Architecture Search

Simulate quantization and quantization aware training for MXNet-Gluon models.

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (CVPR 2019 Oral)

pytorch implementation of "Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks"

A repository that shares tuning results of trained models generated by TensorFlow / Keras. Post-training quantization (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization), Quantization-aware training. TensorFlow Lite. OpenVINO. CoreML. TensorFlow.js. TF-TRT. MediaPipe. ONNX. [.tflite,.h5,.pb,saved_model,tfjs,tftrt,mlmodel,.xml/.bin, .onnx]

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

Embedded and mobile deep learning research resources

reference pytorch code for named entity tagging

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Collections of model quantization algorithms

knowledge distillation papers

Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。

Lossy PNG compressor — pngquant command based on libimagequant library

Reducing the size of convolutional neural networks

Brevitas: quantization-aware training in PyTorch

Compressing Representations for Self-Supervised Learning

MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images

Must-read papers on deep learning to hash (DeepHash)

[ECCV 2018] AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Papers for deep neural network compression and acceleration

Powerful, automated analysis and design of quantum microwave chips & devices [Energy-Participation Ratio and more]

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

Dataflow compiler for QNN inference on FPGAs

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

QKeras: a quantization deep learning library for Tensorflow Keras

The codes for recent knowledge distillation algorithms and benchmark results via TF2.0 low-level API

SDK for TEE AI Stick (includes model training script, inference library, examples)

Easy-to-use UI for automatically sparsifying neural networks and creating sparsification recipes for better inference performance and a smaller footprint

[NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

Trainable models and NN optimization tools

Model optimizer used in Adlik.

MWPToolkit is an open-source framework for math word problem(MWP) solvers.