30 Open source projects that are alternatives of or similar to Awesome Knowledge Distillation

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)

A school management Software

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

The codes for recent knowledge distillation algorithms and benchmark results via TF2.0 low-level API

Model optimizer used in Adlik.

BigGAN; Knowledge Distillation; Black-Box; Fast Training; 16x compression

Focal and Global Knowledge Distillation for Detectors (CVPR 2022)

Efficient-Computing

noisy labels; missing labels; semi-supervised learning; entropy; uncertainty; robustness and generalisation.

This is the official implementation for the AAAI-2021 paper (Cross-Layer Distillation with Semantic Calibration).

Reimplementing cool papers in PyTorch...

A Fast Knowledge Distillation Framework for Visual Recognition

Adversarial Adaptation with Distillation for BERT Unsupervised Domain Adaptation

OpenMMLab Model Compression Toolbox and Benchmark.

Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals. WSDM 2021.

ZSKD with PyTorch

Mobile(iOS) Text-to-Image search powered by multimodal semantic representation models(e.g., OpenAI's CLIP). Accepted at NAACL 2022.

Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection

Intel® Neural Compressor (formerly known as Intel® Low Precision Optimization Tool), targeting to provide unified APIs for network compression technologies, such as low precision quantization, sparsity, pruning, knowledge distillation, across different deep learning frameworks to pursue optimal inference performance.

[ECCV 2020] Scale Adaptive Network: Learning to Learn Parameterized Classification Networks for Scalable Input Images

Localization Distillation for Dense Object Detection (CVPR 2022)

Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (ACM MM 2018)

Research code for ACL 2020 paper: "Distilling Knowledge Learned in BERT for Text Generation".

Official PyTorch Implementation of Embedding Transfer with Label Relaxation for Improved Metric Learning, CVPR 2021

Code and resources on scalable and efficient Graph Neural Networks

Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification

An Object Detection Knowledge Distillation framework powered by pytorch, now having SSD and yolov5.

Authors official PyTorch implementation of the "DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval" [IJCV 2022]

[IEEE Access] "Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-constrained Edge Computing Systems" and [ACM MobiCom HotEdgeVideo 2019] "Distilled Split Deep Neural Networks for Edge-assisted Real-time Systems"

Knowledge Distillation using Tensorflow