Adventures-in-TensorFlow-Lite

This repository contains notebooks that show the usage of TensorFlow Lite (TF Lite) for quantizing deep neural networks in TensorFlow 2. It is currently under active development so, there might be some inconsistencies in the description below.

About the notebooks

• A_tale_of_quantization.ipynb: A comprehensive notebook showing different ways to quantize a model in tf.keras (with fine-tuning). It includes both quantization-aware training as well as post-training quantization. This notebook is accompanied by this report that has some additional details and performance considerations. The notebook is best referred with this report.
• Custom_Image_Classification_EdgeTPU.ipynb: It shows how to use post-training quantization with a representative dataset to calibrate the dynamic ranges of activations. It also shows how to quantize the model in a way that is compatible with an Edge TPU USB Accelerator.
• DeepLabV3/DeepLab_TFLite_*.ipynb: These notebooks show how to convert several DeepLabV3 based segmentation models (trained on the PASCAL VOC 2012, ADE20k, and CityScapes datasets) to TF Lite and run inference with them.
• ESRGAN_TFLite.ipynb: Shows how to run inference with this ESRGAN module from TF Hub, convert it to TF Lite, run inference with the model. A bonus includes running inference with a distilled version of the main model (only 33 KB).
• Magenta_arbitrary_style_transfer_model_conversion.ipynb: Shows to how apply apply different post-training quantization schemes to the arbitrary style transfer model provided by Magenta. This generates image stylization models that produce higher-quality images than these ones. Inference notebook is available here - Style_Transfer_Demo_InceptionV3.ipynb. Additionally, in this notebook you can find how to run inference with the same TF Lite models but with dynamic shape support: Style_Transfer_Demo_InceptionV3_Dynamic_Shape.ipynb.
• Model_Pruning_in_Deep_Learning_with_tfmot.ipynb: Shows how to use the pruning APIs of TensorFlow Model Optimization toolkit along with TF Lite.
• Selfie2Anime_TFLite(50_Checkpoints).ipynb: Shows how to convert the UGATIT model (also known as Selfie2Anime GAN) to TF Lite. Inference notebook is available here. Here's a multi-part tutorial of this project - Part I, Part II, Part III. This was jointly done with Margaret.
• Semantic_Segmentation_+_Background_Removal_+_Style_Transfer.ipynb: Presents a demo on how to use multiple TF Lite models to run segmentation, remove background, and apply stylization.
• Style_Transfer_Demo.ipynb & Style_Transfer_Demo_InceptionV3.ipynb: Present interactive image stylization demos.
• TUNIT_Conversion_to_TF_Lite.ipynb: Shows the PyTorch -> TF Lite model conversion worflow (this is currently buggy).
• CartoonGAN_TFLite.ipynb: Shows to how to convert the CartoonGAN model (proposed here) to a TF Lite model. Includes how to run inference as well. Here's the official TensorFlow blog in case you are interested to know more.
• EAST_TFLite.ipynb: Shows how to convert the famous EAST model for text detection to TensorFlow Lite.
• Boundless_TFLite.ipynb: Shows how to convert the famous Boundless model for image extrapolation to TensorFlow Lite.

References

• Post-training quantization
• Quantization aware training with TensorFlow Model Optimization Toolkit

Accompanying materials

• Doing more with TF Lite
• Model optimization 101
• A few good stuff in TF Lite
• A Tale of Model Quantization in TF Lite
• Plunging into Model Pruning in Deep Learning
• Optimizing MobileDet for Mobile Deployments
• Peer Bonus Experiences: Building tiny models for the ML community with TensorFlow
• A Battle of Text Detectors for Mobile Deployments: CRAFT vs. EAST

Find the models on TF Hub

https://tfhub.dev/s?publisher=sayakpaul

Acknowledgements

• Arun Venkatesan
• Khanh LeViet
• Pulkit Bhuwalka