Neural-Style-Transfer-Papers

Selected papers, corresponding codes and pre-trained models in our review paper "Neural Style Transfer: A Review" [arXiv Version] [IEEE Version]

The corresponding OSF repository can be found at: https://osf.io/f8tu4/.

If I missed your paper in this review, please email me or just pull a request here. I am more than happy to add it. Thanks!

Citation

If you find this repository useful for your research, please consider citing

@article{jing2019neural,
  title={Neural Style Transfer: A Review},
  author={Jing, Yongcheng and Yang, Yezhou and Feng, Zunlei and Ye, Jingwen and Yu, Yizhou and Song, Mingli},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2019}
}

Please also consider citing our ECCV paper and AAAI (Oral) paper:

@inproceedings{jing2018stroke,
  title={Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields},
  author={Jing, Yongcheng and Liu, Yang and Yang, Yezhou and Feng, Zunlei and Yu, Yizhou and Tao, Dacheng and Song, Mingli},
  booktitle={ECCV},
  year={2018}
}

@inproceedings{jing2020dynamic,
  title={Dynamic Instance Normalization for Arbitrary Style Transfer},
  author={Jing, Yongcheng and Liu, Xiao and Ding, Yukang and Wang, Xinchao and Ding, Errui and Song, Mingli and Wen, Shilei},
  booktitle={AAAI},
  year={2020}
}

Thanks!

Framework

There is a recent nice NST framework called pystiche, developed by Philip Meier. If you are interested, please refer to https://github.com/pmeier/pystiche. A package that comprises reference implementations of NST papers with pystiche can be found at pystiche_papers (work in progress).

News!

• [June, 2019] Update the Images (TVCG) (.png) and Supplementary Material (TVCG) in the Materials. Warmly welcome to use Images (TVCG) for comparison results in your paper!

• [May, 2019] Our paper Neural Style Transfer: A Review has been accepted by TVCG as a regular paper. This repository will be updated soon.

• [July, 2018] Our paper Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields has been accepted by ECCV 2018. Our review will be updated correspondingly.

• [June, 2018] Upload a new version of our paper on arXiv which adds several missing papers (e.g., the work of Wang et al. ZM-Net: Real-time Zero-shot Image Manipulation Network).

• [Apr, 2018] We have released a new version of the paper with significant changes at: https://arxiv.org/pdf/1705.04058.pdf Appreciate the feedback!

• [Feb, 2018] Update the Images (Images_neuralStyleTransferReview_v2) in the Materials. Add the results of Li et al.'s NIPS 2017 paper.

• [Jan, 2018] Pre-trained models and all the content images, the style images, and the stylized results in the paper have been released.

Materials corresponding to Our Paper

✅ Supplementary Material (TVCG)

✅ Pre-trained Models

✅ Images (TVCG)(.png)

A Taxonomy of Current Methods

1. Image-Optimisation-Based Online Neural Methods

1.1. Parametric Neural Methods with Summary Statistics

✅ [A Neural Algorithm of Artistic Style] [Paper] (First Neural Style Transfer Paper)

❇️ Code:

• Torch-based
• TensorFlow-based
• TensorFlow-based with L-BFGS optimizer support
• Caffe-based
• Keras-based
• MXNet-based
• MatConvNet-based

✅ [Image Style Transfer Using Convolutional Neural Networks] [Paper] (CVPR 2016)

✅ [Incorporating Long-range Consistency in CNN-based Texture Generation] [Paper] (ICLR 2017)

❇️ Code:

•   Theano-based

✅ [Laplacian-Steered Neural Style Transfer] [Paper] (ACM MM 2017)

❇️ Code:

• Torch-based & TensorFlow-based

✅ [Demystifying Neural Style Transfer] [Paper] (Theoretical Explanation) (IJCAI 2017)

❇️ Code:

•   MXNet-based

✅ [Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses] [Paper]

1.2. Non-parametric Neural Methods with MRFs

✅ [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis] [Paper] (CVPR 2016)

❇️ Code:

• Torch-based

✅ [Arbitrary Style Transfer with Deep Feature Reshuffle] [Paper] (CVPR 2018)

2. Model-Optimisation-Based Offline Neural Methods

2.1. Per-Style-Per-Model Neural Methods

✅ [Perceptual Losses for Real-Time Style Transfer and Super-Resolution] [Paper] (ECCV 2016)

❇️ Code:

• Torch-based
• TensorFlow-based
• Chainer-based

❇️ Pre-trained Models:

• Torch-models
• Chainer-models

✅ [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images] [Paper] (ICML 2016)

❇️ Code:

• Torch-based
• TensorFlow-based

✅ [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks] [Paper] (ECCV 2016)

❇️ Code:

• Torch-based

2.2. Multiple-Style-Per-Model Neural Methods

✅ [A Learned Representation for Artistic Style] [Paper] (ICLR 2017)

❇️ Code:

• TensorFlow-based

✅ [Multi-style Generative Network for Real-time Transfer] [Paper]  （arXiv, 03/2017）

❇️ Code:

• PyTorch-based
• Torch-based

✅ [Diversified Texture Synthesis With Feed-Forward Networks] [Paper] (CVPR 2017)

❇️ Code:

•   Torch-based

✅ [StyleBank: An Explicit Representation for Neural Image Style Transfer] [Paper] (CVPR 2017)

2.3. Arbitrary-Style-Per-Model Neural Methods

✅ [Fast Patch-based Style Transfer of Arbitrary Style] [Paper]

❇️ Code:

• Torch-based

✅ [Exploring the Structure of a Real-time, Arbitrary Neural Artistic Stylization Network] [Paper] (BMVC 2017)

❇️ Code:

• TensorFlow-based

✅ [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization] [Paper] (ICCV 2017)

❇️ Code:

• Torch-based
• TensorFlow-based with Keras
• TensorFlow-based without Keras

✅ [Dynamic Instance Normalization for Arbitrary Style Transfer] [Paper] (AAAI 2020)

✅ [Universal Style Transfer via Feature Transforms] [Paper] (NIPS 2017)

❇️ Code:

• Torch-based
• TensorFlow-based
• PyTorch-based #1
• PyTorch-based #2

✅ [Meta Networks for Neural Style Transfer] [Paper] (CVPR 2018)

❇️ Code:

• Caffe-based

✅ [ZM-Net: Real-time Zero-shot Image Manipulation Network] [Paper]

✅ [Avatar-Net: Multi-Scale Zero-Shot Style Transfer by Feature Decoration] [Paper] (CVPR 2018)

❇️ Code:

• TensorFlow-based

✅ [Learning Linear Transformations for Fast Arbitrary Style Transfer] [Paper]

❇️ Code:

• PyTorch-based

Improvements and Extensions

✅ [Preserving Color in Neural Artistic Style Transfer] [Paper]

✅ [Controlling Perceptual Factors in Neural Style Transfer] [Paper] (CVPR 2017)

❇️ Code:

• Torch-based

✅ [Content-Aware Neural Style Transfer] [Paper]

✅ [Towards Deep Style Transfer: A Content-Aware Perspective] [Paper] (BMVC 2016)

✅ [Neural Doodle_Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artwork] [Paper]

✅ [Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artwork] [Paper]

❇️ Code:

• Torch-based

✅ [The Contextual Loss for Image Transformation with Non-Aligned Data] [Paper] (ECCV 2018)

❇️ Code:

• TensorFlow-based

✅ [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis] [Paper] (CVPR 2017)

❇️ Code:

• Torch-based

✅ [Instance Normalization：The Missing Ingredient for Fast Stylization] [Paper]

❇️ Code:

• Torch-based

✅ [A Style-Aware Content Loss for Real-time HD Style Transfer] [Paper] (ECCV 2018)

✅ [Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer] [Paper] (CVPR 2017)

❇️ Code:

• TensorFlow-based

✅ [Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields] [Paper] (ECCV 2018)

❇️ Code:

• TensorFlow-based

✅ [Depth-Preserving Style Transfer] [Paper]

❇️ Code:

• Torch-based

✅ [Depth-Aware Neural Style Transfer] [Paper] (NPAR 2017)

✅ [Neural Style Transfer: A Paradigm Shift for Image-based Artistic Rendering?] [Paper] (NPAR 2017)

✅ [Pictory: Combining Neural Style Transfer and Image Filtering] [Paper] (ACM SIGGRAPH 2017 Appy Hour)

✅ [Painting Style Transfer for Head Portraits Using Convolutional Neural Networks] [Paper] (SIGGRAPH 2016)

✅ [Son of Zorn's Lemma Targeted Style Transfer Using Instance-aware Semantic Segmentation] [Paper] (ICASSP 2017)

✅ [Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN] [Paper] (ACPR 2017)

✅ [Artistic Style Transfer for Videos] [Paper] (GCPR 2016)

❇️ Code:

• Torch-based

✅ [DeepMovie: Using Optical Flow and Deep Neural Networks to Stylize Movies] [Paper]

✅ [Characterizing and Improving Stability in Neural Style Transfer] [Paper]) (ICCV 2017)

✅ [Coherent Online Video Style Transfer] [Paper] (ICCV 2017)

✅ [Real-Time Neural Style Transfer for Videos] [Paper] (CVPR 2017)

✅ [A Common Framework for Interactive Texture Transfer] [Paper] (CVPR 2018)

✅ [Deep Photo Style Transfer] [Paper] (CVPR 2017)

❇️ Code:

• Torch-based
• TensorFlow-based

✅ [A Closed-form Solution to Photorealistic Image Stylization] [Paper] (ECCV 2018)

❇️ Code:

• PyTorch-based

✅ [Photorealistic Style Transfer via Wavelet Transforms] [Paper]

❇️ Code:

• PyTorch-based

✅ [Decoder Network Over Lightweight Reconstructed Feature for Fast Semantic Style Transfer] [Paper] (ICCV 2017)

✅ [Stereoscopic Neural Style Transfer] [Paper] (CVPR 2018)

✅ [Awesome Typography: Statistics-based Text Effects Transfer] [Paper] (CVPR 2017)

❇️ Code:

• Matlab-based

✅ [Neural Font Style Transfer] [Paper] (ICDAR 2017)

✅ [Rewrite: Neural Style Transfer For Chinese Fonts] [Project]

✅ [Separating Style and Content for Generalized Style Transfer] [Paper] (CVPR 2018)

✅ [Visual Attribute Transfer through Deep Image Analogy] [Paper] (SIGGRAPH 2017)

❇️ Code:

• Caffe-based

✅ [Fashion Style Generator] [Paper] (IJCAI 2017)

✅ [Deep Painterly Harmonization] [Paper]

❇️ Code:

• Torch-based

✅ [Fast Face-Swap Using Convolutional Neural Networks] [Paper] (ICCV 2017)

✅ [Learning Selfie-Friendly Abstraction from Artistic Style Images] [Paper] (ACML 2018)

✅ [Style Transfer with Adaptation to the Central Objects of the Scene] [Paper] (NEUROINFORMATICS 2019)

Application

✅ Prisma

✅ Ostagram

✅ AlterDraw

✅ Vinci

✅ Artisto

❇️ Code:

• Website code

✅ Deep Forger

✅ NeuralStyler

✅ Style2Paints

❇️ Code:

• Website code

Application Papers

✅ [Bringing Impressionism to Life with Neural Style Transfer in Come Swim] [Paper]

✅ [Imaging Novecento. A Mobile App for Automatic Recognition of Artworks and Transfer of Artistic Styles] [Paper]

✅ [ProsumerFX: Mobile Design of Image Stylization Components] [Paper]

✅ [Pictory - Neural Style Transfer and Editing with coreML] [Paper]

✅ [Tiny Transform Net for Mobile Image Stylization] [Paper] (ICMR 2017)

Blogs

✅ [Caffe2Go][https://code.facebook.com/posts/196146247499076/delivering-real-time-ai-in-the-palm-of-your-hand/]

✅ [Supercharging Style Transfer][https://research.googleblog.com/2016/10/supercharging-style-transfer.html]

✅ [Issue of Layer Chosen Strategy][http://yongchengjing.com/pdf/Issue_layerChosenStrategy_neuralStyleTransfer.pdf]

✅ [Picking an optimizer for Style Transfer][https://blog.slavv.com/picking-an-optimizer-for-style-transfer-86e7b8cba84b]

✅ [Enhanced Color Style Transfer (Photo-surrealism Style Transfer)] [Project]

Others

✅ [Conditional Fast Style Transfer Network] [Paper]

✅ [Unseen Style Transfer Based on a Conditional Fast Style Transfer Network] [Paper]

✅ [DeepStyleCam: A Real-time Style Transfer App on iOS] [Paper]