All Projects → ajbrock → Neural Photo Editor

ajbrock / Neural Photo Editor

Licence: mit
A simple interface for editing natural photos with generative neural networks.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Neural Photo Editor

Fewshot gan Unet3d
Tensorflow implementation of our paper: Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
Stars: ✭ 272 (-86.61%)
Mutual labels:  convolutional-neural-networks, gans
Iseebetter
iSeeBetter: Spatio-Temporal Video Super Resolution using Recurrent-Generative Back-Projection Networks | Python3 | PyTorch | GANs | CNNs | ResNets | RNNs | Published in Springer Journal of Computational Visual Media, September 2020, Tsinghua University Press
Stars: ✭ 202 (-90.05%)
Mutual labels:  convolutional-neural-networks, gans
Vdcnn
Implementation of Very Deep Convolutional Neural Network for Text Classification
Stars: ✭ 158 (-92.22%)
Mutual labels:  convolutional-neural-networks
Code for fun
Some of my scripts to automate the boring stuff around me
Stars: ✭ 167 (-91.78%)
Mutual labels:  convolutional-neural-networks
Iresnet
Improved Residual Networks (https://arxiv.org/pdf/2004.04989.pdf)
Stars: ✭ 163 (-91.97%)
Mutual labels:  convolutional-neural-networks
Deblurgan
Image Deblurring using Generative Adversarial Networks
Stars: ✭ 2,033 (+0.1%)
Mutual labels:  convolutional-neural-networks
A Journey Into Convolutional Neural Network Visualization
A journey into Convolutional Neural Network visualization
Stars: ✭ 165 (-91.88%)
Mutual labels:  convolutional-neural-networks
A Pytorch Tutorial To Super Resolution
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network | a PyTorch Tutorial to Super-Resolution
Stars: ✭ 157 (-92.27%)
Mutual labels:  gans
Antialiased Cnns
Repository has been moved: https://github.com/adobe/antialiased-cnns
Stars: ✭ 169 (-91.68%)
Mutual labels:  convolutional-neural-networks
Dynamic Image Nets
Dynamic Image Networks for Action Recognition
Stars: ✭ 163 (-91.97%)
Mutual labels:  convolutional-neural-networks
Tensorflow Cnn Finetune
Finetuning AlexNet, VGGNet and ResNet with TensorFlow
Stars: ✭ 166 (-91.83%)
Mutual labels:  convolutional-neural-networks
Tf Adnet Tracking
Deep Object Tracking Implementation in Tensorflow for 'Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning(CVPR 2017)'
Stars: ✭ 162 (-92.02%)
Mutual labels:  convolutional-neural-networks
Motion Sense
MotionSense Dataset for Human Activity and Attribute Recognition ( time-series data generated by smartphone's sensors: accelerometer and gyroscope)
Stars: ✭ 159 (-92.17%)
Mutual labels:  convolutional-neural-networks
Aulas
Aulas da Escola de Inteligência Artificial de São Paulo
Stars: ✭ 166 (-91.83%)
Mutual labels:  gans
Simple cnn
Simple Convolutional Neural Network Library
Stars: ✭ 158 (-92.22%)
Mutual labels:  convolutional-neural-networks
Deep image prior
Image reconstruction done with untrained neural networks.
Stars: ✭ 168 (-91.73%)
Mutual labels:  convolutional-neural-networks
Sign Language Interpreter Using Deep Learning
A sign language interpreter using live video feed from the camera.
Stars: ✭ 157 (-92.27%)
Mutual labels:  convolutional-neural-networks
Cs231n
My assignment solutions for CS231n - Convolutional Neural Networks for Visual Recognition
Stars: ✭ 162 (-92.02%)
Mutual labels:  convolutional-neural-networks
Visual Interaction Networks Pytorch
This's an implementation of deepmind Visual Interaction Networks paper using pytorch
Stars: ✭ 164 (-91.93%)
Mutual labels:  convolutional-neural-networks
2048 Deep Reinforcement Learning
Trained A Convolutional Neural Network To Play 2048 using Deep-Reinforcement Learning
Stars: ✭ 169 (-91.68%)
Mutual labels:  convolutional-neural-networks

Neural Photo Editor

A simple interface for editing natural photos with generative neural networks.

GUI1 GUI2 GUI3

This repository contains code for the paper "Neural Photo Editing with Introspective Adversarial Networks," and the Associated Video.

Installation

To run the Neural Photo Editor, you will need:

  • Python, likely version 2.7. You may be able to use early versions of Python2, but I'm pretty sure there's some incompatibilities with Python3 in here.
  • Theano, development version.
  • lasagne, development version.
  • I highly recommend cuDNN as speed is key, but it is not a dependency.
  • numpy, scipy, PIL, Tkinter and tkColorChooser, but it is likely that your python distribution already has those.

Running the NPE

By default, the NPE runs on IAN_simple. This is a slimmed-down version of the IAN without MDC or RGB-Beta blocks, which runs without lag on a laptop GPU with ~1GB of memory (GT730M)

If you're on a Windows machine, you will want to create a .theanorc file and at least set the flag FLOATX=float32.

If you're on a linux machine, you can just insert THEANO_FLAGS=floatX=float32 before the command line call.

If you don't have cuDNN, simply change line 56 of the NPE.py file from dnn=True to dnn=False. Note that I presently only have the non-cuDNN option working for IAN_simple.

Then, run the command:

python NPE.py

If you wish to use a different model, simply edit the line with "config path" in the NPE.py file.

You can make use of any model with an inference mechanism (VAE or ALI-based GAN).

Commands

  • You can paint the image by picking a color and painting on the image, or paint in the latent space canvas (the red and blue tiles below the image).
  • The long horizontal slider controls the magnitude of the latent brush, and the smaller horizontal slider controls the size of both the latent and the main image brush.
  • You can select different entries from the subset of the celebA validation set (included in this repository as an .npz) by typing in a number from 0-999 in the bottom left box and hitting "infer."
  • Use the reset button to return to the ground truth image.
  • Press "Update" to update the ground-truth image and corresponding reconstruction with the current image. Use "Infer" to return to an original ground truth image from the dataset.
  • Use the sample button to generate a random latent vector and corresponding image.
  • Use the scroll wheel to lighten or darken an image patch (equivalent to using a pure white or pure black paintbrush). Note that this automatically returns you to sample mode, and may require hitting "infer" rather than "reset" to get back to photo editing.

Training an IAN on celebA

You will need Fuel along with the 64x64 version of celebA. See here for instructions on downloading and preparing it.

If you wish to train a model, the IAN.py file contains the model configuration, and the train_IAN.py file contains the training code, which can be run like this:

python train_IAN.py IAN.py

By default, this code will save (and overwrite!) the weights to a .npz file with the same name as the config.py file (i.e. "IAN.py -> IAN.npz"), and will output a jsonl log of the training with metrics recorded after every chunk.

Use the --resume=True flag when calling to resume training a model--it will automatically pick up from the most recent epoch.

Sampling the IAN

You can generate a sample and reconstruction+interpolation grid with:

python sample_IAN.py IAN.py

Note that you will need matplotlib. to do so.

Known Issues/Bugs

My MADE layer currently only accepts hidden unit sizes that are equal to the size of the latent vector, which will present itself as a BAD_PARAM error.

Since the MADE really only acts as an autoregressive randomizer I'm not too worried about this, but it does bear looking into.

I messed around with the keywords for get_model, you'll need to deal with these if you wish to run any model other than IAN_simple through the editor.

Everything is presently just dumped into a single, unorganized directory. I'll be adding folders and cleaning things up soon.

Notes

Remainder of the IAN experiments (including SVHN) coming soon.

I've integrated the plat interface which makes the NPE itself independent of framework, so you should be able to run it with Blocks, TensorFlow, PyTorch, PyCaffe, what have you, by modifying the IAN class provided in models.py.

Acknowledgments

This code contains lasagne layers and other goodies adopted from a number of places:

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].