style_transfer

Data-parallel image stylization using Caffe. Implements A Neural Algorithm of Artistic Style [1].

Dependencies:

• Python 3.6+
• Caffe, with pycaffe compiled for Python 3.6+
• Python packages aiohttp, aiohttp_index, average, matplotlib, numpy, Pillow, scipy, and shared_ndarray.

The current preferred Python distribution for style_transfer is Anaconda (Python 3.6+ version). style_transfer will run faster with Anaconda than with other Python distributions due to its inclusion of the MKL BLAS (mathematics) library. In addition, if you are running Caffe without a GPU, style_transfer will run a great deal faster if compiled with MKL (BLAS := mkl in Makefile.config).

Cloud computing images are available with style_transfer and its dependencies preinstalled.

Command line arguments are documented in detail in the work-in-progress parameter usage guide.

Features

• The image is divided into tiles which are processed one per GPU at a time. Since the tiles can be sized so as to fit into GPU memory, this allows arbitrary size images to be processed—including print size. Tile seam suppression is applied after every iteration so that seams do not accumulate and become visible. (ex: --size 2048 --tile-size 1024)
• Images are processed at multiple scales. Each scale's final iterate is used as the initial iterate for the following scale. Processing a large image at smaller scales first markedly improves output quality.
• Multi-GPU support (ex: --devices 0 1 2 3). Four GPUs, for instance, can process four tiles at a time.
• Can perform simultaneous Deep Dream and image stylization.
• L-BFGS [2] and Adam (gradient descent) [4] optimizers.

Examples

The obligatory Golden Gate Bridge + The Starry Night (van Gogh) style transfer (big version):

Golden Gate Bridge + The Shipwreck of the Minotaur (Turner) (big version):

barn and pond (Cindy Branham) + The Banks of the River (Renoir) (big version):

Installation

pycaffe and Anaconda (Python 3.6 version)

On macOS (with Homebrew-provided Boost.Python):

ANACONDA_HOME := $(HOME)/anaconda3
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		$(ANACONDA_HOME)/include/python3.6m \
		$(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include
PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_LIB := $(ANACONDA_HOME)/lib

The exact name of the Boost.Python library will differ on Linux but the rest should be the same.

Building pycaffe for Python 3.7 (macOS)

On macOS, you can install Python 3 and Boost.Python using Homebrew:

brew install python
brew install numpy
brew install boost-python3

Then insert these lines into Caffe's Makefile.config to build against the Homebrew-provided Python 3.7:

PYTHON_DIR := /usr/local/opt/python3/Frameworks/Python.framework/Versions/3.7
PYTHON_LIBRARIES := boost_python37 python3.7m
PYTHON_INCLUDE := $(PYTHON_DIR)/include/python3.7m \
	/usr/local/lib/python3.7/site-packages/numpy/core/include
PYTHON_LIB := $(PYTHON_DIR)/lib

make pycaffe ought to compile the Python 3 bindings now.

Note that on macOS you currently have to set an environment variable before running style_transfer to prevent a crash on forking:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Building pycaffe for Python 3.5 (Ubuntu 16.04)

Note that Python 3.6+ is required now and these instructions need to be updated.

On Ubuntu 16.04, follow Caffe's Ubuntu 15.10/16.04 install guide. The required Makefile.config lines for Python 3.5 are:

PYTHON_LIBRARIES := boost_python-py35 python3.5m
PYTHON_INCLUDE := /usr/include/python3.5m \
                  /usr/local/lib/python3.5/dist-packages/numpy/core/include
PYTHON_LIB := /usr/lib

Installing style_transfer's Python dependencies (all systems)

Using pip:

pip3 install -Ur requirements.txt

References

[1] L. Gatys, A. Ecker, M. Bethge, "A Neural Algorithm of Artistic Style"

[2] D. Liu, J. Nocedal, "On the limited memory BFGS method for large scale optimization"

[3] A. Mahendran, A. Vedaldi, "Understanding Deep Image Representations by Inverting Them"

[4] D. Kingma, J. Ba, "Adam: A Method for Stochastic Optimization"

[5] K. Simonyan, A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition"