TensorFlow CMake/C++ Collection

Looking at the official docs: What do you see? The usual fare? Now, guess what: This is a bazel-free zone. We use CMake here!

This collection contains reliable and dead-simple examples to use TensorFlow in C, C++, Go and Python: load a pre-trained model or compile a custom operation with or without CUDA. All builds are tested against the most recent stable TensorFlow version and rely on CMake with a custom FindTensorFlow.cmake. This cmake file includes common work arounds for bugs in specific TF versions.

TensorFlow	Status
1.14.0
1.13.1
1.12.0
1.11.0
1.10.0
1.9.0

The repository contains the following examples.

Custom Operation

This example illustrates the process of creating a custom operation using C++/CUDA and CMake. It is not intended to show an implementation obtaining peak-performance. Instead, it is just a boilerplate-template.

[email protected] $ pip install tensorflow-gpu --user # solely the pip package is needed
[email protected] $ cd custom_op/user_ops
[email protected] $ cmake .
[email protected] $ make
[email protected] $ python test_matrix_add.py
[email protected] $ cd ..
[email protected] $ python example.py

TensorFlow Graph within C++

This example illustrates the process of loading an image (using OpenCV or TensorFlow), resizing the image saving the image as a JPG or PNG (using OpenCV or TensorFlow).

[email protected] $ cd examples/resize
[email protected] $ export TENSORFLOW_BUILD_DIR=...
[email protected] $ export TENSORFLOW_SOURCE_DIR=...
[email protected] $ cmake .
[email protected] $ make

TensorFlow-Serving

There are two examples demonstrating the handling of TensorFlow-Serving: using a vector input and using an encoded image input.

[email protected] $ CHOOSE=basic # or image
[email protected] $ cd serving/${CHOOSE}/training
[email protected] $ python create.py # create some model
[email protected] $ cd serving/server/
[email protected] $ ./run.sh # start server

# some some queries

[email protected] $ cd client/bash
[email protected] $ ./client.sh
[email protected] $ cd client/python
# for the basic-example
[email protected] $ python client_rest.py
[email protected] $ python client_grpc.py
# for the image-example
[email protected] $ python client_rest.py /path/to/img.[png,jpg]
[email protected] $ python client_grpc.py /path/to/img.[png,jpg]

Inference

Create a model in Python, save the graph to disk and load it in C/C+/Go/Python to perform inference. As these examples are based on the TensorFlow C-API they require the libtensorflow_cc.so library which is not shipped in the pip-package (tensorfow-gpu). Hence, you will need to build TensorFlow from source beforehand, e.g.,

[email protected] $ ls ${TENSORFLOW_SOURCE_DIR}

ACKNOWLEDGMENTS     bazel-genfiles      configure          pip
ADOPTERS.md         bazel-out           configure.py       py.pynano
ANDROID_NDK_HOME    bazel-tensorflow    configure.py.bkp   README.md
...
[email protected] $ cd ${TENSORFLOW_SOURCE_DIR}
[email protected] $  ./configure
[email protected] $  # ... or whatever options you used here
[email protected] $ bazel build -c opt --copt=-mfpmath=both --copt=-msse4.2 --config=cuda //tensorflow:libtensorflow.so
[email protected] $ bazel build -c opt --copt=-mfpmath=both --copt=-msse4.2 --config=cuda //tensorflow:libtensorflow_cc.so

[email protected] $ export TENSORFLOW_BUILD_DIR=/tensorflow_dist
[email protected] $ mkdir ${TENSORFLOW_BUILD_DIR}
[email protected] $ cp ${TENSORFLOW_SOURCE_DIR}/bazel-bin/tensorflow/*.so ${TENSORFLOW_BUILD_DIR}/
[email protected] $ cp ${TENSORFLOW_SOURCE_DIR}/bazel-genfiles/tensorflow/cc/ops/*.h ${TENSORFLOW_BUILD_DIR}/includes/tensorflow/cc/ops/

1. Save Model

We just run a very basic model

x = tf.placeholder(tf.float32, shape=[1, 2], name='input')
output = tf.identity(tf.layers.dense(x, 1), name='output')

Therefore, save the model like you regularly do. This is done in example.py besides some outputs

[email protected] $ python example.py

[<tf.Variable 'dense/kernel:0' shape=(2, 1) dtype=float32_ref>, <tf.Variable 'dense/bias:0' shape=(1,) dtype=float32_ref>]
input            [[1. 1.]]
output           [[2.1909506]]
dense/kernel:0   [[0.9070684]
 [1.2838823]]
dense/bias:0     [0.]

2. Run Inference

Python

[email protected] $ python python/inference.py

[<tf.Variable 'dense/kernel:0' shape=(2, 1) dtype=float32_ref>, <tf.Variable 'dense/bias:0' shape=(1,) dtype=float32_ref>]
input            [[1. 1.]]
output           [[2.1909506]]
dense/kernel:0   [[0.9070684]
 [1.2838823]]
dense/bias:0     [0.]

C++

[email protected] $ cd cc
[email protected] $ cmake .
[email protected] $ make
[email protected] $ cd ..
[email protected] $ ./cc/inference_cc

input           Tensor<type: float shape: [1,2] values: [1 1]>
output          Tensor<type: float shape: [1,1] values: [2.19095063]>
dense/kernel:0  Tensor<type: float shape: [2,1] values: [0.907068372][1.28388226]>
dense/bias:0    Tensor<type: float shape: [1] values: 0>

C

[email protected] $ cd c
[email protected] $ cmake .
[email protected] $ make
[email protected] $ cd ..
[email protected] $ ./c/inference_c

2.190951

Go

[email protected] $ go get github.com/tensorflow/tensorflow/tensorflow/go
[email protected] $ cd go
[email protected] $ ./build.sh
[email protected] $ cd ../
[email protected] $ ./inference_go

input           [[1 1]]
output          [[2.1909506]]
dense/kernel:0  [[0.9070684] [1.2838823]]
dense/bias:0    [0]