Is there a way to use make_tensor_proto without having to install the entire TensorFlow package?

Issue

I’m trying to serve a TensorFlow model (using TensorFlow Serving). It seems that in order to use the TensorFlow Serving APIs and make predictions, I need to use the make_tensor_proto method.

For my Docker image, I’m installing 1.3GB of the TensorFlow package just for that one method. Obviously, this is not ideal. So, I’m wondering if I can import make_tensor_proto from a different (leaner) package or use an alternative method.

Solution

You can follow the steps mentioned in the link below.

https://www.mux.com/blog/tuning-performance-of-tensorflow-serving-pipeline

Simplified steps:

1. Copy the tensor protos to the local machine.
2. Compile the protos to python module and place it under the project directory
3. Replace tensorflow library with proto pythonic functions.

From this line of code

import tensorflow as tf  
...
tensor = tf.contrib.util.make_tensor_proto(features)  
request.inputs['inputs'].CopyFrom(tensor)

To this line of code

from protos.tensorflow.core.framework import tensor_pb2  
from protos.tensorflow.core.framework import tensor_shape_pb2  
from protos.tensorflow.core.framework import types_pb2  
...
# ensure NHWC shape and build tensor proto
tensor_shape = [1]+list(img.shape)  
dims = [tensor_shape_pb2.TensorShapeProto.Dim(size=dim) for dim in tensor_shape]  
tensor_shape = tensor_shape_pb2.TensorShapeProto(dim=dims)  
tensor = tensor_pb2.TensorProto(  
              dtype=types_pb2.DT_FLOAT,
              tensor_shape=tensor_shape,
              float_val=list(img.reshape(-1)))
request.inputs['inputs'].CopyFrom(tensor) 

You can copy this compiled pythonic functions into your projects working directory and copy into the docker which should work.

Answered By – Rajesh Somasundaram

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0