Unable to take Input values from Tensorflow Serving client request - arguments

Question

I am trying to deploy my tensorflow model in a tensorflow serving using tensorflow saved model. Input for my tf model is a string value and i have defined my signature as below
prediction_signature = (
tf.saved_model.signature_def_utils.build_signature_def(
inputs={'input_path': tensor_info_input},
outputs={'output_prediction': tensor_info_output},
method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))
builder.add_meta_graph_and_variables(
sess, [tf.saved_model.tag_constants.SERVING],
signature_def_map={
'predict_images':
prediction_signature,
})
My objective is to read a image from the path specified in signature definition. How do i get the input_path from input definition and convert the tensor into actual string value to read the path

Answer 1

I think you need to add some "reading operation" in your model file.
Like this:
def inference(image_path_string):
# read image from a image path string
image_file = tf.read_file(image_path_string)
image_raw = tf.cond(tf.image.is_jpeg(image_file),
lambda: tf.image.decode_jpeg(image_file, channels= 3 if rgb, else 1),
lambda: tf.image.decode_bmp(image_file))
# image preprocessing code
...
...
# your prediction model code
...
...

Answer 2

You can place the below mentioned code in a Python file say, Client.py. And for inference, you can run the below mentioned command in the Terminal
python Client.py --image abc/0.png --model mnist --signature_name predict
This will take the path of the image, "abc/0.png" and will convert it to Numeric Values, and will run the inference.
Code for Client.py is mentioned below. The below code is for MNIST Images. You can reshape your images accordingly:
from __future__ import print_function
import argparse
import time
import numpy as np
from scipy.misc import imread
import grpc
from tensorflow.contrib.util import make_tensor_proto
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc
def run(host, port, image, model, signature_name):
channel = grpc.insecure_channel('{host}:{port}'.format(host=host, port=port))
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
# Read an image
data = imread(image)
data = data.astype(np.float32)
print(data)
start = time.time()
# Call classification model to make prediction on the image
request = predict_pb2.PredictRequest()
request.model_spec.name = model
request.model_spec.signature_name = signature_name
request.inputs['image'].CopyFrom(make_tensor_proto(data, shape=[1, 28, 28, 1]))
result = stub.Predict(request, 10.0)
end = time.time()
time_diff = end - start
# Reference:
# How to access nested values
# https://stackoverflow.com/questions/44785847/how-to-retrieve-float-val-from-a-predictresponse-object
print(result)
print('time elapased: {}'.format(time_diff))
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--host', help='Tensorflow server host name', default='localhost', type=str)
parser.add_argument('--port', help='Tensorflow server port number', default=8500, type=int)
parser.add_argument('--image', help='input image', type=str)
parser.add_argument('--model', help='model name', type=str)
parser.add_argument('--signature_name', help='Signature name of saved TF model',
default='serving_default', type=str)
args = parser.parse_args()
run(args.host, args.port, args.image, args.model, args.signature_name)
For more information, you can refer this beautiful article on Tensorflow Serving,
https://medium.com/#yuu.ishikawa/serving-pre-modeled-and-custom-tensorflow-estimator-with-tensorflow-serving-12833b4be421