Tensorflow loads examples of multiple models at the same time

Sometimes we want to load multiple models in one python's file space at the same time. For example, we build 10 CNN models, and then we write a prediction class Predict, which restates the diagram structure and model parameters from the saved model restore. We then create 10 Predict objects, Instance, each responsible for 1 model prediction.

The core of Predict is:

class Predict:
 def __init__(self....):
   create sess
   Create a restorer tf.train.Saver
   Recovery parameters from the recovery point: tf.train.Saver.restore(...)


 def predict(self,...):
  sess.run(output,feed_dict={ The input })

If we take turns generating 10 different Predict objects, we find that tensorflow will report an error similar to the following:

 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
 pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,512] rhs shape= [640,512]
   [[Node: save/Assign_14 = Assign[T=DT_FLOAT, _class=["loc:@fullcont/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](fullcont/Variable, save/RestoreV2_14)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "PREDICT_WITH_SPARK_DATAFLOW_WA.py", line 121, in <module>
 pre2=Predict(label=new_list[1])
 File "PREDICT_WITH_SPARK_DATAFLOW_WA.py", line 47, in __init__
 self.saver.restore(self.sess,self.ckpt.model_checkpoint_path)
 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1560, in restore
 {self.saver_def.filename_tensor_name: save_path})
 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 895, in run
 run_metadata_ptr)
 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1124, in _run
 feed_dict_tensor, options, run_metadata)
 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
 options, run_metadata)
 File "/home/jiangminghao/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
 raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,512] rhs shape= [640,512]

The key is:

Assign requires shapes of both tensors to match. This means that the assignment failed while loading the model. This is mainly because the different sess in the different objects USES the same default figure graph in the 1 process space.

The right solution:

class Predict:
 def __init__(self....):
  self.graph=tf.Graph()# For each class ( The instance ) Create alone 1 a graph
  with self.graph.as_default():
    self.saver=tf.train.import_meta_graph(...)# Create a restorer 
    # Attention! The restorer must be generated in the newly created diagram , Otherwise you make a mistake. 
  self.sess=tf.Session(graph=self.graph)# Create a new sess
  with self.sess.as_default():
    with self.graph.as_default():
     self.saver.restore(self.sess,...)# Restore parameters from the recovery point 

 def predict(self,...):
  sess.run(output,feed_dict={ The input })