How to use tf.reset_default_graph() - machine-learning

Question

Whenever I try to use tf.reset_default_graph(), I get this error: IndexError: list index out of range or ``. At which part of my code should I use this? When should I be using this?
Edit:
I updated the code, but the error still occurs.
def evaluate():
with tf.name_scope("loss"):
global x # x is a tf.placeholder()
xentropy = tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=neural_network(x))
loss = tf.reduce_mean(xentropy, name="loss")
with tf.name_scope("train"):
optimizer = tf.train.AdamOptimizer()
training_op = optimizer.minimize(loss)
with tf.name_scope("exec"):
with tf.Session() as sess:
for i in range(1, 2):
sess.run(tf.global_variables_initializer())
sess.run(training_op, feed_dict={x: np.array(train_data).reshape([-1, 1]), y: label})
print "Training " + str(i)
saver = tf.train.Saver()
saver.save(sess, "saved_models/testing")
print "Model Saved."
def predict():
with tf.name_scope("predict"):
tf.reset_default_graph()
with tf.Session() as sess:
saver = tf.train.import_meta_graph("saved_models/testing.meta")
saver.restore(sess, "saved_models/testing")
output_ = tf.get_default_graph().get_tensor_by_name('output_layer:0')
print sess.run(output_, feed_dict={x: np.array([12003]).reshape([-1, 1])})
def main():
print "Starting Program..."
evaluate()
writer = tf.summary.FileWriter("mygraph/logs", tf.get_default_graph())
predict()
If I remove the tf.reset_default_graph() from the updated code, I get this error: ValueError: cannot add op with name hidden_layer1/kernel/Adam as that name is already used
From my current understanding, tf.reset_default_graph() removes all graphs, hence I avoided the error I mention above(ValueError: cannot add op with name hidden_layer1/kernel/Adam as that name is already used)

Answer 1

This is probably how you use it:
import tensorflow as tf
a = tf.constant(1)
with tf.Session() as sess:
tf.reset_default_graph()
You get an error because you use it in a session. From the tf.reset_default_graph() documentation:
Calling this function while a tf.Session or tf.InteractiveSession is
active will result in undefined behavior. Using any previously created
tf.Operation or tf.Tensor objects after calling this function will
result in undefined behavior
tf.reset_default_graph() can be helpful (at least for me) during the testing phase while I experiment in jupyter notebook. However, I have never used it in production and do not see how it would be helpful there.
Here is an example that could be in a notebook:
import tensorflow as tf
# create some graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(...)
Now I do not need this stuff anymore, but if I create another graph and visualize it in tensorboard I will see old nodes and the new nodes. To solve this, I could restart the kernel and run only the next cell. However, I can just do:
tf.reset_default_graph()
# create a new graph
with tf.Session() as sess:
print sess.run(...)
Edit after OP added his code:
with tf.name_scope("predict"):
tf.reset_default_graph()
Here is what approximately happens. Your code fails because tf.name_scope already added something to a graph. While being inside of this "adding something to the graph", you tell TF to remove the graph completely, but it can't because it is busy adding something.

Answer 2

For some reason, I need to build a new graph FOR LOTS OF TIMES, and I have just tested, which works eventually! Many thanks for Salvador Dali's answer:-)
import tensorflow as tf
from my_models import Classifier
for i in range(10):
tf.reset_default_graph()
# build the graph
global_step = tf.get_variable('global_step', [], initializer=tf.constant_initializer(0), trainable=False)
classifier = Classifier(global_step)
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
print("do sth here.")

Answer 3

With TensorFlow 2.0 coming out, now it's better to use tf.compat.v1.reset_default_graph() in order to avoid getting warning. Link to the documentation: https://www.tensorflow.org/api_docs/python/tf/compat/v1/reset_default_graph

Answer 4

simply put,
use to clear previous placeholder which you using sess.run() created