Tensorflow debug or print statements - debugging

Question

I am very new to TensorFlow and trying to learn it. I copied a program from tutorial website. As I modified it, there are issues with the program and I have to debug. I am looking for help to understand how I can print certain values such as cost and optimizer. I have to figure out to see the value being updated in each iteration. I understand that notes cannot be printed but I take that cost and optimizers are inputs which should be printable, right?
plt.ion()
n_observations = 100
xs = np.linspace(-3, 3, n_observations)
ys = np.sin(xs) + np.random.uniform(-0.5, 0.5, n_observations)
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
Y_pred = tf.Variable(tf.random_normal([1]), name='bias')
for pow_i in range(1, 5):
W = tf.Variable(tf.random_normal([1]), name='weight_%d' % pow_i)
Y_pred = tf.add(tf.multiply(tf.pow(X, pow_i), W), Y_pred)
cost = tf.reduce_sum(tf.pow(Y_pred - Y, 2)) / (n_observations - 1)
d = tf.Print(cost, [cost, 2.0], message="Value of cost id:")
learning_rate = 0.01
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
n_epochs = 10
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
prev_training_cost = 0.0
for epoch_i in range(n_epochs):
for (x, y) in zip(xs, ys):
print("Msg2 x, y ", x, y, cost);
sess.run(optimizer, feed_dict={X: x, Y: y})
sess.run(d)
print("Msg3 x, y ttt ", x, y, optimizer);
training_cost = sess.run(
cost, feed_dict={X: xs, Y: ys})
print(training_cost)
print("Msg3 cost, xs ys", cost, xs, ys);
if epoch_i % 100 == 0:
ax.plot(xs, Y_pred.eval(
feed_dict={X: xs}, session=sess),
'k', alpha=epoch_i / n_epochs)
fig.show()
#plt.draw()
# Allow the training to quit if we've reached a minimum
if np.abs(prev_training_cost - training_cost) < 0.001:
break
prev_training_cost = training_cost
ax.set_ylim([-3, 3])
fig.show()
plt.waitforbuttonpress()

Answer 1

In your example, cost and optimizer refer to tensors in the graph, not inputs to your graph. The need to be fetched in a session.run call to be able to print their python values. For example, in your example, printing training_cost should be printing the cost. Similarly, if you return the value you of optimizer from session.run(optimizer, ...), it should return the correct printable value.
If you are interested in debugging and printing values check out:
tfdbg
tf.Print
Hope that helps!