# Liu Sida's Homepage

Machine Learning & Human Learning

© 2014. liusida All rights reserved.

# Use Tensorflow to Compute Gradient

In most of Tensorflow tutorials, we use minimize(loss) to automatically update parameters of the model.

In fact, minimize() is an integration of two steps: computing gradients, and applying the gradients to update parameters.

Let’s take a look at an example:

What is the gradient of W and B when W=1.0, B=1.0?

We can calculate them by hand:

let $N = 100 - 3W - B$, so that $Y = N^2$

ok, now let use tensorflow to compute that:

import tensorflow as tf

# make an example:
# Y = (100 - W X - B)^2
X = tf.constant(3.)
W = tf.Variable(1.)
B = tf.Variable(1.)
Y = tf.square(100 - W*X - B)

#the lr here is not about gradient computing. it only effect when appling
Ops = tf.train.GradientDescentOptimizer(learning_rate=0.001)
grads_and_vars = Ops.compute_gradients(Y)
# we can modify the gradient here and then:
# Op_update = Ops.apply_gradients(grads_and_vars)

with tf.Session() as sess:

sess.run(tf.global_variables_initializer())
print(sess.run(grads_and_vars))


run it, and we get:
 [(-576.0, 1.0), (-192.0, 1.0)] 

So next time your professor ask you to implement a back-propagation for some complex networks by your self, maybe this trick can help you double-check your implementation. Hooray!