Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf ·...
Transcript of Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf ·...
![Page 1: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/1.jpg)
Introduction toPresented by Xie Yaochen
2017.05.13
![Page 2: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/2.jpg)
A brief introduction to Deep Learning
![Page 3: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/3.jpg)
Get started with a Neural Networks
![Page 4: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/4.jpg)
![Page 5: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/5.jpg)
Advanced Models
Convolutional Neural Networks (CNN)
![Page 6: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/6.jpg)
Advanced Models
Convolutional Neural Networks (CNN)
Convolutional Layers
![Page 7: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/7.jpg)
Advanced Models
Convolutional Neural Networks (CNN)
Pooling Layers
![Page 8: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/8.jpg)
Advanced Models
Recurrent Neural Networks (RNN)
![Page 9: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/9.jpg)
Advanced Models
Recurrent Neural Networks (RNN)
Variants of RNN:
LSTM
GRU
![Page 10: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/10.jpg)
Advanced Models
Methods/Tricks to deal with Overfitting
Regularization
Activation (Relu…)
Dropout
Batch & Batch normalization
….
![Page 11: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/11.jpg)
Advanced Models
Optimizer
Stochastic gradient descent (SGD)
Momentum
Adagrad
Adam
….
![Page 12: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/12.jpg)
What is
TensorFlow™ is an open source software library for numerical computation using data flow graphs open-sourced by Google.
But what does it actually do? TensorFlow provides primitives for defining functions on tensors and
automatically computing their derivatives.
![Page 13: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/13.jpg)
Okay, I know. So what is a Tensor?
Tensor: N-dimensional array
A scalar is a tensor
A vector is a tensor
A matrix is a tensor
e.g. Image represented as 3-d tensor rows, cols, channels(RGB)
3 # a rank 0 tensor; this is a scalar with shape []
[1. ,2., 3.] # a rank 1 tensor; this is a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]
![Page 14: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/14.jpg)
When and Why should we use TensorFlow?
Frame work
Developing Language
Supported API 安装难度 灵活性 上⼿手难度
Caffe C++/CUDA C++/python/ matlab
*** ** **
mxNet C++/CUDA Matlab/JS/ C++/Scala
** ** *
Tensorflow C++/CUDA/ python
C++/python * *** ***
Other: Theano, Torch… (CPU only)
![Page 15: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/15.jpg)
How to use
One command to install TensorFlow
$ pip install tensorflow
( for Linux / Mac OS )
or install by Anaconda
( for Windows)
Two steps to run your TensorFlow
Building the computational graph
Running the computational graph
![Page 16: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/16.jpg)
Re : Zero 从零开始的Tensorflow
(假设⼤大家都会python以及常⽤用库,如numpy,的基本使⽤用⽅方法)
Define a constant:
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0) # also tf.float32 implicitly
print(node1, node2)
Output
Tensor("Const:0", shape=(), dtype=float32) Tensor("Const_1:0", shape=(), dtype=float32)
![Page 17: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/17.jpg)
Re : Zero 从零开始的Tensorflow
Get value of a tensor
sess = tf.Session()
print(sess.run([node1, node2]))
Output
[3.0, 4.0]
![Page 18: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/18.jpg)
Re : Zero 从零开始的Tensorflow
Operations
node3 = tf.add(node1, node2)
print("node3: ", node3)
print("sess.run(node3): ",sess.run(node3))
Output
node3: Tensor("Add_2:0", shape=(), dtype=float32)
sess.run(node3): 7.0
![Page 19: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/19.jpg)
Re : Zero 从零开始的Tensorflow
Placeholder
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b # + provides a shortcut for tf.add(a, b)
print(sess.run(adder_node, {a: 3, b:4.5}))
print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))
Output
7.5
[ 3. 7.]
![Page 20: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/20.jpg)
Re : Zero 从零开始的Tensorflow
Variable
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b
Initialize and run session
init = tf.global_variables_initializer()
sess.run(init)
![Page 21: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/21.jpg)
Re : Zero 从零开始的Tensorflow
Variable
print(sess.run(linear_model, {x:[1,2,3,4]}))
Output
[ 0. 0.30000001 0.60000002 0.90000004]
![Page 22: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/22.jpg)
Re : Zero 从零开始的Tensorflow
To evaluate the model
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))
Output
23.66
![Page 23: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/23.jpg)
Re : Zero 从零开始的Tensorflow
Optimizing
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
sess.run(train, {x:[1,2,3,4], y:[0,-1,-2,-3]})
print(sess.run([W, b]))
![Page 24: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/24.jpg)
MNIST - To Say “Hello World!”
MNIST is a simple computer vision dataset. It consists of images of handwritten digits like these:
Each image is 28 pixels by 28 pixels. We can interpret this as a big array of numbers:
![Page 25: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/25.jpg)
MNIST - To Say “Hello World!”
Step 1: Pretreat the images and labels (total size = 55000)
Flatten this array into a vector of 28x28 = 784 numbers
Convert the labels into one-hot vectors
(For example, 3 would be [0,0,0,1,0,0,0,0,0,0] )
![Page 26: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/26.jpg)
MNIST - To Say “Hello World!”
Step 2: Softmax Regressions
![Page 27: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/27.jpg)
MNIST - To Say “Hello World!”
Step 3: To train and evaluate the model
Loss : Cross-entropy
Where y is our predicted probability distribution, and y′ is the true distribution (the one-hot vector with the digit labels)
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
![Page 28: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/28.jpg)
MNIST - A Further Example (CNN)
Convolution and Pooling (see example3.py)
![Page 29: Introduction to tf - Texas A&M Universitypeople.tamu.edu/~ethanycx/slides/Introduction_to_tf.pdf · 2018-09-03 · A brief introduction to Deep Learning. Get started with a Neural](https://reader034.fdocuments.us/reader034/viewer/2022042218/5ec47e2712ed45645146338d/html5/thumbnails/29.jpg)
References:
1. The MNIST Database: http://yann.lecun.com/exdb/mnist/
2. TensorFlow Official Document: https://www.tensorflow.org/
3. Colah’s Blog: http://colah.github.io/
4. Stanford Course CS224d: https://cs224d.stanford.edu/lectures/
5. Prof. Jordi Torres’ Home Page: http://www.jorditorres.org/
6. Fabien Baradel’s Blog: https://fabienbaradel.github.io/
Also where you could learn more about Deep Learning and TensorFlow