莫烦---Tensorflow学习
生活随笔
收集整理的這篇文章主要介紹了
莫烦---Tensorflow学习
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
加粗樣式邊學邊記錄，感謝莫煩大神的教學視頻，獲益良多，之前已經看完了吳恩達Andrew Ng的視頻，但對Tensorflow的使用還是有很多不懂的地方，還是要花些時間好好學學tensorflow和keras。
莫煩大神的視頻學習地址
Tensorflow 基礎框架
基于tensorflow構建的三層（單隱層）神經網絡如下圖所示：
上圖中，圓形和正方形的節點被稱為node，在node間流動的數據流（多維數組）被稱為張量（tensor）。
張量 Tensor ：
| 0階張量 = 標量（Scalar） | 也就是一個數值，如[1][1][1] |
| 1階張量 = 向量（Vector） | 比如一維的[1,2,3][1 , 2 , 3][1,2,3] |
| 2階張量 = 矩陣（Matrix） | 比如[[1,2,3],[4,5,6],[7,8,9]][[1,2,3],[4,5,6],[7,8,9]][[1,2,3],[4,5,6],[7,8,9]] |
| …\dots… | …\dots… |
| n階張量 = n維數組 |
tensor和node之間關系：
如果輸入tensor的維度是5000×645000\times645000×64，表示有5000個訓練樣本，每個樣本有64個特征，則輸入層必須有64個node來接受這些特征。
例子
創建數據訓練函數，目標是要獲得公式y = 0.1*x+0.3中的0.1和0.3這兩個值，即0.1為權重weight和0.3為偏執bias
import tensorflow as tf import numpy as np # create data x_data=np.random.rand(100).astype(np.float32) y_data=x_data*0.1+0.3### create tensorflow structure start## #參數定義（一維,第二個是取值范圍的下限，第三個參數是取值范圍的上限） Weights = tf.Variable(tf.random_uniform([1],-1.0,1.0)) biases = tf.Variable(tf.zeros([1]))y = Weights*x_data+biases#計算預測值和真實值的差值 loss=tf.reduce_mean(tf.square(y-y_data)) optimizer = tf.train.GradientDescentOptimizer(0.5) #learning_rate train= optimizer.minimize(loss)#建立完以上這些變量，還需要在NN中初始化這些變量variables init=tf.initialize_all_variables() ###create tensorflow structure end ###sess = tf.Session() sess.run(init) # Very importantfor step in range(201):sess.run(train)if step % 20 == 0: #每隔20步打印一次print(step,sess.run(Weights),sess.run(biases)) WARNING:tensorflow:From /home/will/anaconda3/envs/py35/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:118: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use `tf.global_variables_initializer` instead. 0 [0.36647058] [0.20860067] 20 [0.16722272] [0.25959927] 40 [0.11913844] [0.28849784] 60 [0.10544876] [0.2967253] 80 [0.10155129] [0.2990677] 100 [0.10044166] [0.2997346] 120 [0.10012573] [0.29992446] 140 [0.10003577] [0.29997852] 160 [0.10001019] [0.2999939] 180 [0.10000289] [0.29999828] 200 [0.10000083] [0.2999995]這時輸出的Weight就很接近1了，bias也很接近0.3
Session 會話控制
import tensorflow as tfmatrix1 = tf.constant([[3,3]]) # 一行兩列的matrix matrix2 = tf.constant([[2],[2]]) #兩行一列的matrixproduct = tf.matmul(matrix1,matrix2) #matrix multipy矩陣乘法。而在np中，是np.dot(m1,m2)類似的功能#method 1 #sess = tf.Session() #result = sess.run(product) # tensorflow思考模式：每run一次就會執行一次這個結構 #print(result) #sess.close() #有跟沒有差不多，有就系統一點#method 2 with tf.Session() as sess: #意思是打開tf.Session并以sess命名它result2 = sess.run(product)print(result2)Variable變量
import tensorflow as tfstate = tf.Variable(0,name='counter') # 只有定義成一個變量才是一個變量，不像Python里的 #print(state.name) one = tf.constant(1) #變量 + 常亮 等于 變量 new_value = tf.add(state,one) update = tf.assign(state,new_value) #將new_value變量加載到state上，所以當前state的狀態等于new_value #這里的state就作為儲存新數值的載體#如果在tensorflow中設置了變量，那么接下來這一步很重要 init=tf.initialize_all_variables()#must have if define variable#初始化素有的變量，還需要session.run才能激活 with tf.Session() as sess:sess.run(init)for _ in range(3):sess.run(update)print(sess.run(state)) 1 2 3只要在tensorflow中設置了變量，記得要在初始化.
Placeholder傳入值
placeholder 是 Tensorflow 中的占位符，暫時儲存變量.
Tensorflow 如果想要從外部傳入data, 那就需要用到 tf.placeholder(), 然后以這種形式傳輸數據 sess.run(***, feed_dict={input: **})
也就是喂數據。
placeholder和variable的區別：
通俗講：placeholder 是你輸入自己數據的接口, variable 是網絡自身的變量, 通常不是你來進行修改, 而是網絡自身會改動更新.
添加層 def add_layer()
定義添加神經層的函數def add_layer(),它有四個參數：輸入值、輸入的大小、輸出的大小和激勵函數，我們設定默認的激勵函數是None。
import tensorflow as tfdef add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機器學習中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputs建造神經網絡
import tensorflow as tf import numpy as npdef add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機器學習中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputsx_data = np.linspace(-1,1,300)[:,np.newaxis] #-1到1的范圍內300個單位,即有300行，300個例子 noise = np.random.normal(0,0.05,x_data.shape) y_data = np.square(x_data) - 0.5 + noisexs = tf.placeholder(tf.float32,[None,1]) #輸入只有1個特征，所以這里是1，輸出也一樣 ys = tf.placeholder(tf.float32,[None,1]) # None的意思是無論輸入多少個例子都okl1 = add_layer(xs,1,10,activation_function=tf.nn.relu) prediction = add_layer(l1,10,1,activation_function=None) #構建的是——輸入層1個、隱藏層10個、輸出層1個的神經網絡。# 對求和的所有數求一個平均值，也就是平均的誤差是多少 loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1])) train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) #learning_rateinit = tf.initialize_all_variables() sess = tf.Session() sess.run(init)for i in range(1000):sess.run(train_step,feed_dict={xs:x_data,ys:y_data})if i % 50 == 0:print(sess.run(loss,feed_dict={xs:x_data,ys:y_data})) #loss也是要用到placeholder，只要用到placeholder都要加上feed_dict 0.53408515 0.0043351008 0.003585369 0.0032615305 0.0030836272 0.0029586344 0.0028774897 0.0028177483 0.0027668795 0.0027297884 0.0026985474 0.0026706108 0.0026462309 0.0026261543 0.0026080583 0.002592326 0.0025791153 0.0025680475 0.0025586013 0.0025521303訓練結果
輸出結果可視化
優化過程中，結果可視化可以給我們指引優化方向
import tensorflow as tf import numpy as np import matplotlib.pyplot as plt # 作為python中輸出結果可視化的一個模塊 %matplotlib #在jupyter notebook加上這個才能看到動態圖def add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機器學習中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputsx_data = np.linspace(-1,1,300)[:,np.newaxis] #-1到1的范圍內300個單位,即有300行，300個例子 noise = np.random.normal(0,0.05,x_data.shape) y_data = np.square(x_data) - 0.5 + noisexs = tf.placeholder(tf.float32,[None,1]) #輸入只有1個特征，所以這里是1，輸出也一樣 ys = tf.placeholder(tf.float32,[None,1]) # None的意思是無論輸入多少個例子都okl1 = add_layer(xs,1,10,activation_function=tf.nn.relu) prediction = add_layer(l1,10,1,activation_function=None) #構建的是——輸入層1個、隱藏層10個、輸出層1個的神經網絡。# 對求和的所有數求一個平均值，也就是平均的誤差是多少 loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1])) train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) #learning_rateinit = tf.initialize_all_variables() sess = tf.Session() sess.run(init)fig = plt.figure() ax = fig.add_subplot(1,1,1) ax.scatter(x_data,y_data) plt.ion() #主程序如果不加這個，就會在運行完plt.show()之后暫停 plt.show() for i in range(1000):sess.run(train_step,feed_dict={xs:x_data,ys:y_data})if i % 50 == 0:#print(sess.run(loss,feed_dict={xs:x_data,ys:y_data})) #loss也是要用到placeholder，只要用到placeholder都要加上feed_dicttry:ax.lines.remove(lines[0]) #在圖片里去除lines的第一個線段，lines也只有一個線段，不然會有很多條線，最后密密麻麻的線except Exception:passprediction_value = sess.run(prediction,feed_dict={xs:x_data})lines = ax.plot(x_data,prediction_value,'r-',lw=5) #將prediction的結果以曲線的形式plot上去，不是點的形式, lw是線的寬度plt.pause(0.1) #plot過程中暫停0.1s如果是在jupyter notebook上跑的注意了，如果不加%matplotlib只會輸出散點圖，連紅色曲線都沒有，加上這個指令可以另外打開一個窗口顯示。
各種優化器
Tensorflow 中的優化器會有很多不同的種類。最基本, 也是最常用的一種就是GradientDescentOptimizer。
其實都是優化learning_rate
Tensorboard 可視化好幫手
先放圖片，下圖這里一個塊都可以打開，而且塊需要在原來的代碼中加入一些代碼把它們框起來，以及可以命名它們
然后可以在/home/username下看到logs文件夾，然后打開terminal，輸入tensorboard --logdir='logs/'
然后會看到
按住Ctrl點擊里面的網址就可以打開tensorboard可視化助手查看當前的神經網絡結構了。
如果在jupyter notebook里面重復運行同一段代碼生成可視化結果，可能會報錯
這時把jupyter notebook的kernel重啟把里面的緩存數據清空（最簡單就是把整個jupyter notebook重啟），再把/logs目錄下的結果清空，再打開jupyter notebook運行代碼就可以了。
可視化Weights、bias及loss等數據
這個的代碼運行完了，只能看到graph的內容，看不到histogram的內容和event的內容，奇怪，不知道什么情況，有哪位大神知道的，望指教
from __future__ import print_function import tensorflow as tf import numpy as np#這里加多了一個參數，n_layer是添加層的名字 def add_layer(inputs, in_size, out_size, n_layer, activation_function=None):# add one more layer and return the output of this layerlayer_name = 'layer%s' % n_layer #從這里可以看出with tf.name_scope(layer_name):with tf.name_scope('weights'):Weights = tf.Variable(tf.random_normal([in_size, out_size]), name='W')#可視化Weights的變化量，第一個參數是名稱，第二參數是輸入tf.summary.histogram(layer_name + '/weights', Weights)with tf.name_scope('biases'):biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, name='b')#可視化biases的變化量tf.summary.histogram(layer_name + '/biases', biases)with tf.name_scope('Wx_plus_b'):Wx_plus_b = tf.add(tf.matmul(inputs, Weights), biases)if activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b, )#可視化outputs的變化量tf.summary.histogram(layer_name + '/outputs', outputs)return outputs# Make up some real data x_data = np.linspace(-1, 1, 300)[:, np.newaxis] noise = np.random.normal(0, 0.05, x_data.shape) y_data = np.square(x_data) - 0.5 + noise# define placeholder for inputs to network with tf.name_scope('inputs'):xs = tf.placeholder(tf.float32, [None, 1], name='x_input')ys = tf.placeholder(tf.float32, [None, 1], name='y_input')# add hidden layer l1 = add_layer(xs, 1, 10, n_layer=1, activation_function=tf.nn.relu) # add output layer prediction = add_layer(l1, 10, 1, n_layer=2, activation_function=None)# the error between prediciton and real data with tf.name_scope('loss'):loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1]))#可視化loss的變化量，這個很重要，用scalar的話，不會出現在histogram里面，而是出現在event里面#也可以改成在histogram里面顯示，跟上面變量顯示一樣的代碼格式tf.summary.scalar('loss', loss)with tf.name_scope('train'):train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)sess = tf.Session() #給所有訓練圖合并，tf.merge_all_summaries()方法會對我們所有的 summaries 合并到一起. merged = tf.summary.merge_all()writer = tf.summary.FileWriter("logs/", sess.graph)init = tf.global_variables_initializer() sess.run(init)for i in range(1000):sess.run(train_step, feed_dict={xs: x_data, ys: y_data})if i % 50 == 0:result = sess.run(merged,feed_dict={xs: x_data, ys: y_data}) #merged也是要運行才能記錄結果的 writer.add_summary(result, i)高階內容
Classification 分類任務
這里需要MNIST的數據集，這個需要翻墻才能下載，一開始不知道老是報錯，數據集不大，建議找找資源跑一下，挺有收獲的。
數據集下載地址：http://yann.lecun.com/exdb/mnist/
其實運行代碼就會自動下載，不需要手動下載，手動下載需要把壓縮包放在/home/user/MNIST_data/目錄下。
這一節沒有跑，之前做Andrew Ng的作業也跑過一個類似的，識別數字手勢的。
Dropout解決overfitting
""" Please note, this code is only for python 3+. If you are using python 2+, please modify the code accordingly. """ import tensorflow as tf from sklearn.datasets import load_digits from sklearn.model_selection import train_test_split from sklearn.preprocessing import LabelBinarizer# load data digits = load_digits() X = digits.data y = digits.target y = LabelBinarizer().fit_transform(y) #使y的標簽成為one-hot vector，在對應位置上顯示1表示對應的數字 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3) #拆分數據集def add_layer(inputs, in_size, out_size, layer_name, activation_function=None, ):# add one more layer and return the output of this layerWeights = tf.Variable(tf.random_normal([in_size, out_size]))biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, )Wx_plus_b = tf.matmul(inputs, Weights) + biases# here to dropout，dropout操作Wx_plus_b = tf.nn.dropout(Wx_plus_b, keep_prob)if activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b, )tf.summary.histogram(layer_name + '/outputs', outputs)return outputs# define placeholder for inputs to network keep_prob = tf.placeholder(tf.float32) #dropout操作 xs = tf.placeholder(tf.float32, [None, 64]) # 8x8 ys = tf.placeholder(tf.float32, [None, 10])# add output layer l1 = add_layer(xs, 64, 50, 'l1', activation_function=tf.nn.tanh) prediction = add_layer(l1, 50, 10, 'l2', activation_function=tf.nn.softmax)# the loss between prediction and real data cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction),reduction_indices=[1])) # loss tf.summary.scalar('loss', cross_entropy) train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)sess = tf.Session() merged = tf.summary.merge_all() # summary writer goes in here train_writer = tf.summary.FileWriter("logs/train", sess.graph) test_writer = tf.summary.FileWriter("logs/test", sess.graph)# tf.initialize_all_variables() no long valid from # 2017-03-02 if using tensorflow >= 0.12 if int((tf.__version__).split('.')[1]) < 12 and int((tf.__version__).split('.')[0]) < 1:init = tf.initialize_all_variables() else:init = tf.global_variables_initializer() sess.run(init) for i in range(500):# here to determine the keeping probability，0.5表示保留50%的節點，0.6就保留60%sess.run(train_step, feed_dict={xs: X_train, ys: y_train, keep_prob: 0.5})if i % 50 == 0:# record loss #這里的keep_prob要設置成1，因為這里是要記錄數據，所以要記錄全部的數據train_result = sess.run(merged, feed_dict={xs: X_train, ys: y_train, keep_prob: 1}) test_result = sess.run(merged, feed_dict={xs: X_test, ys: y_test, keep_prob: 1})train_writer.add_summary(train_result, i) test_writer.add_summary(test_result, i)生成的logs/文件夾下有test和train兩個文件夾，terminal輸入tensorboard --logdir='logs/'，然后打開網址，就可以查看dropout的效果，但是很奇怪，test數據集好像沒跑成功，只顯示train的效果。
CNN部分
關鍵內容：建立卷積層、池化層
import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data # number 1 to 10 data mnist = input_data.read_data_sets('MNIST_data', one_hot=True)def compute_accuracy(v_xs, v_ys):global predictiony_pre = sess.run(prediction, feed_dict={xs: v_xs, keep_prob: 1})correct_prediction = tf.equal(tf.argmax(y_pre,1), tf.argmax(v_ys,1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))result = sess.run(accuracy, feed_dict={xs: v_xs, ys: v_ys, keep_prob: 1})return resultdef weight_variable(shape):initial = tf.truncated_normal(shape,stddev=0.1) #shape表示生成張量的維度，mean是均值，stddev是標準差return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape) #創建常量return tf.Variable(initial)def conv2d(x,W):#stride[1,x_movement,y_movement,1] #Must have strides[0]=strides[3]=1,就是上面的第一和第四個數return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME') def max_pool_2x2(x):#stride[1,x_movement,y_movement,1] return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')# define placeholder for inputs to network xs = tf.placeholder(tf.float32, [None, 784]) # 28x28 ys = tf.placeholder(tf.float32, [None, 10]) keep_prob = tf.placeholder(tf.float32) #-1代表先不考慮輸入的圖片例子多少這個維度 #后面的1是channel的數量，因為我們輸入的圖片是黑白的，因此channel是1，例如如果是RGB圖像，那么channel就是3 x_image = tf.reshape(xs,[-1,28,28,1]) # print(x_image.shape) #[n_samples,28,28,1]## conv1 layer ## #卷積核patch/kernel 5×5 #in size 1是image的厚度，因為是黑白圖片channel是1 #out size 32輸出是32個featuremap即32個filters W_conv1 = weight_variable([5,5,1,32]) # b_conv1 = bias_variable([32]) h_conv1 = tf.nn.relu(conv2d(x_image,W_conv1)+b_conv1) #output size 28*28*32 h_pool1 = max_pool_2x2(h_conv1) #output size 14*14*32## conv2 layer ## W_conv2 = weight_variable([5,5,32,64]) # b_conv2 = bias_variable([64]) h_conv2 = tf.nn.relu(conv2d(h_pool1,W_conv2)+b_conv2) #output size 14*14*64 h_pool2 = max_pool_2x2(h_conv2) #output size 7*7*64## func1 layer ## W_fc1 = weight_variable([7*7*64,1024]) b_fc1 = bias_variable([1024])h_pool2_flat = tf.reshape(h_pool2,[-1,7*7*64]) #[n_smaples,7,7,64] ->> [n_samples,7*7*64] h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1) h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)## func2 layer ## W_fc2 = weight_variable([1024,10]) b_fc2 = bias_variable([10]) prediction = tf.nn.softmax(tf.matmul(h_fc1_drop,W_fc2) + b_fc2)# the error between prediction and real data cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction),reduction_indices=[1])) # loss train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #這里用Adam優化器，不用Gradient Descentsess = tf.Session() # important step # tf.initialize_all_variables() no long valid from # 2017-03-02 if using tensorflow >= 0.12 if int((tf.__version__).split('.')[1]) < 12 and int((tf.__version__).split('.')[0]) < 1:init = tf.initialize_all_variables() else:init = tf.global_variables_initializer() sess.run(init)for i in range(1000):batch_xs, batch_ys = mnist.train.next_batch(100)sess.run(train_step, feed_dict={xs: batch_xs, ys: batch_ys, keep_prob: 0.5})if i % 50 == 0:print(compute_accuracy( mnist.test.images[:1000], mnist.test.labels[:1000]))運行結果：
Extracting MNIST_data/train-images-idx3-ubyte.gz Extracting MNIST_data/train-labels-idx1-ubyte.gz Extracting MNIST_data/t10k-images-idx3-ubyte.gz Extracting MNIST_data/t10k-labels-idx1-ubyte.gz 0.124 0.736 0.824 0.88 0.899 0.908 0.92 0.932 0.931 0.94 0.949 0.949 0.944 0.958 0.955 0.956 0.96 0.963 0.959 0.965測試集的準確率已經很高了。
Saver保存和讀取
參數保存方法（目前TF只能保存參數，不能保存整個網絡結構）：
import tensorflow as tf## Save to file # remember to define the same dtype and shape when restore W = tf.Variable([[1,2,3],[3,4,5]],dtype=tf.float32,name="weights") b = tf.Variable([[1,2,3]],dtype=tf.float32,name="biases") #定義dtype很重要init = tf.global_variables_initializer()saver = tf.train.Saver() #存儲變量with tf.Session() as sess:sess.run(init)save_path = saver.save(sess,"my_net/save_net.ckpt") #第二個是保存路徑print("Save to path:",save_path)提取方法：
import tensorflow as tf import numpy as np # restore variables # redefine the same shape and same type for your variables #即重新加載參數需要重定義相同形狀和類型的變量在裝載參數 W = tf.Variable(np.arange(6).reshape((2,3)),dtype=tf.float32,name="weights") b = tf.Variable(np.arange(3).reshape((1,3)),dtype=tf.float32,name="biases")# not need init step saver = tf.train.Saver() with tf.Session() as sess:#提取變量saver.restore(sess,"my_net/save_net.ckpt")print("weights:",sess.run(W))print("biases:",sess.run(b))如果使用jupyter notebook在同一個窗口使用保存和提取，會報錯，分開使用就可以了。
輸出結果：
遷移學習transfer learning
""" This is a simple example of transfer learning using VGG. Fine tune a CNN from a classifier to regressor. Generate some fake data for describing cat and tiger length. Fake length setting: Cat - Normal distribution (40, 8) Tiger - Normal distribution (100, 30) The VGG model and parameters are adopted from: https://github.com/machrisaa/tensorflow-vgg Learn more, visit my tutorial site: [莫煩Python](https://morvanzhou.github.io) """from urllib.request import urlretrieve import os import numpy as np import tensorflow as tf import skimage.io import skimage.transform import matplotlib.pyplot as pltdef download(): # download tiger and kittycat imagecategories = ['tiger', 'kittycat']for category in categories:os.makedirs('./for_transfer_learning/data/%s' % category, exist_ok=True)with open('./for_transfer_learning/imagenet_%s.txt' % category, 'r') as file:urls = file.readlines()n_urls = len(urls)for i, url in enumerate(urls):try:urlretrieve(url.strip(), './for_transfer_learning/data/%s/%s' % (category, url.strip().split('/')[-1]))print('%s %i/%i' % (category, i, n_urls))except:print('%s %i/%i' % (category, i, n_urls), 'no image')def load_img(path):img = skimage.io.imread(path)img = img / 255.0# print "Original Image Shape: ", img.shape# we crop image from centershort_edge = min(img.shape[:2])yy = int((img.shape[0] - short_edge) / 2)xx = int((img.shape[1] - short_edge) / 2)crop_img = img[yy: yy + short_edge, xx: xx + short_edge]# resize to 224, 224resized_img = skimage.transform.resize(crop_img, (224, 224))[None, :, :, :] # shape [1, 224, 224, 3]return resized_imgdef load_data():imgs = {'tiger': [], 'kittycat': []}for k in imgs.keys():dir = './for_transfer_learning/data/' + kfor file in os.listdir(dir):if not file.lower().endswith('.jpg'):continuetry:resized_img = load_img(os.path.join(dir, file))except OSError:continueimgs[k].append(resized_img) # [1, height, width, depth] * nif len(imgs[k]) == 400: # only use 400 imgs to reduce my memory loadbreak# fake length data for tiger and cattigers_y = np.maximum(20, np.random.randn(len(imgs['tiger']), 1) * 30 + 100)cat_y = np.maximum(10, np.random.randn(len(imgs['kittycat']), 1) * 8 + 40)return imgs['tiger'], imgs['kittycat'], tigers_y, cat_yclass Vgg16:vgg_mean = [103.939, 116.779, 123.68]def __init__(self, vgg16_npy_path=None, restore_from=None):# pre-trained parameterstry:self.data_dict = np.load(vgg16_npy_path, encoding='latin1').item()except FileNotFoundError:print('Please download VGG16 parameters from here https://mega.nz/#!YU1FWJrA!O1ywiCS2IiOlUCtCpI6HTJOMrneN-Qdv3ywQP5poecM\nOr from my Baidu Cloud: https://pan.baidu.com/s/1Spps1Wy0bvrQHH2IMkRfpg')self.tfx = tf.placeholder(tf.float32, [None, 224, 224, 3])self.tfy = tf.placeholder(tf.float32, [None, 1])# Convert RGB to BGRred, green, blue = tf.split(axis=3, num_or_size_splits=3, value=self.tfx * 255.0)bgr = tf.concat(axis=3, values=[blue - self.vgg_mean[0],green - self.vgg_mean[1],red - self.vgg_mean[2],])# pre-trained VGG layers are fixed in fine-tuneconv1_1 = self.conv_layer(bgr, "conv1_1")conv1_2 = self.conv_layer(conv1_1, "conv1_2")pool1 = self.max_pool(conv1_2, 'pool1')conv2_1 = self.conv_layer(pool1, "conv2_1")conv2_2 = self.conv_layer(conv2_1, "conv2_2")pool2 = self.max_pool(conv2_2, 'pool2')conv3_1 = self.conv_layer(pool2, "conv3_1")conv3_2 = self.conv_layer(conv3_1, "conv3_2")conv3_3 = self.conv_layer(conv3_2, "conv3_3")pool3 = self.max_pool(conv3_3, 'pool3')conv4_1 = self.conv_layer(pool3, "conv4_1")conv4_2 = self.conv_layer(conv4_1, "conv4_2")conv4_3 = self.conv_layer(conv4_2, "conv4_3")pool4 = self.max_pool(conv4_3, 'pool4')conv5_1 = self.conv_layer(pool4, "conv5_1")conv5_2 = self.conv_layer(conv5_1, "conv5_2")conv5_3 = self.conv_layer(conv5_2, "conv5_3")pool5 = self.max_pool(conv5_3, 'pool5')# detach original VGG fc layers and# reconstruct your own fc layers serve for your own purposeself.flatten = tf.reshape(pool5, [-1, 7*7*512])self.fc6 = tf.layers.dense(self.flatten, 256, tf.nn.relu, name='fc6')self.out = tf.layers.dense(self.fc6, 1, name='out')self.sess = tf.Session()if restore_from: saver = tf.train.Saver()#初始restore_from為None，訓練完之后就可以使用相應的參數保存文件，這是路徑saver.restore(self.sess, restore_from) else: # training graphself.loss = tf.losses.mean_squared_error(labels=self.tfy, predictions=self.out)self.train_op = tf.train.RMSPropOptimizer(0.001).minimize(self.loss)self.sess.run(tf.global_variables_initializer())def max_pool(self, bottom, name):return tf.nn.max_pool(bottom, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name=name)def conv_layer(self, bottom, name):with tf.variable_scope(name): # CNN's filter is constant, NOT Variable that can be trainedconv = tf.nn.conv2d(bottom, self.data_dict[name][0], [1, 1, 1, 1], padding='SAME')lout = tf.nn.relu(tf.nn.bias_add(conv, self.data_dict[name][1]))return loutdef train(self, x, y):loss, _ = self.sess.run([self.loss, self.train_op], {self.tfx: x, self.tfy: y})return lossdef predict(self, paths):fig, axs = plt.subplots(1, 2)for i, path in enumerate(paths):x = load_img(path)length = self.sess.run(self.out, {self.tfx: x})axs[i].imshow(x[0])axs[i].set_title('Len: %.1f cm' % length)axs[i].set_xticks(()); axs[i].set_yticks(())plt.show()def save(self, path='./for_transfer_learning/model/transfer_learn'):saver = tf.train.Saver()saver.save(self.sess, path, write_meta_graph=False)def train():tigers_x, cats_x, tigers_y, cats_y = load_data()# plot fake length distributionplt.hist(tigers_y, bins=20, label='Tigers')plt.hist(cats_y, bins=10, label='Cats')plt.legend()plt.xlabel('length')plt.show()xs = np.concatenate(tigers_x + cats_x, axis=0)ys = np.concatenate((tigers_y, cats_y), axis=0)vgg = Vgg16(vgg16_npy_path='./for_transfer_learning/vgg16.npy')print('Net built')for i in range(100):b_idx = np.random.randint(0, len(xs), 6)train_loss = vgg.train(xs[b_idx], ys[b_idx])print(i, 'train loss: ', train_loss)vgg.save('./for_transfer_learning/model/transfer_learn') # save learned fc layersdef eval():vgg = Vgg16(vgg16_npy_path='./for_transfer_learning/vgg16.npy',restore_from='./for_transfer_learning/model/transfer_learn')vgg.predict(['./for_transfer_learning/data/kittycat/000129037.jpg', './for_transfer_learning/data/tiger/391412.jpg'])if __name__ == '__main__':# download()# train()eval()先運行download()把圖片都下載下來，只需要訓練最后的兩層，全連接層和輸出層，運行train()，最后運行eval()用樣本里的圖片測試模型。
總結
以上是生活随笔為你收集整理的莫烦---Tensorflow学习的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： NTLM
- 下一篇： CSS 中的定位：relative,ab