• Tensorflow版本(# 2017-06-24)：1.2.0
• Python版本：3.5.3
• 包括：
  • Tensorboard可視化
  • tfdbg調試
  • 常用的高級函數

一、TensorBoard?可視化

1、可視化計算圖

• 全部代碼：點擊查看
• 數據集使用MNIST手寫數字
• 加載數據

123456

'''加載數據'''data = input_data.read_data_sets('MNIST_data', one_hot=True)print("Size of:")print("\t\t training set:\t\t{}".format(len(data.train.labels)))print("\t\t test set: \t\t\t{}".format(len(data.test.labels)))print("\t\t validation set:\t{}".format(len(data.validation.labels)))

(1) 全連接網絡

• 超參數

123456

'''超參數'''img_size = 28img_flatten_size = img_size ** 2img_shape = (img_size, img_size)num_classes = 10learning_rate = 1e-4

• 定義添加一層的函數

  • num_layer指定是第幾層
  • activation指定激勵函數，若不指定跳過

    1234567891011121314

    '''定義添加一層'''def add_fully_layer(inputs, input_size, output_size, num_layer, activation=None): with tf.name_scope('layer_'+num_layer): with tf.name_scope('Weights'):W = tf.Variable(initial_value=tf.random_normal(shape=[input_size, output_size]), name='W') with tf.name_scope('biases'):b = tf.Variable(initial_value=tf.zeros(shape=[1, output_size]) + 0.1, name='b') with tf.name_scope('Wx_plus_b'):Wx_plus_b = tf.matmul(inputs, W) + b if activation is not None:outputs = activation(Wx_plus_b) else:outputs = Wx_plus_b return outputs

• 定義輸入，計算圖結構，loss和優化器

1234567891011121314151617

'''placehoder'''with tf.name_scope('inputs'):x = tf.placeholder(tf.float32, shape=[None, img_flatten_size], name='x')y = tf.placeholder(tf.float32, shape=[None, num_classes], name='y')'''結構'''hidden_layer1 = add_fully_layer(x, img_flatten_size, 20, '1', activation=tf.nn.relu)logits = add_fully_layer(hidden_layer1, 20, num_classes, '2')predictions = tf.nn.softmax(logits)'''loss'''#cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=logits)cross_entropy = -tf.reduce_sum(y*tf.log(predictions), reduction_indices=[1])with tf.name_scope('losses'):losses = tf.reduce_mean(cross_entropy)with tf.name_scope('train'):train_step = tf.train.AdamOptimizer(learning_rate).minimize(losses)

• 定義Session和tf.summary.FileWriter

1234	'''session'''with tf.Session() as sess:sess.run(tf.global_variables_initializer())writer = tf.summary.FileWriter('logs', sess.graph) # 將計算圖寫入文件

• 最后在logs的上級目錄打開命令行輸入：tensorboard --logdir=logs/，瀏覽器中輸入網址：http://localhost:6006即可查看

• 結果

  • 自定義的cross_entropy = -tf.reduce_sum(y*tf.log(predictions), reduction_indices=[1])
    
  • 使用tensorflow中自帶的cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=predictions)
    
• 可以看出tf.name_scope定義的名字就是其中的方框，點擊里面的可以查看里面對應的內容

(2)?CNN卷積神經網絡

• 添加一層卷積層和pooling層

  • 這里默認pooling使用maxpooling, 大小為2

    12345678910111213

    '''CNN 定義添加一層卷積層，包括pooling(使用maxpooling, size=2)'''def add_conv_layer(inputs, filter_size, input_channels, output_channels, num_layer, activation=tf.nn.relu): with tf.name_scope('conv_layer_'+num_layer): with tf.name_scope('Weights'):Weights = tf.Variable(tf.truncated_normal(stddev=0.1, shape=[filter_size, filter_size, input_channels, output_channels]), name='W') with tf.name_scope('biases'):b = tf.Variable(tf.constant(0.1, shape=[output_channels])) with tf.name_scope('conv2d'):conv2d_plus_b = tf.nn.conv2d(inputs, Weights, strides=[1,1,1,1], padding='SAME', name='conv') + bactivation_conv_outputs = activation(conv2d_plus_b) with tf.name_scope('max_pool'):max_pool_outputs = tf.nn.max_pool(activation_conv_outputs, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME') return max_pool_outputs

• 將卷積層展開

  • 返回展開層和數量（因為全連接會用到）

    123456789

    '''將卷積層展開'''def flatten_layer(layer): '''@param layer: the conv layer'''layer_shape = layer.get_shape() # 獲取形狀(layer_shape == [num_images, img_height, img_width, num_channels])num_features = layer_shape[1:4].num_elements() # [1:4] 是最后3個維度，就是展開的長度layer_flat = tf.reshape(layer, [-1, num_features]) # 展開 return layer_flat, num_features

• 定義輸入

  • 需要將x轉成圖片矩陣的形式

    12345

    '''placehoder'''with tf.name_scope('inputs'):x = tf.placeholder(tf.float32, shape=[None, img_flatten_size], name='x')y = tf.placeholder(tf.float32, shape=[None, num_classes], name='y')x_image = tf.reshape(x, shape = [-1, img_size, img_size, n_channels], name='x_images')

• 定義計算圖結構

12345678910111213

'''CNN卷積網絡結構'''conv_layer1 = add_conv_layer(x_image, filter_size=5, input_channels=1, output_channels=32, num_layer='1')conv_layer2 = add_conv_layer(conv_layer1, filter_size=5, input_channels=32, output_channels=64, num_layer='2')'''全連接層'''conv_layer2_flat, num_features = flatten_layer(conv_layer2) # 將最后操作的數據展開hidden_layer1 = add_fully_layer(conv_layer2_flat, num_features, 1000, num_layer='1', activation=tf.nn.relu)logits = add_fully_layer(hidden_layer1, 1000, num_classes, num_layer='2')predictions = tf.nn.softmax(logits)

• 結果
  • CNN總結構
    
  • 第一層卷積和pooling內部結構
    

(3)?RNN_LSTM循環神經網絡

• 聲明placeholder

  • 圖片中每一行當做當前的輸入，共有n_steps=28步遍歷完一張圖片，所以輸入x的shape=(batch_size, n_steps, n_inputs)
  • n_inputs就是一行的像素值

    12345

    '''placehoder'''with tf.name_scope('inputs'): '''RNN'''x = tf.placeholder(tf.float32, shape=[batch_size, n_steps, n_inputs], name='x')y = tf.placeholder(tf.float32, shape=[batch_size, num_classes], name='y')

• 添加一層cell

  • 我們最后只需要遍歷n_steps之后的輸出即可（遍歷完一張圖然后分類），所以對應的是final_state[1]（有兩個state, 一個是c state,一個是h state， 輸出是h state）

  • 12345678910111213

    '''RNN 添加一層cell'''def add_RNN_Cell(inputs): with tf.name_scope('RNN_LSTM_Cell'): with tf.name_scope('weights'):weights = tf.Variable(tf.random_normal(shape=[state_size, num_classes]), name='W') with tf.name_scope('biases'):biases = tf.Variable(tf.constant(0.1, shape=[num_classes,]), name='b')cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=state_size)init_state = cell.zero_state(batch_size, dtype=tf.float32)rnn_outputs, final_state = tf.nn.dynamic_rnn(cell=cell, inputs=x, initial_state=init_state)logits = tf.matmul(final_state[1], weights) + biases return logits

• 網絡結果和loss

123456789

'''RNN網絡結構'''logits = add_RNN_Cell(inputs=x)predictions = tf.nn.softmax(logits) '''loss'''#cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=predictions)cross_entropy = -tf.reduce_sum(y*tf.log(predictions), reduction_indices=[1])with tf.name_scope('losses'):losses = tf.reduce_mean(cross_entropy)

• 結果
  • 總體結構
    
  • RNN內部結構
    

2、可視化訓練過程

• 全部代碼：點擊查看

  (1) 權重weights，偏置biases，損失值loss

• 加入一層全連接層的函數變成這樣

  • 加入tf.summary.histogram(name=layer_name+'/Weights', values=W)即可

    123456789101112131415161718192021222324

    '''定義添加一層全連接層'''def add_fully_layer(inputs, input_size, output_size, num_layer, activation=None):layer_name = 'layer_' + num_layer with tf.name_scope(layer_name): with tf.name_scope('Weights'):low = -4*np.sqrt(6.0/(input_size + output_size)) # use 4 for sigmoid, 1 for tanh activation high = 4*np.sqrt(6.0/(input_size + output_size)) #'''xavier方法初始化''' ##sigmoid #Weights = tf.Variable(tf.random_uniform(shape=[input_size, output_size], minval=low, maxval=high, dtype=tf.float32), name='W') ##reluW = tf.Variable(initial_value=tf.random_uniform(shape=[input_size, output_size], minval=low, maxval=high, dtype=tf.float32)/2, name='W')tf.summary.histogram(name=layer_name+'/Weights', values=W) # summary.histogram with tf.name_scope('biases'):b = tf.Variable(initial_value=tf.zeros(shape=[1, output_size]) + 0.1, name='b')tf.summary.histogram(name=layer_name+'/biases', values=b) # summary.histogram with tf.name_scope('Wx_plus_b'):Wx_plus_b = tf.matmul(inputs, W) + b if activation is not None:outputs = activation(Wx_plus_b) else:outputs = Wx_plus_btf.summary.histogram(name=layer_name+'/outputs', values=outputs) # summary.histogram return outputs

• 損失

  • 損失因為是個具體的數，所以使用scalar?（上面的權重和偏置都是矩陣，向量）

    1	tf.summary.scalar(name='loss_value', tensor=losses)

• 訓練時merge

  • merged = tf.summary.merge_all()合并所有的summary
  • 對于loss, 訓練時執行merge, 然后隨步數不斷加入
    • merged_result = sess.run(merged, feed_dict=feed_dict_train) # 執行merged
    • writer.add_summary(summary=merged_result, global_step=i)

      123456789101112131415

      '''訓練'''def optimize(n_epochs): with tf.Session() as sess:merged = tf.summary.merge_all()writer = tf.summary.FileWriter('logs', sess.graph) # 將計算圖寫入文件sess.run(tf.global_variables_initializer()) for i in range(n_epochs):batch_x, batch_y = data.train.next_batch(batch_size)feed_dict_train = {x: batch_x, y: batch_y}sess.run(train_step, feed_dict=feed_dict_train) if i % 20 == 0:print("epoch:{0}, accuracy:{1}".format(i, sess.run(accuracy, feed_dict=feed_dict_train)))merged_result = sess.run(merged, feed_dict=feed_dict_train) # 執行mergedwriter.add_summary(summary=merged_result, global_step=i) # 加入到writeroptimize(1001)

• 結果

  • loss
    
  • 權重和偏置的數據分布
    

二、Tensorflow?調試tfdbg

• 官網教程：點擊查看
• Youtube視頻簡短教程：點擊查看
• Tensorflow Debugger (tfdbg)是專業的調試工具，可以查看計算圖中內容部的數據等

  1、本地調試

(1) 加入代碼

• 導入調試的包：from tensorflow.python import debug as tf_debug
• Wrapper Session和添加filter:
  • filter也可以自己定義

    123	with tf.Session() as sess:sess = tf_debug.LocalCLIDebugWrapperSession(sess)sess.add_tensor_filter(filter_name='inf or nan', tensor_filter=tf_debug.has_inf_or_nan)

(2) 運行

• 命令行中執行：python xxx.py --debug即可進入調試
  • 支持鼠標點擊的可以直接點擊查看變量的信息
• run或者r可以查看所有的tensor的名字等信息
  • 第一次run還沒有初始化變量，pt tenser_name打印tensor的信息
    
• 再執行一次就是初始化變量
  
  • 可以進行slice
    
• 更多命令行
  

原文地址：　http://lawlite.me/2017/06/24/Tensorflow%E5%AD%A6%E4%B9%A0-%E5%B7%A5%E5%85%B7%E7%9B%B8%E5%85%B3/

總結

以上是生活随笔為你收集整理的Tensorflow学习-工具相关的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。