在看文章時(shí)，一篇文章提到了使用elman神經(jīng)網(wǎng)絡(luò)來對(duì)癲癇病人的腦電信號(hào)與正常人的腦電信號(hào)進(jìn)行區(qū)分，并且取得了較好的分類結(jié)果。于是就想自己寫一個(gè)elman神經(jīng)網(wǎng)絡(luò)demo看看效果。

elman神經(jīng)網(wǎng)絡(luò)和感知機(jī)的差別通過下面的圖片可以很明顯的看出哪里不一樣，elman在隱藏層多了一個(gè)“context units“，用來保存隱藏層的輸出，并作用到下一次隱藏層的計(jì)算中，關(guān)于elman原理的說明，大家可以自己查閱一些資料，這里不再贅述。（圖片來自維基百科https://en.wikipedia.org/wiki/Recurrent_neural_network）

""" coding: utf-8 @author: zhangxiang """ """ 在對(duì)腦電信號(hào)進(jìn)行分類的時(shí)候，發(fā)現(xiàn)一篇文章對(duì)健康人，癲癇患者未發(fā)作時(shí)的腦電信號(hào)和癲癇發(fā)作時(shí)的腦電信號(hào)的分類使用了基于時(shí)序的 elman_RNN 神經(jīng)網(wǎng)絡(luò)進(jìn)行建模，于是想在預(yù)測(cè)麻醉深度類別及其它時(shí)序相關(guān)的分類問題上使用這一模型。 就寫了一個(gè)demo """ import numpy as npclass ELMAN_RNN(object):def __init__(self, input_num, hidden_num, output_num, learning_rate):self.input_num = input_numself.hidden_num = hidden_numself.output_num = output_numself.learning_rate = learning_rateself.hidden_weights = np.random.random((self.input_num, self.hidden_num))self.output_weights = np.random.random((self.hidden_num, self.output_num))self.rnn_weights = np.random.random((self.hidden_num, self.hidden_num))self.hidden_bias = np.random.rand(1)self.output_bias = np.random.rand(1)self.hidden_output = np.zeros((1, self.hidden_num))def training(self, train_input, train_output):"""training one time"""output = self.feed_forward(train_input)self.bptt(train_input, output, train_output)def calculate_the_cross_entropy(self, training_set):"""get the total error loss"""loss = 0for i in range(np.array(training_set).shape[0]):x, y = training_set[i]y = np.array(y).reshape(1,2)result = self.feed_forward(x)loss += self.get_the_total_error(y, result)return lossdef get_the_total_error(self, y, result):"""loss = -∑yi*ln(ai), y is the real label, result is the softmax result"""loss = -np.sum(y*np.log(result))return lossdef feed_forward(self, input):"""calculate feed_forward value"""self.hidden_output = self.sigmoid(np.dot(np.array(input).reshape(1,2), self.hidden_weights) + np.dot(self.hidden_output, self.rnn_weights) + self.hidden_bias)return self.softmax(np.dot(self.hidden_output, self.output_weights) + self.output_bias)def bptt(self,input, output, train_output):"""update the weights of all layers"""# claculate delta of output layersdelta_of_output_layers = [0]*self.output_numfor i in range(self.output_num):delta_of_output_layers[i] = self.calculate_output_wrt_rawout(output[0, i], train_output[i])# caculate delta of hidden layersdelta_of_hidden_layers = [0]*self.hidden_numfor i in range(self.hidden_num):d_error_wrt_hidden_output = 0.0for o in range(self.output_num):d_error_wrt_hidden_output += delta_of_output_layers[o]*self.output_weights[i, o]delta_of_hidden_layers[i] = d_error_wrt_hidden_output*self.calculate_output_wrt_netinput(self.hidden_output[0,i])# get the δw of output layers and update the weightsfor i in range(self.output_num):for weight_j in range(self.output_weights.shape[0]):delta_wrt_weight_j = delta_of_output_layers[i]*self.hidden_output[0,weight_j]self.output_weights[weight_j, i] -= self.learning_rate*delta_wrt_weight_j# get the δw of hidden layers and update the weightsfor i in range(self.hidden_num):for weight_j in range(self.hidden_weights.shape[0]):delta_wrt_weight_j = delta_of_hidden_layers[i]*input[weight_j]self.hidden_weights[weight_j, i] -= self.learning_rate*delta_wrt_weight_jdef sigmoid(self, x):"""activation function"""return 1.0/(1.0 + np.exp(-x))def softmax(self, x):"""the activation for multiple output function"""return np.exp(x)/np.sum(np.exp(x))def calculate_output_wrt_rawout(self, output, train_output):"""derivative of softmax function, actually in classification train_output equal to 1"""return (output - train_output)def calculate_output_wrt_netinput(self, output):"""the derivative of sigmoid function"""return output*(1 - output)if __name__ == "__main__":import matplotlib.pyplot as pltelman = ELMAN_RNN(input_num=2, hidden_num=4, output_num=2, learning_rate=0.02)train_x = [[1,2], [1,1], [1.5, 1.5], [2,1], [-1,-1], [-0.5, -0.5], [-1, -2], [-2, -1.5]]label_y = [[1,0], [1,0], [1,0], [1,0], [0,1], [0,1], [0,1], [0,1]]training_sets = [[[2,2],[1,0]], [[0.2, 0.8], [1,0]], [[-0.5, -0.8], [0, 1]], [[-1.2, -0.5], [0, 1]]]loss = []for i in range(1000):for x, y in zip(train_x, label_y):elman.training(x, y)loss.append(elman.calculate_the_cross_entropy(training_sets))plt.figure()plt.plot(loss)plt.title('the loss with the training')plt.show()print('training finished!')

loss函數(shù)的變化如下：

總結(jié)

以上是生活随笔為你收集整理的elman神经网络的实现的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。