引言

本文主要用于對(duì)7個(gè)經(jīng)典卷積神經(jīng)網(wǎng)絡(luò)的初識(shí)，大致了解各個(gè)網(wǎng)絡(luò)提出的背景，以及各自對(duì)卷積神經(jīng)網(wǎng)絡(luò)發(fā)展的作用（即網(wǎng)絡(luò)的特點(diǎn)）。

• 經(jīng)典的卷積神經(jīng)網(wǎng)絡(luò)：
  • LeNet
  • AlexNet
  • ZF Net
  • VGG
  • GoogLeNet
  • ResNet
  • DenseNet

參考： CNN網(wǎng)絡(luò)架構(gòu)演進(jìn)：從LeNet到DenseNet（本文內(nèi)容來(lái)源，強(qiáng)力推薦）

LeNet（5層）

• 提出背景：LeCun在1998年提出，用于解決手寫(xiě)數(shù)字識(shí)別的視覺(jué)任務(wù)。

• 歷史意義：定義了CNN的基本組件，是CNN的鼻祖。自此，CNN的最基本的架構(gòu)確定下來(lái)：卷積層、池化層、全連接層。如今各大深度學(xué)習(xí)框架中所使用的LeNet都是簡(jiǎn)化改進(jìn)過(guò)的LeNet-5（-5表示具有5個(gè)層），和原始的LeNet有些許不同，比如把激活函數(shù)改為了現(xiàn)在很常用的ReLu。

• 經(jīng)典的LeNet-5網(wǎng)絡(luò)結(jié)構(gòu)：如下圖。

• 數(shù)據(jù)矩陣計(jì)算：
  • 輸入是單通道的28*28大小矩陣，用矩陣表示就是[1,28,28]；
  • cov1：20個(gè)5*5的卷積核，滑動(dòng)步長(zhǎng)為1，經(jīng)過(guò)該層后尺寸變?yōu)?4，28-5+1=24，輸出矩陣為[20,24,24]；
  • pool1：2*2，步長(zhǎng)2，池化操作后，尺寸減半，變?yōu)?2×12，輸出矩陣為[20,12,12]；
  • cov2：50個(gè)5*5的卷積核，步長(zhǎng)1，卷積后尺寸變?yōu)?,12-5+1=8，輸出矩陣為[50,8,8]；
  • pool2：2*2，步長(zhǎng)2，池化操作后，尺寸減半，變?yōu)?×4，輸出矩陣為[50,4,4]；
  • fc1：輸入為[50,4,4]，神經(jīng)元數(shù)目為500，再接relu激活函數(shù)，輸出為500；
  • fc2：輸入為500，神經(jīng)元個(gè)數(shù)為10，得到10維的特征向量；
  • output：輸入為10維的特征向量，送入softmaxt分類(lèi)，得到分類(lèi)結(jié)果的概率output。

    cov為卷積層，pool為池化層，fc的全連接層。

AlexNet（8層）

• 提出背景：AlexNet在2012年ImageNet競(jìng)賽中以超過(guò)第二名10.9個(gè)百分點(diǎn)的絕對(duì)優(yōu)勢(shì)一舉奪冠，從此深度學(xué)習(xí)和卷積神經(jīng)網(wǎng)絡(luò)名聲鵲起，深度學(xué)習(xí)的研究如雨后春筍般出現(xiàn)，AlexNet的出現(xiàn)可謂是卷積神經(jīng)網(wǎng)絡(luò)的王者歸來(lái)。

• 歷史意義：

  • 更深的網(wǎng)絡(luò)
  • 數(shù)據(jù)增廣：數(shù)據(jù)增廣技巧來(lái)增加模型泛化能力；
  • ReLU：ReLU代替Sigmoid來(lái)加快SGD的收斂速度；
  • dropout：有效緩解了模型的過(guò)擬合；
  • LRN：局部響應(yīng)歸一化，減小高激活神經(jīng)元的使用，后發(fā)現(xiàn)這個(gè)不起作用，目前已棄用。

Dropout原理類(lèi)似于淺層學(xué)習(xí)算法的中集成算法，該方法通過(guò)讓全連接層的神經(jīng)元（該模型在前兩個(gè)全連接層引入Dropout）以一定的概率失去活性（比如0.5）失活的神經(jīng)元不再參與前向和反向傳播，相當(dāng)于約有一半的神經(jīng)元不再起作用。在測(cè)試的時(shí)候，讓所有神經(jīng)元的輸出乘0.5。Dropout的引用，有效緩解了模型的過(guò)擬合。

Local Responce Normalization（LRN）：局部響應(yīng)歸一層的基本思路是，假如這是網(wǎng)絡(luò)的一塊，比如是 13×13×256， LRN 要做的就是選取一個(gè)位置，比如說(shuō)這樣一個(gè)位置，從這個(gè)位置穿過(guò)整個(gè)通道，能得到 256 個(gè)數(shù)字，并進(jìn)行歸一化。進(jìn)行局部響應(yīng)歸一化的動(dòng)機(jī)是，對(duì)于這張 13×13 的圖像中的每個(gè)位置來(lái)說(shuō)，我們可能并不需要太多的高激活神經(jīng)元。但是后來(lái)，很多研究者發(fā)現(xiàn) LRN 起不到太大作用，因?yàn)椴⒉恢匾?#xff0c;而且我們現(xiàn)在并不用 LRN 來(lái)訓(xùn)練網(wǎng)絡(luò)。

ZF-Net（8層）

• 提出背景：ZFNet是2013ImageNet分類(lèi)任務(wù)的冠軍，其網(wǎng)絡(luò)結(jié)構(gòu)沒(méi)什么改進(jìn)，主要是參數(shù)的變化，性能較Alex提升了不少。ZF-Net只是將AlexNet第一層卷積核由11變成7，步長(zhǎng)由4變?yōu)?，第3，4，5卷積層轉(zhuǎn)變?yōu)?84，384，256。

VGG-Nets（19層）

• 提出背景：VGG-Nets是由牛津大學(xué)VGG（Visual Geometry Group）提出，是2014年ImageNet競(jìng)賽定位任務(wù)的第一名和分類(lèi)任務(wù)的第二名的中的基礎(chǔ)網(wǎng)絡(luò)。VGG可以看成是加深版本的AlexNet. 都是conv layer + FC layer，在當(dāng)時(shí)看來(lái)這是一個(gè)非常深的網(wǎng)絡(luò)了，因?yàn)閷訑?shù)高達(dá)十多層，我們從其論文名字就知道了（《Very Deep Convolutional Networks for Large-Scale Visual Recognition》），當(dāng)然以現(xiàn)在的目光看來(lái)VGG真的稱(chēng)不上是一個(gè)very deep的網(wǎng)絡(luò)。

• 歷史意義：

  • 卷積層使用更小的filter尺寸和間隔：VGG-Nets，用到的卷積核的尺寸無(wú)非都是1×1和3×3的小卷積核，可以替代大的filter尺寸；

3×3卷積核的優(yōu)點(diǎn)：

• 多個(gè)3×3的卷基層比一個(gè)大尺寸filter卷基層有更多的非線(xiàn)性，使得判決函數(shù)更加具有判決性
• 多個(gè)3×3的卷積層比一個(gè)大尺寸的filter有更少的參數(shù)，假設(shè)卷基層的輸入和輸出的特征圖大小相同為C，那么三個(gè)3×3的卷積層參數(shù)個(gè)數(shù)3×（3×3×C×C）=27CC；一個(gè)7×7的卷積層參數(shù)為49CC；所以可以把三個(gè)3×3的filter看成是一個(gè)7×7filter的分解（中間層有非線(xiàn)性的分解）

1*1卷積核的優(yōu)點(diǎn)：作用是在不影響輸入輸出維數(shù)的情況下，對(duì)輸入進(jìn)行線(xiàn)性形變，然后通過(guò)Relu進(jìn)行非線(xiàn)性處理，增加網(wǎng)絡(luò)的非線(xiàn)性表達(dá)能力。

GoogLeNet（22層）

• 提出背景：GoogLeNet在2014的ImageNet分類(lèi)任務(wù)上擊敗了VGG-Nets奪得冠軍，其實(shí)力肯定是非常深厚的，GoogLeNet跟AlexNet,VGG-Nets這種單純依靠加深網(wǎng)絡(luò)結(jié)構(gòu)進(jìn)而改進(jìn)網(wǎng)絡(luò)性能的思路不一樣，它另辟幽徑，在加深網(wǎng)絡(luò)的同時(shí)（22層），也在網(wǎng)絡(luò)結(jié)構(gòu)上做了創(chuàng)新，引入Inception結(jié)構(gòu)代替了單純的卷積+激活的傳統(tǒng)操作（這思路最早由Network in Network提出）。GoogLeNet進(jìn)一步把對(duì)卷積神經(jīng)網(wǎng)絡(luò)的研究推上新的高度。

• 歷史意義：
  • 引入Inception結(jié)構(gòu)；
  • 中間層的輔助LOSS單元；
  • 后面的全連接層全部替換為簡(jiǎn)單的全局平均pooling。

ResNet（152層）

• 提出背景：2015年何愷明推出的ResNet在ISLVRC和COCO上橫掃所有選手，獲得冠軍。ResNet在網(wǎng)絡(luò)結(jié)構(gòu)上做了大創(chuàng)新，而不再是簡(jiǎn)單的堆積層數(shù)，ResNet在卷積神經(jīng)網(wǎng)絡(luò)的新思路，絕對(duì)是深度學(xué)習(xí)發(fā)展歷程上里程碑式的事件。

• 歷史意義：

  • 層數(shù)非常深，已經(jīng)超過(guò)百層；
  • 引入殘差單元來(lái)解決退化問(wèn)題；

• 維度匹配方案：

  • zero_padding:對(duì)恒等層進(jìn)行0填充的方式將維度補(bǔ)充完整，這種方法不會(huì)增加額外的參數(shù)；
  • projection:在恒等層采用1x1的卷積核來(lái)增加維度。這種方法會(huì)增加額外的參數(shù)。

DenseNet

• 提出背景：自Resnet提出以后，ResNet的變種網(wǎng)絡(luò)層出不窮，都各有其特點(diǎn)，網(wǎng)絡(luò)性能也有一定的提升。本文介紹的最后一個(gè)網(wǎng)絡(luò)是CVPR 2017最佳論文DenseNet，論文中提出的DenseNet（Dense Convolutional Network）主要還是和ResNet及Inception網(wǎng)絡(luò)做對(duì)比，思想上有借鑒，但卻是全新的結(jié)構(gòu)，網(wǎng)絡(luò)結(jié)構(gòu)并不復(fù)雜，卻非常有效，在CIFAR指標(biāo)上全面超越ResNet。可以說(shuō)DenseNet吸收了ResNet最精華的部分，并在此上做了更加創(chuàng)新的工作，使得網(wǎng)絡(luò)性能進(jìn)一步提升。

• 歷史意義：密集連接：緩解梯度消失問(wèn)題，加強(qiáng)特征傳播，鼓勵(lì)特征復(fù)用，極大的減少了參數(shù)量

在同層深度下獲得更好的收斂率，DenseNet具有非常強(qiáng)大的內(nèi)存占用。

keras代碼實(shí)現(xiàn)：

• LeNet的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10

  def LeNet(): model = Sequential() model.add(Conv2D(32,(5,5),strides=(1,1),input_shape=(28,28,1),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(64,(5,5),strides=(1,1),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Flatten()) model.add(Dense(100,activation='relu')) model.add(Dense(10,activation='softmax')) return model

• AlexNet的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

  def AlexNet(): model = Sequential() model.add(Conv2D(96,(11,11),strides=(4,4),input_shape=(227,227,3),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(256,(5,5),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• ZF-Net的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

  def ZF_Net(): model = Sequential() model.add(Conv2D(96,(7,7),strides=(2,2),input_shape=(224,224,3),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(256,(5,5),strides=(2,2),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• VGG-16的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

  def VGG_16(): model = Sequential() model.add(Conv2D(64,(3,3),strides=(1,1),input_shape=(224,224,3),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(64,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(128,(3,2),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(128,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• GoogLeNet的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

  def Conv2d_BN(x, nb_filter,kernel_size, padding='same',strides=(1,1),name=None): if name is not None: bn_name = name + '_bn' conv_name = name + '_conv' else: bn_name = None conv_name = None x = Conv2D(nb_filter,kernel_size,padding=padding,strides=strides,activation='relu',name=conv_name)(x) x = BatchNormalization(axis=3,name=bn_name)(x) return x def Inception(x,nb_filter): branch1x1 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch3x3 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch3x3 = Conv2d_BN(branch3x3,nb_filter,(3,3), padding='same',strides=(1,1),name=None) branch5x5 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch5x5 = Conv2d_BN(branch5x5,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branchpool = MaxPooling2D(pool_size=(3,3),strides=(1,1),padding='same')(x) branchpool = Conv2d_BN(branchpool,nb_filter,(1,1),padding='same',strides=(1,1),name=None) x = concatenate([branch1x1,branch3x3,branch5x5,branchpool],axis=3) return x def GoogLeNet(): inpt = Input(shape=(224,224,3)) #padding = 'same'，填充為(步長(zhǎng)-1）/2,還可以用ZeroPadding2D((3,3)) x = Conv2d_BN(inpt,64,(7,7),strides=(2,2),padding='same') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Conv2d_BN(x,192,(3,3),strides=(1,1),padding='same') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,64)#256 x = Inception(x,120)#480 x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,128)#512 x = Inception(x,128) x = Inception(x,128) x = Inception(x,132)#528 x = Inception(x,208)#832 x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,208) x = Inception(x,256)#1024 x = AveragePooling2D(pool_size=(7,7),strides=(7,7),padding='same')(x) x = Dropout(0.4)(x) x = Dense(1000,activation='relu')(x) x = Dense(1000,activation='softmax')(x) model = Model(inpt,x,name='inception') return model

• ResNet-50的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

  def Conv2d_BN(x, nb_filter,kernel_size, strides=(1,1), padding='same',name=None): if name is not None: bn_name = name + '_bn' conv_name = name + '_conv' else: bn_name = None conv_name = None x = Conv2D(nb_filter,kernel_size,padding=padding,strides=strides,activation='relu',name=conv_name)(x) x = BatchNormalization(axis=3,name=bn_name)(x) return x def Conv_Block(inpt,nb_filter,kernel_size,strides=(1,1), with_conv_shortcut=False): x = Conv2d_BN(inpt,nb_filter=nb_filter[0],kernel_size=(1,1),strides=strides,padding='same') x = Conv2d_BN(x, nb_filter=nb_filter[1], kernel_size=(3,3), padding='same') x = Conv2d_BN(x, nb_filter=nb_filter[2], kernel_size=(1,1), padding='same') if with_conv_shortcut: shortcut = Conv2d_BN(inpt,nb_filter=nb_filter[2],strides=strides,kernel_size=kernel_size) x = add([x,shortcut]) return x else: x = add([x,inpt]) return x def ResNet50(): inpt = Input(shape=(224,224,3)) x = ZeroPadding2D((3,3))(inpt) x = Conv2d_BN(x,nb_filter=64,kernel_size=(7,7),strides=(2,2),padding='valid') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3),strides=(1,1),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3)) x = AveragePooling2D(pool_size=(7,7))(x) x = Flatten()(x) x = Dense(1000,activation='softmax')(x) model = Model(inputs=inpt,outputs=x) return model

• DenseNet-121的Keras實(shí)現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160

  def DenseNet121(nb_dense_block=4, growth_rate=32, nb_filter=64, reduction=0.0, dropout_rate=0.0, weight_decay=1e-4, classes=1000, weights_path=None): '''Instantiate the DenseNet 121 architecture, # Arguments nb_dense_block: number of dense blocks to add to end growth_rate: number of filters to add per dense block nb_filter: initial number of filters reduction: reduction factor of transition blocks. dropout_rate: dropout rate weight_decay: weight decay factor classes: optional number of classes to classify images weights_path: path to pre-trained weights # Returns A Keras model instance. ''' eps = 1.1e-5 # compute compression factor compression = 1.0 - reduction # Handle Dimension Ordering for different backends global concat_axis if K.image_dim_ordering() == 'tf': concat_axis = 3 img_input = Input(shape=(224, 224, 3), name='data') else: concat_axis = 1 img_input = Input(shape=(3, 224, 224), name='data') # From architecture for ImageNet (Table 1 in the paper) nb_filter = 64 nb_layers = [6,12,24,16] # For DenseNet-121 # Initial convolution x = ZeroPadding2D((3, 3), name='conv1_zeropadding')(img_input) x = Convolution2D(nb_filter, 7, 7, subsample=(2, 2), name='conv1', bias=False)(x) x = BatchNormalization(epsilon=eps, axis=concat_axis, name='conv1_bn')(x) x = Scale(axis=concat_axis, name='conv1_scale')(x) x = Activation('relu', name='relu1')(x) x = ZeroPadding2D((1, 1), name='pool1_zeropadding')(x) x = MaxPooling2D((3, 3), strides=(2, 2), name='pool1')(x) # Add dense blocks for block_idx in range(nb_dense_block - 1): stage = block_idx+2 x, nb_filter = dense_block(x, stage, nb_layers[block_idx], nb_filter, growth_rate, dropout_rate=dropout_rate, weight_decay=weight_decay) # Add transition_block x = transition_block(x, stage, nb_filter, compression=compression, dropout_rate=dropout_rate, weight_decay=weight_decay) nb_filter = int(nb_filter * compression) final_stage = stage + 1 x, nb_filter = dense_block(x, final_stage, nb_layers[-1], nb_filter, growth_rate, dropout_rate=dropout_rate, weight_decay=weight_decay) x = BatchNormalization(epsilon=eps, axis=concat_axis, name='conv'+str(final_stage)+'_blk_bn')(x) x = Scale(axis=concat_axis, name='conv'+str(final_stage)+'_blk_scale')(x) x = Activation('relu', name='relu'+str(final_stage)+'_blk')(x) x = GlobalAveragePooling2D(name='pool'+str(final_stage))(x) x = Dense(classes, name='fc6')(x) x = Activation('softmax', name='prob')(x) model = Model(img_input, x, name='densenet') if weights_path is not None: model.load_weights(weights_path) return model def conv_block(x, stage, branch, nb_filter, dropout_rate=None, weight_decay=1e-4): '''Apply BatchNorm, Relu, bottleneck 1x1 Conv2D, 3x3 Conv2D, and option dropout # Arguments x: input tensor stage: index for dense block branch: layer index within each dense block nb_filter: number of filters dropout_rate: dropout rate weight_decay: weight decay factor ''' eps = 1.1e-5 conv_name_base = 'conv' + str(stage) + '_' + str(branch) relu_name_base = 'relu' + str(stage) + '_' + str(branch) # 1x1 Convolution (Bottleneck layer) inter_channel = nb_filter * 4 x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_x1_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_x1_scale')(x) x = Activation('relu', name=relu_name_base+'_x1')(x) x = Convolution2D(inter_channel, 1, 1, name=conv_name_base+'_x1', bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) # 3x3 Convolution x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_x2_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_x2_scale')(x) x = Activation('relu', name=relu_name_base+'_x2')(x) x = ZeroPadding2D((1, 1), name=conv_name_base+'_x2_zeropadding')(x) x = Convolution2D(nb_filter, 3, 3, name=conv_name_base+'_x2', bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) return x def transition_block(x, stage, nb_filter, compression=1.0, dropout_rate=None, weight_decay=1E-4): ''' Apply BatchNorm, 1x1 Convolution, averagePooling, optional compression, dropout # Arguments x: input tensor stage: index for dense block nb_filter: number of filters compression: calculated as 1 - reduction. Reduces the number of feature maps in the transition block. dropout_rate: dropout rate weight_decay: weight decay factor ''' eps = 1.1e-5 conv_name_base = 'conv' + str(stage) + '_blk' relu_name_base = 'relu' + str(stage) + '_blk' pool_name_base = 'pool' + str(stage) x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_scale')(x) x = Activation('relu', name=relu_name_base)(x) x = Convolution2D(int(nb_filter * compression), 1, 1, name=conv_name_base, bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) x = AveragePooling2D((2, 2), strides=(2, 2), name=pool_name_base)(x) return x def dense_block(x, stage, nb_layers, nb_filter, growth_rate, dropout_rate=None, weight_decay=1e-4, grow_nb_filters=True): ''' Build a dense_block where the output of each conv_block is fed to subsequent ones # Arguments x: input tensor stage: index for dense block nb_layers: the number of layers of conv_block to append to the model. nb_filter: number of filters growth_rate: growth rate dropout_rate: dropout rate weight_decay: weight decay factor grow_nb_filters: flag to decide to allow number of filters to grow ''' eps = 1.1e-5 concat_feat = x for i in range(nb_layers): branch = i+1 x = conv_block(concat_feat, stage, branch, growth_rate, dropout_rate, weight_decay) concat_feat = merge([concat_feat, x], mode='concat', concat_axis=concat_axis, name='concat_'+str(stage)+'_'+str(branch)) if grow_nb_filters: nb_filter += growth_rate return concat_feat, nb_filter

總結(jié)

以上是生活随笔為你收集整理的经典的卷积神经网络简介的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。