http://arxiv.org/abs/1412.6806

CNN網絡貌似由簡單到復雜，再由復雜到簡單的趨勢

當前主流的CNN網絡一般都包括以下幾個模塊：convolution and max-pooling layers和交替，最后是少數全鏈接層。本文對此深入分析，提出了只有卷積層的網絡，發現效果不錯。

首先來看看 pooling， why pooling can help in CNNs，可能的解釋有三個：
1） the p-norm makes the representation in a CNN more invariant
2） the spatial dimensionality reduction performed by pooling makes
covering larger parts of the input in higher layers possible
3） the feature-wise nature of the pooling operation (as opposed to a convolutional layer where features get mixed) could make optimization easier.

假定第二條是最重要的，那么我們可以有2中方法得到去除pooling但保留spatial dimensionality reduction
1）We can remove each pooling layer and increase the stride of the convolutional layer that preceded it accordingly.
2）We can replace the pooling layer by a normal convolution with stride larger than one。（對于一個k=3,r=2的池化層，我們可以用一個對應的stride和kernel的卷積層取代，輸入個數輸出個數一樣。）

第一個方法有一個缺陷： we significantly reduce the overlap of the convolutional layer that preceded the pooling layer. It is equivalent to a pooling operation in which only the top-left feature response is considered and can result in less accurate recognition.

第二個方法則沒有這個問題。

對于全鏈接層， if the image area covered by units in the topmost convolutional layer covers a portion of the image large enough to recognize its content (i.e. the object we want to recognize) then fully connected layers can also be replaced by simple 1-by-1 convolutions.

總結

以上是生活随笔為你收集整理的Striving for Simplicity: The All Convolutional Net的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。