Global Sensing and Measurements Reuse for Image Compressed Sensing

文章目錄

• Global Sensing and Measurements Reuse for Image Compressed Sensing
• • Abstract
  • Introduction
  • Related work
  • • Traditional Compressed Sensing
    • Deep Compressed Sensing
  • Methodology
  • • GSM
    • GSM+
    • MRB
    • Loss function
  • Experiments
  • Conclusion
  • • Future work

Abstract

現有方法僅僅是從網絡中的部分特征獲得測量值，并且將其用于圖像重建一次。

【Low-level feature： 通常是指圖像中的一些小的細節信息，例如邊緣（edge),角(corner),顏色(color),像素(pixeles), 梯度(gradients)等，這些信息可以通過濾波器、SIFT或HOG獲取；
High level feature：是建立在low level feature之上的，可以用于圖像中目標或物體形狀的識別和檢測，具有更豐富的語義信息。】

所以提出了MR-CCSNet(Measurements Reuse Convolutional Compressed Sensing Network)，其中GSM(Global Sensing Module)用于提取所有特征，MRB(Measurements Reuse Block)用于多次重建。

Introduction

GSM：

用卷積層獲得高維特征

通過多個卷積層逐步壓縮特征圖

收集網絡中所有級別特征

通過$1\times 1$卷積獲得測量值

為了匹配維度，將池層添加到快捷連接中。

MRB：

壓縮相位重建(phased reconstructed)結果并獲得多個特征圖

從測量中提取匹配信息

在多尺度上融合它們

實驗數據集：BSDS500[2]、Set5[4]、Set14[39]

評估指標：PSNR、SSIM

消融實驗：GSM和MRB是有效的

貢獻：

GSM可以實現有效的采樣

MRB：充分利用測量

構建了基于GSM和MRB的圖像CS端到端MR-CSSNet，并測試了有效性

Related work

Traditional Compressed Sensing

即解決sparsity-regularized optimization problem
$$\underset{x}{\min }\frac{1}{2}\Vert \Phi x?y{\Vert }_2^2+\lambda \Vert \Psi x{\Vert }_1$$
$\Psi x$是$x$相對于域$\Psi$的變換系數，$\Psi x$的稀疏性由$1$范數表示

Deep Compressed Sensing

即用神經網絡來求解逆問題，損失函數為
$$\underset{\theta }{\min }\frac{1}{2}\sum _{i=1}^k\Vert x_i?F(y_i,\theta ){\Vert }_2^2$$
$x_i$是原始圖像，$y_i$是觀測，$F$是神經網絡，$\theta$是參數

[37]中是LAPRAN，來通過不同分辨率的多個階段同時重建原始圖像。

Methodology

采樣率：6.25%

MR-CCSNet：【a sensing network GSM】+【an initial reconstruction network】+【a deep reconstruction network】

Obtain the measurements $y$ from the sensing network

The initial reconstruction network generates initial reconstructed image by a linear mapping

Refine the initial reconstructed image by a non-linear deep reconstruction network【initial reconstructed image is not enough】

【stack multiple MRBs in the deep reconstruction network】

Sensing network $S(?)$（GSM?）：

Use convolutional layers for the whole images.【直接對整個圖像進行卷積，而不是將圖像劃分為非重疊塊】

There is no bias and activation function.【為了滿足線性特性】

$y=S(x)$，$x\in {\mathbb{R}}^{1\times H\times W}$，$y\in {\mathbb{R}}^{4\times \frac{H}{8}\times \frac{W}{8}}$

Initial reconstruction network $I(?)$：the first time to utilize the measurements

Depth-wise convolution layer expands the measurements in channel dimension and the shape of channel becomes $64\times \frac{H}{8}\times \frac{W}{8}$.

Get a $1\times H\times W$ tensor by a pixel shuffle layer.

【pixel shuffle layer：一種對低分辨率特征圖上采樣的思路，假設打算將$H\times W\times C$的特征圖在長和寬的維度上擴大$r$倍變成$rH\times rW\times C$，則通過深度為$r^{2}C$的卷積對$H\times W\times C$的特征圖進行卷積得到$H\times W\times r^{2}C$的特征圖，再通過“周期洗牌”的操作變成$rH\times rW\times C$】

【pixel shuffle layer就是輸入$H\times W$的低分辨率的圖像，輸出$rH\times rW$的高分辨率的圖像】

Deep reconstruction network $D(?)$：the second time to utilize the measurements

Convert the initial reconstructed $I(y)$ image to a high dimensional feature by a convolutional layer.

Repeat MRBs.

Fuse the MRBs with matching features extracted from measurements $y$ multiple times on multi-scale.【用于將它們與多尺度上多次從測量$y$中提取的匹配特征融合】

Finally：

Use a convolutional layer to reconstruct the image from high dimensional features.

Add a shortcut connection to the deep reconstruction network.【shortcut connection是快捷連接，用于解決網絡退化問題】

The final reconstructed image $\hat{x}$：
$$\hat{x}=D(I(y))+I(y)$$

GSM

【卷積神經網絡以分層方式提取特征，則靠近輸入的層學習低級特征，如線條和簡單紋理。而深的層學習高級特征，如形狀】

First stage：Use $3\times 3$ convolution layers to extract features.

Second stage：Collect all level features in the network.

Use a $1\times 1$ convolution layer to sample, rather than only from the low features (i.e. CSNet+) or high features (i.e. RK-CCSNet).

【To collect all level features for sampling, we use a shortcut connection to pass the features of different layers to the end, and the pooling layer is added for matching the dimensions.】

【采樣率變化時，GSM不能很好的適配，故提出GSM+】

GSM+

Different from GSM：

Add a shortcut connection between two successive layers rather than add it from different layers to the end directly.

The building block of GSM+：
$$y_{t+1}=Conv(y_t)+P(y_t)$$
Conv and P denote convolution layer and meanpooling layer.

The sampling ratio is controlled by the number of building block and the blue block.

When the sampling ratio is $50\%$, there is only one building block in GSM+, so GSM+ degenerate into GSM.

MRB

Phased recontructed result $f_t\in {\mathbb{R}}^{C\times H\times W}$ and measurements $y\in {\mathbb{R}}^{C\times \frac{H}{4}\times \frac{W}{4}}$ are fed into MRB.

Use two convolutional layers, denoted as $Con{v}_1$ and $Con{v}_2$, to obtain a compacted feature map $f^{\downarrow }\in {\mathbb{R}}^{C\times \frac{H}{2}\times \frac{W}{2}}$ and $f^{\downarrow \downarrow }\in {\mathbb{R}}^{C\times \frac{H}{4}\times \frac{W}{4}}$.

$$\begin{gathered} f^{\downarrow }=Con{v}_1(f_t), \\ {f}^{\downarrow \downarrow }=Con{v}_2(f^{\downarrow }). \end{gathered}$$

Fig.5 extract matching information from measurements and obtain three feature maps $y_1\in {\mathbb{R}}^{C\times \frac{H}{4}\times \frac{W}{4}}$, $y_2\in {\mathbb{R}}^{C\times \frac{H}{2}\times \frac{W}{2}}$ and $y_3\in {\mathbb{R}}^{C\times H\times W}$ by Multi-Scale Reusing.

$F_1=Con{v}_3(f^{\downarrow \downarrow }\oplus y_1)$, in the third block. Then copy the $f^{\downarrow \downarrow }$ and fuse them with $F_1$.【怎么fuse的】

接著pixel shuffle + Conv，即$f^{\uparrow }=Pixel(Con{v}_4(F_1\oplus f^{\downarrow \downarrow }))$，接著與$y_2$進行上述步驟。（$F_1\in {\mathbb{R}}^{C\times \frac{H}{4}\times \frac{W}{4}}$, $f^{\uparrow }\in {\mathbb{R}}^{C\times \frac{H}{2}\times \frac{W}{2}}$）后面依次下去，得到$f_{t+1}\in {\mathbb{R}}^{C\times H\times W}$

對于這塊的channel融合不是很理解，如果有大佬明白的話，希望給我講一下。

Loss function

For the initial reconstruction network, $l_{int}={\sum }_{k=1}^n\Vert I(S(y_k;\theta );{?}_{int})?x_k{\Vert }_F^2$.

For the deep reconstruction network, $l_{deep}={\sum }_{k=1}^n\Vert D(I(S(y_k;\theta );{?}_{int});{?}_{deep})?x_k{\Vert }_F^2$

$\theta$, ${?}_{int}$, ${?}_{deep}$ denote the parameters of $S(?)$, $I(?)$ and $D(?)$

The loss of MR-CCSNet is $l=l_{deep}+l_{int}$.

Experiments

Training datasets: 400 images from BSDS500[2]

Three standard benchmark datasets: Set5[4], Set14[39], BSDS100[2]

Conclusion

Future work

In the sensing network, pooling operation loses information about the low-level features.

Attention mechanism can effectively help us in extracting matching features from measurements.

In the real-world, because there are noise in the measurements, using them multiple times will introduce noise in the reconstruction process.
ching features from measurements.

In the real-world, because there are noise in the measurements, using them multiple times will introduce noise in the reconstruction process.

總結

以上是生活随笔為你收集整理的Global Sensing and Measurements Reuse for Image Compressed Sensing的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。