總結自論文：Faster_RCNN，與Pytorch代碼：

本文主要介紹代碼最后部分：trainer.py ?、train.py ， 首先分析一些主要理論操作，然后在代碼分析里詳細介紹其具體實現。首先是訓練與測試的過程圖：

? ? ? ? ?

?

還是要再次強調：

AnchorTargetCreator和ProposalTargetCreator是為了生成訓練的目標（或稱ground truth），只在訓練階段用到，ProposalCreator是RPN為Fast R-CNN生成RoIs，在訓練和測試階段都會用到。所以測試階段直接輸進來300個RoIs，而訓練階段會有AnchorTargetCreator的再次干預。

在ROI Pooling過程中，首先sample_rois中的坐標將feature（512，w/16,h/16）劃分為不同的roi_feature_map(_,512,w/16,w/16),再經過ROI Pooling操作，類似SPP那樣將特征圖下采樣到同樣的大小（_,512,7,7）。

?

?

一. 代碼分析

1.trainer.py

from collections import namedtuple import time from torch.nn import functional as F from model.utils.creator_tool import AnchorTargetCreator, ProposalTargetCreatorfrom torch import nn import torch as t from torch.autograd import Variable from utils import array_tool as at from utils.vis_tool import Visualizerfrom utils.config import opt from torchnet.meter import ConfusionMeter, AverageValueMeterLossTuple = namedtuple('LossTuple',['rpn_loc_loss','rpn_cls_loss','roi_loc_loss','roi_cls_loss','total_loss'])class FasterRCNNTrainer(nn.Module):"""wrapper for conveniently training. return lossesThe losses include:* :obj:`rpn_loc_loss`: The localization loss for \Region Proposal Network (RPN).* :obj:`rpn_cls_loss`: The classification loss for RPN.* :obj:`roi_loc_loss`: The localization loss for the head module.* :obj:`roi_cls_loss`: The classification loss for the head module.* :obj:`total_loss`: The sum of 4 loss above.Args:faster_rcnn (model.FasterRCNN):A Faster R-CNN model that is going to be trained."""def __init__(self, faster_rcnn):super(FasterRCNNTrainer, self).__init__()self.faster_rcnn = faster_rcnnself.rpn_sigma = opt.rpn_sigmaself.roi_sigma = opt.roi_sigma# target creator create gt_bbox gt_label etc as training targets. self.anchor_target_creator = AnchorTargetCreator()self.proposal_target_creator = ProposalTargetCreator()self.loc_normalize_mean = faster_rcnn.loc_normalize_meanself.loc_normalize_std = faster_rcnn.loc_normalize_stdself.optimizer = self.faster_rcnn.get_optimizer()# visdom wrapperself.vis = Visualizer(env=opt.env)# indicators for training statusself.rpn_cm = ConfusionMeter(2)self.roi_cm = ConfusionMeter(21)self.meters = {k: AverageValueMeter() for k in LossTuple._fields} # average lossdef forward(self, imgs, bboxes, labels, scale):"""Forward Faster R-CNN and calculate losses.Here are notations used.* :math:`N` is the batch size.* :math:`R` is the number of bounding boxes per image.Currently, only :math:`N=1` is supported.Args:imgs (~torch.autograd.Variable): A variable with a batch of images.bboxes (~torch.autograd.Variable): A batch of bounding boxes.Its shape is :math:`(N, R, 4)`.labels (~torch.autograd..Variable): A batch of labels.Its shape is :math:`(N, R)`. The background is excluded fromthe definition, which means that the range of the valueis :math:`[0, L - 1]`. :math:`L` is the number of foregroundclasses.scale (float): Amount of scaling applied tothe raw image during preprocessing.Returns:namedtuple of 5 losses"""n = bboxes.shape[0]if n != 1:raise ValueError('Currently only batch size 1 is supported.')_, _, H, W = imgs.shapeimg_size = (H, W)features = self.faster_rcnn.extractor(imgs)rpn_locs, rpn_scores, rois, roi_indices, anchor = \self.faster_rcnn.rpn(features, img_size, scale)# Since batch size is one, convert variables to singular formbbox = bboxes[0]label = labels[0]rpn_score = rpn_scores[0]rpn_loc = rpn_locs[0]roi = rois# Sample RoIs and forward# it's fine to break the computation graph of rois, # consider them as constant inputsample_roi, gt_roi_loc, gt_roi_label = self.proposal_target_creator(roi,at.tonumpy(bbox),at.tonumpy(label),self.loc_normalize_mean,self.loc_normalize_std)# NOTE it's all zero because now it only support for batch=1 nowsample_roi_index = t.zeros(len(sample_roi))roi_cls_loc, roi_score = self.faster_rcnn.head(features,sample_roi,sample_roi_index)# ------------------ RPN losses -------------------#gt_rpn_loc, gt_rpn_label = self.anchor_target_creator(at.tonumpy(bbox),anchor,img_size)gt_rpn_label = at.tovariable(gt_rpn_label).long()gt_rpn_loc = at.tovariable(gt_rpn_loc)rpn_loc_loss = _fast_rcnn_loc_loss(rpn_loc,gt_rpn_loc,gt_rpn_label.data,self.rpn_sigma)# NOTE: default value of ignore_index is -100 ...rpn_cls_loss = F.cross_entropy(rpn_score, gt_rpn_label.cuda(), ignore_index=-1)_gt_rpn_label = gt_rpn_label[gt_rpn_label > -1]_rpn_score = at.tonumpy(rpn_score)[at.tonumpy(gt_rpn_label) > -1]self.rpn_cm.add(at.totensor(_rpn_score, False), _gt_rpn_label.data.long())# ------------------ ROI losses (fast rcnn loss) -------------------#n_sample = roi_cls_loc.shape[0]roi_cls_loc = roi_cls_loc.view(n_sample, -1, 4)roi_loc = roi_cls_loc[t.arange(0, n_sample).long().cuda(), \at.totensor(gt_roi_label).long()]gt_roi_label = at.tovariable(gt_roi_label).long()gt_roi_loc = at.tovariable(gt_roi_loc)roi_loc_loss = _fast_rcnn_loc_loss(roi_loc.contiguous(),gt_roi_loc,gt_roi_label.data,self.roi_sigma)roi_cls_loss = nn.CrossEntropyLoss()(roi_score, gt_roi_label.cuda())self.roi_cm.add(at.totensor(roi_score, False), gt_roi_label.data.long())losses = [rpn_loc_loss, rpn_cls_loss, roi_loc_loss, roi_cls_loss]losses = losses + [sum(losses)]return LossTuple(*losses)def train_step(self, imgs, bboxes, labels, scale):self.optimizer.zero_grad()losses = self.forward(imgs, bboxes, labels, scale)losses.total_loss.backward()self.optimizer.step()self.update_meters(losses)return lossesdef save(self, save_optimizer=False, save_path=None, **kwargs):"""serialize models include optimizer and other inforeturn path where the model-file is stored.Args:save_optimizer (bool): whether save optimizer.state_dict().save_path (string): where to save model, if it's None, save_pathis generate using time str and info from kwargs.Returns:save_path(str): the path to save models."""save_dict = dict()save_dict['model'] = self.faster_rcnn.state_dict()save_dict['config'] = opt._state_dict()save_dict['other_info'] = kwargssave_dict['vis_info'] = self.vis.state_dict()if save_optimizer:save_dict['optimizer'] = self.optimizer.state_dict()if save_path is None:timestr = time.strftime('%m%d%H%M')save_path = 'checkpoints/fasterrcnn_%s' % timestrfor k_, v_ in kwargs.items():save_path += '_%s' % v_t.save(save_dict, save_path)self.vis.save([self.vis.env])return save_pathdef load(self, path, load_optimizer=True, parse_opt=False, ):state_dict = t.load(path)if 'model' in state_dict:self.faster_rcnn.load_state_dict(state_dict['model'])else: # legacy way, for backward compatibility self.faster_rcnn.load_state_dict(state_dict)return selfif parse_opt:opt._parse(state_dict['config'])if 'optimizer' in state_dict and load_optimizer:self.optimizer.load_state_dict(state_dict['optimizer'])return selfdef update_meters(self, losses):loss_d = {k: at.scalar(v) for k, v in losses._asdict().items()}for key, meter in self.meters.items():meter.add(loss_d[key])def reset_meters(self):for key, meter in self.meters.items():meter.reset()self.roi_cm.reset()self.rpn_cm.reset()def get_meter_data(self):return {k: v.value()[0] for k, v in self.meters.items()}def _smooth_l1_loss(x, t, in_weight, sigma):sigma2 = sigma ** 2diff = in_weight * (x - t)abs_diff = diff.abs()flag = (abs_diff.data < (1. / sigma2)).float()flag = Variable(flag)y = (flag * (sigma2 / 2.) * (diff ** 2) +(1 - flag) * (abs_diff - 0.5 / sigma2))return y.sum()def _fast_rcnn_loc_loss(pred_loc, gt_loc, gt_label, sigma):in_weight = t.zeros(gt_loc.shape).cuda()# Localization loss is calculated only for positive rois.# NOTE: unlike origin implementation, # we don't need inside_weight and outside_weight, they can calculate by gt_labelin_weight[(gt_label > 0).view(-1, 1).expand_as(in_weight).cuda()] = 1loc_loss = _smooth_l1_loss(pred_loc, gt_loc, Variable(in_weight), sigma)# Normalize by total number of negtive and positive rois.loc_loss /= (gt_label >= 0).sum() # ignore gt_label==-1 for rpn_lossreturn loc_loss View Code

此腳本定義了類FasterRCNNTrainer，在初始化中用到了之前定義的類FasterRCNNVGG16為faster_rcnn。 ?此外在初始化中有引入了其他creator、vis、optimizer等。

定義了四個損失函數和一個總損失函數：rpn_loc_loss、rpn_cls_loss、roi_loc_loss、roi_cls_loss以及total_loss。

前向傳播：

因為只支持batch_size=1的訓練，所以n=1。 每個batch輸入一張圖片、一張圖片上的所有bbox及label，以及圖像經過預處理后的尺度scale。

對于兩個分類損失調用cross_entropy即可?；貧w損失調用smooth_l1_loss。這里要注意的一點是例如roi回歸輸出的是128*84，然而真實位置參數是128*4和真實標簽128*1，利用這個真實標簽將回歸輸出索引成為128*4即可。然后在計算過程中只計算非背景類的回歸損失。具體實現與Fast-RCNN略有不同（sigma設置不同）。

此外定義了保存模型、可視化信息、具體配置、導入權重、配置信息等函數。

此外還從torchnet.meter 引入了 ConfusionMeter, AverageValueMeter。

?

2. trainer.py

import osimport ipdb import matplotlib from tqdm import tqdmfrom utils.config import opt from data.dataset import Dataset, TestDataset, inverse_normalize from model import FasterRCNNVGG16 from torch.autograd import Variable from torch.utils import data as data_ from trainer import FasterRCNNTrainer from utils import array_tool as at from utils.vis_tool import visdom_bbox from utils.eval_tool import eval_detection_voc# fix for ulimit # https://github.com/pytorch/pytorch/issues/973#issuecomment-346405667 import resourcerlimit = resource.getrlimit(resource.RLIMIT_NOFILE) resource.setrlimit(resource.RLIMIT_NOFILE, (20480, rlimit[1]))matplotlib.use('agg')def eval(dataloader, faster_rcnn, test_num=10000):pred_bboxes, pred_labels, pred_scores = list(), list(), list()gt_bboxes, gt_labels, gt_difficults = list(), list(), list()for ii, (imgs, sizes, gt_bboxes_, gt_labels_, gt_difficults_) in tqdm(enumerate(dataloader)):sizes = [sizes[0][0], sizes[1][0]]pred_bboxes_, pred_labels_, pred_scores_ = faster_rcnn.predict(imgs, [sizes])gt_bboxes += list(gt_bboxes_.numpy())gt_labels += list(gt_labels_.numpy())gt_difficults += list(gt_difficults_.numpy())pred_bboxes += pred_bboxes_pred_labels += pred_labels_pred_scores += pred_scores_if ii == test_num: breakresult = eval_detection_voc(pred_bboxes, pred_labels, pred_scores,gt_bboxes, gt_labels, gt_difficults,use_07_metric=True)return resultdef train(**kwargs):opt._parse(kwargs)dataset = Dataset(opt)print('load data')dataloader = data_.DataLoader(dataset, \batch_size=1, \shuffle=True, \# pin_memory=True,num_workers=opt.num_workers)testset = TestDataset(opt)test_dataloader = data_.DataLoader(testset,batch_size=1,num_workers=opt.test_num_workers,shuffle=False, \pin_memory=True)faster_rcnn = FasterRCNNVGG16()print('model construct completed')trainer = FasterRCNNTrainer(faster_rcnn).cuda()if opt.load_path:trainer.load(opt.load_path)print('load pretrained model from %s' % opt.load_path)trainer.vis.text(dataset.db.label_names, win='labels')best_map = 0lr_ = opt.lrfor epoch in range(opt.epoch):trainer.reset_meters()for ii, (img, bbox_, label_, scale) in tqdm(enumerate(dataloader)):scale = at.scalar(scale)img, bbox, label = img.cuda().float(), bbox_.cuda(), label_.cuda()img, bbox, label = Variable(img), Variable(bbox), Variable(label)trainer.train_step(img, bbox, label, scale)if (ii + 1) % opt.plot_every == 0:if os.path.exists(opt.debug_file):ipdb.set_trace()# plot loss trainer.vis.plot_many(trainer.get_meter_data())# plot groud truth bboxesori_img_ = inverse_normalize(at.tonumpy(img[0]))gt_img = visdom_bbox(ori_img_,at.tonumpy(bbox_[0]),at.tonumpy(label_[0]))trainer.vis.img('gt_img', gt_img)# plot predicti bboxes_bboxes, _labels, _scores = trainer.faster_rcnn.predict([ori_img_], visualize=True)pred_img = visdom_bbox(ori_img_,at.tonumpy(_bboxes[0]),at.tonumpy(_labels[0]).reshape(-1),at.tonumpy(_scores[0]))trainer.vis.img('pred_img', pred_img)# rpn confusion matrix(meter)trainer.vis.text(str(trainer.rpn_cm.value().tolist()), win='rpn_cm')# roi confusion matrixtrainer.vis.img('roi_cm', at.totensor(trainer.roi_cm.conf, False).float())eval_result = eval(test_dataloader, faster_rcnn, test_num=opt.test_num)if eval_result['map'] > best_map:best_map = eval_result['map']best_path = trainer.save(best_map=best_map)if epoch == 9:trainer.load(best_path)trainer.faster_rcnn.scale_lr(opt.lr_decay)lr_ = lr_ * opt.lr_decaytrainer.vis.plot('test_map', eval_result['map'])log_info = 'lr:{}, map:{},loss:{}'.format(str(lr_),str(eval_result['map']),str(trainer.get_meter_data()))trainer.vis.log(log_info)if epoch == 13: breakif __name__ == '__main__':import firefire.Fire() View Code

訓練Faster-RCNN。

總共迭代14個epoch，第9個epoch時學習率衰減0.1倍。每100個batch在visdom中更新損失變化曲線及顯示訓練與測試圖像。

?

二. 補充內容

1. RPN網絡

RPN作用是通過網絡訓練的方式從feature map中獲取目標的大致位置。RPN做兩件事：1、把feature map分割成多個小區域，識別出哪些小區域是前景，哪些是背景，簡稱RPN Classification；2、獲取前景區域的大致坐標，簡稱RPN bounding box regression。RPN可以獨立使用，而不需要第二階段的模型。在只有一類對象的問題中，目標性概率可以用作最終的類別概率。這是因為在這種情況下，「前景」=「目標類別」以及「背景」=「不是目標類別」。一些從獨立使用 RPN 中受益的機器學習問題的例子包括流行的（但仍然是具有挑戰性的）人臉檢測和文本檢測。僅使用 RPN 的優點之一是訓練和預測的速度都有所提高。由于 RPN 是一個非常簡單的僅使用卷積層的網絡，所以預測時間比使用分類基礎網絡更快。

?

2. 回歸

兩次位置參數回歸中的h，w都采用的是取對數操作，用對數來表示長寬的差別，是為了在差別大時能快速收斂，差別小時能較慢收斂來保證精度。

?

3 . chainer框架與pytorch等其他框架的比較：

chainer利用python重造了所有輪子（所有layer、正反向傳播等），而pytorch借用現有C語言的輪子（TH、THNN）。

下圖是chainer實現Faster-RCNN的流程圖：

?

?

Reference:

從編程實現角度學習Faster R-CNN（附極簡實現）

深度 | 像玩樂高一樣拆解Faster R-CNN：詳解目標檢測的實現過程

從結構、原理到實現，Faster R-CNN全解析

一文讀懂Faster R-CNN

?chainer簡介?、 chainer官網（包含 chainer MN、chainer RL、chainer CV）

目標檢測模型的評估指標mAP詳解(附代碼）

轉載于:https://www.cnblogs.com/king-lps/p/8995412.html

總結

以上是生活随笔為你收集整理的Faster_RCNN 4.训练模型的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。