當前位置：首頁 > 人文社科 > 生活经验 >内容正文

生活经验

优达学城《DeepLearning》项目2：犬种分类器

發布時間：2023/11/27 生活经验 36 豆豆

生活随笔收集整理的這篇文章主要介紹了优达学城《DeepLearning》项目2：犬种分类器小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

在這個notebook中，你將邁出第一步，來開發可以作為移動端或 Web應用程序一部分的算法。在這個項目的最后，你的程序將能夠把用戶提供的任何一個圖像作為輸入。如果可以從圖像中檢測到一只狗，它會輸出對狗品種的預測。如果圖像中是一個人臉，它會預測一個與其最相似的狗的種類。下面這張圖展示了完成項目后可能的輸出結果。

Step 0: 導入數據集
Step 1: 檢測人臉
Step 2: 檢測狗狗
Step 3: 從頭創建一個CNN來分類狗品種
Step 4: 使用一個CNN來區分狗的品種(使用遷移學習)
Step 5: 完成你的算法
Step 6: 測試你的算法

步驟 0: 導入數據集

下載狗數據集：dog dataset，放置在data文件夾內并解壓。

下載人數據集：human dataset（一個人臉識別數據集），放置在data文件夾內并解壓。

步驟1：檢測人臉

我們將使用 OpenCV 中的?Haar feature-based cascade classifiers?來檢測圖像中的人臉。OpenCV 提供了很多預訓練的人臉檢測模型，它們以XML文件保存在?github。我們已經下載了其中一個檢測模型，并且把它存儲在?haarcascades?的目錄中。

在如下代碼單元中，我們將演示如何使用這個檢測模型在樣本圖像中找到人臉。

寫一個人臉探測器

我們可以使用這個程序編寫一個函數，如果在圖像中檢測到人臉，則返回True，否則返回False。這個函數名為face_detector，將圖像的文件路徑作為輸入，出現在下面的代碼塊中。

評估人臉檢測器

問題1:使用下面的代碼單元格來測試face_detector函數的性能。

在human_files的前100張圖片中有多大概率檢測到人臉?
在dog_files的前100張圖片中中有多大概率檢測到人臉?

步驟 2: 檢測狗狗

在這個部分中，我們使用預訓練的 vgg16 模型去檢測圖像中的狗。

給定一個圖像，這個預先訓練過的vgg16模型將為圖像中包含的對象返回一個預測(來自ImageNet中的1000個可能類別)。

用預先訓練好的模型進行預測

在下一個代碼單元格中，您將編寫一個函數，它接受圖像的路徑作為輸入，并返回由預先訓練的vgg -16模型預測的ImageNet類對應的索引。輸出應該始終是0到999之間的整數(含999)。

from PIL import Image
import torchvision.transforms as transforms# Set PIL to be tolerant of image files that are truncated.
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = Truedef VGG16_predict(img_path):'''Use pre-trained VGG-16 model to obtain index corresponding to predicted ImageNet class for image at specified pathArgs:img_path: path to an imageReturns:Index corresponding to VGG-16 model's prediction'''## TODO: Complete the function.## Load and pre-process an image from the given img_path## Return the *index* of the predicted class for that imagenormalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225])transform = transforms.Compose([transforms.Resize((224,224)),transforms.CenterCrop(224),transforms.ToTensor(),normalize])image = Image.open(img_path)#print(image.size)image = transform(image)image.unsqueeze_(0)#print(image.size)if use_cuda:image = image.cuda()output = VGG16(image)if use_cuda:output = output.cuda()class_index = output.data.cpu().numpy().argmax()    return class_index # predicted class index#print(dog_files[0])
VGG16_predict(dog_files[0])

結果：

寫一個狗狗探測器

在研究該?清單?的時候，你會注意到，狗類別對應的序號為151-268。因此，在檢查預訓練模型判斷圖像是否包含狗的時候，我們只需要檢查如上的?VGG16_predict 函數是否返回一個介于151和268之間（包含區間端點）的值。

我們通過這些想法來完成下方的?dog_detector?函數，如果從圖像中檢測到狗就返回?True，否則返回?False。

問題2:使用下面的代碼單元格來測試dog_detector函數的性能。

在human_files_short中有多少百分比檢測到狗?
在dog_files_short中有多少百分比檢測到狗?

你可以自由地探索其他預先訓練網絡(如Inception-v3, ResNet-50等)。請使用下面的代碼單元來測試其他預先訓練好的PyTorch模型。如果您決定執行這個可選任務，請報告human_files_short和dog_files_short的性能。

步驟 3: 從頭開始創建一個CNN來分類狗品種

現在我們已經實現了一個函數，能夠在圖像中識別人類及狗狗。但我們需要更進一步的方法，來對狗的類別進行識別。在這一步中，你需要實現一個卷積神經網絡來對狗的品種進行分類。你需要從頭實現你的卷積神經網絡（在這一階段，你還不能使用遷移學習），并且你需要達到超過1%的測試集準確率。在本項目的步驟五種，你還有機會使用遷移學習來實現一個準確率大大提高的模型。

值得注意的是，對狗的圖像進行分類是一項極具挑戰性的任務。因為即便是一個正常人，也很難區分布列塔尼犬和威爾士史賓格犬。

不難發現其他的狗品種會有很小的類間差別（比如金毛尋回犬和美國水獵犬）。

同樣，拉布拉多犬（labradors）有黃色、棕色和黑色這三種。那么你設計的基于視覺的算法將不得不克服這種較高的類間差別，以達到能夠將這些不同顏色的同類狗分到同一個品種中。

我們也提到了隨機分類將得到一個非常低的結果：不考慮品種略有失衡的影響，隨機猜測到正確品種的概率是1/133，相對應的準確率是低于1%的。

請記住，在深度學習領域，實踐遠遠高于理論。大量嘗試不同的框架吧，相信你的直覺！當然，玩得開心！

為Dog數據集指定數據加載器

問題3:描述您選擇的數據預處理過程。

你的代碼如何調整圖像的大小(裁剪，拉伸等)?輸入張量的大小是多少，為什么?
你決定擴大數據集了嗎?如果有，如何(通過平移，翻轉，旋轉等)?如果沒有，為什么?

指定損失函數和優化器

訓練和驗證模型

# the following import is required for training to be robust to truncated images
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = Truedef train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):"""returns trained model"""# initialize tracker for minimum validation lossvalid_loss_min = np.Inf for epoch in range(1, n_epochs+1):# initialize variables to monitor training and validation losstrain_loss = 0.0valid_loss = 0.0#################### train the model ####################model.train()for batch_idx, (data, target) in enumerate(loaders['train']):# move to GPUif use_cuda:data, target = data.cuda(), target.cuda()## find the loss and update the model parameters accordingly## record the average training loss, using something like## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))optimizer.zero_grad()    ## find the loss and update the model parameters accordinglyoutput = model(data)loss = criterion(output, target)loss.backward()optimizer.step()## record the average training loss, using something like## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))train_loss += ((1 / (batch_idx + 1)) * (loss.data - train_loss))# print training/validation statistics if (batch_idx+1) % 40 == 0:print('Epoch: {} \tBatch: {} \tTraining Loss: {:.6f}'.format(epoch, batch_idx + 1, train_loss))######################    # validate the model #######################model.eval()for batch_idx, (data, target) in enumerate(loaders['valid']):# move to GPUif use_cuda:data, target = data.cuda(), target.cuda()## update the average validation lossoutput = model(data)loss = criterion(output, target)valid_loss += ((1 / (batch_idx + 1)) * (loss.data - valid_loss))# print training/validation statistics if (batch_idx+1) % 40 == 0:print('Epoch: {} \tBatch: {} \tValidation Loss: {:.6f}'.format(epoch, batch_idx + 1, valid_loss))# print training/validation statistics print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(epoch, train_loss,valid_loss))## TODO: save the model if validation loss has decreasedif valid_loss <= valid_loss_min:print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(valid_loss_min,valid_loss))torch.save(model.state_dict(), save_path)valid_loss_min = valid_loss    # return trained modelreturn modelepoch = 10# train the model
model_scratch = train(epoch, loaders_scratch, model_scratch, optimizer_scratch, criterion_scratch, use_cuda, 'model_scratch.pt')# load the model that got the best validation accuracy
model_scratch.load_state_dict(torch.load('model_scratch.pt'))

結果：

Epoch: 1 	Batch: 40 	Training Loss: 4.889728
Epoch: 1 	Batch: 80 	Training Loss: 4.887787
Epoch: 1 	Batch: 120 	Training Loss: 4.887685
Epoch: 1 	Batch: 160 	Training Loss: 4.885648
Epoch: 1 	Batch: 200 	Training Loss: 4.881847
Epoch: 1 	Batch: 240 	Training Loss: 4.873278
Epoch: 1 	Batch: 280 	Training Loss: 4.862270
Epoch: 1 	Batch: 320 	Training Loss: 4.854340
Epoch: 1 	Batch: 40 	Validation Loss: 4.712207
Epoch: 1 	Training Loss: 4.851301 	Validation Loss: 4.704982
Validation loss decreased (inf --> 4.704982).  Saving model ...
Epoch: 2 	Batch: 40 	Training Loss: 4.730308
Epoch: 2 	Batch: 80 	Training Loss: 4.719476
Epoch: 2 	Batch: 120 	Training Loss: 4.701708
Epoch: 2 	Batch: 160 	Training Loss: 4.695746
Epoch: 2 	Batch: 200 	Training Loss: 4.692133
Epoch: 2 	Batch: 240 	Training Loss: 4.675904
Epoch: 2 	Batch: 280 	Training Loss: 4.663143
Epoch: 2 	Batch: 320 	Training Loss: 4.650386
Epoch: 2 	Batch: 40 	Validation Loss: 4.488307
Epoch: 2 	Training Loss: 4.643542 	Validation Loss: 4.494160
Validation loss decreased (4.704982 --> 4.494160).  Saving model ...
Epoch: 3 	Batch: 40 	Training Loss: 4.474283
Epoch: 3 	Batch: 80 	Training Loss: 4.501595
Epoch: 3 	Batch: 120 	Training Loss: 4.477735
Epoch: 3 	Batch: 160 	Training Loss: 4.488136
Epoch: 3 	Batch: 200 	Training Loss: 4.490754
Epoch: 3 	Batch: 240 	Training Loss: 4.487989
Epoch: 3 	Batch: 280 	Training Loss: 4.490090
Epoch: 3 	Batch: 320 	Training Loss: 4.481546
Epoch: 3 	Batch: 40 	Validation Loss: 4.262285
Epoch: 3 	Training Loss: 4.479444 	Validation Loss: 4.275317
Validation loss decreased (4.494160 --> 4.275317).  Saving model ...
Epoch: 4 	Batch: 40 	Training Loss: 4.402790
Epoch: 4 	Batch: 80 	Training Loss: 4.372338
Epoch: 4 	Batch: 120 	Training Loss: 4.365306
Epoch: 4 	Batch: 160 	Training Loss: 4.367325
Epoch: 4 	Batch: 200 	Training Loss: 4.374326
Epoch: 4 	Batch: 240 	Training Loss: 4.369847
Epoch: 4 	Batch: 280 	Training Loss: 4.365964
Epoch: 4 	Batch: 320 	Training Loss: 4.363493
Epoch: 4 	Batch: 40 	Validation Loss: 4.249445
Epoch: 4 	Training Loss: 4.364571 	Validation Loss: 4.248449
Validation loss decreased (4.275317 --> 4.248449).  Saving model ...
Epoch: 5 	Batch: 40 	Training Loss: 4.229365
Epoch: 5 	Batch: 80 	Training Loss: 4.267400
Epoch: 5 	Batch: 120 	Training Loss: 4.269664
Epoch: 5 	Batch: 160 	Training Loss: 4.257591
Epoch: 5 	Batch: 200 	Training Loss: 4.261866
Epoch: 5 	Batch: 240 	Training Loss: 4.247512
Epoch: 5 	Batch: 280 	Training Loss: 4.239336
Epoch: 5 	Batch: 320 	Training Loss: 4.230827
Epoch: 5 	Batch: 40 	Validation Loss: 4.043582
Epoch: 5 	Training Loss: 4.231559 	Validation Loss: 4.039588
Validation loss decreased (4.248449 --> 4.039588).  Saving model ...
Epoch: 6 	Batch: 40 	Training Loss: 4.180193
Epoch: 6 	Batch: 80 	Training Loss: 4.140314
Epoch: 6 	Batch: 120 	Training Loss: 4.153989
Epoch: 6 	Batch: 160 	Training Loss: 4.140887
Epoch: 6 	Batch: 200 	Training Loss: 4.151268
Epoch: 6 	Batch: 240 	Training Loss: 4.153749
Epoch: 6 	Batch: 280 	Training Loss: 4.153314
Epoch: 6 	Batch: 320 	Training Loss: 4.156451
Epoch: 6 	Batch: 40 	Validation Loss: 3.940857
Epoch: 6 	Training Loss: 4.149529 	Validation Loss: 3.945810
Validation loss decreased (4.039588 --> 3.945810).  Saving model ...
Epoch: 7 	Batch: 40 	Training Loss: 4.060485
Epoch: 7 	Batch: 80 	Training Loss: 4.065772
Epoch: 7 	Batch: 120 	Training Loss: 4.056967
Epoch: 7 	Batch: 160 	Training Loss: 4.068470
Epoch: 7 	Batch: 200 	Training Loss: 4.076772
Epoch: 7 	Batch: 240 	Training Loss: 4.087616
Epoch: 7 	Batch: 280 	Training Loss: 4.074337
Epoch: 7 	Batch: 320 	Training Loss: 4.080192
Epoch: 7 	Batch: 40 	Validation Loss: 3.860693
Epoch: 7 	Training Loss: 4.078263 	Validation Loss: 3.884382
Validation loss decreased (3.945810 --> 3.884382).  Saving model ...
Epoch: 8 	Batch: 40 	Training Loss: 3.960585
Epoch: 8 	Batch: 80 	Training Loss: 3.983979
Epoch: 8 	Batch: 120 	Training Loss: 3.965129
Epoch: 8 	Batch: 160 	Training Loss: 3.965021
Epoch: 8 	Batch: 200 	Training Loss: 3.965830
Epoch: 8 	Batch: 240 	Training Loss: 3.976013
Epoch: 8 	Batch: 280 	Training Loss: 3.975547
Epoch: 8 	Batch: 320 	Training Loss: 3.978744
Epoch: 8 	Batch: 40 	Validation Loss: 3.784086
Epoch: 8 	Training Loss: 3.980776 	Validation Loss: 3.779312
Validation loss decreased (3.884382 --> 3.779312).  Saving model ...
Epoch: 9 	Batch: 40 	Training Loss: 3.917738
Epoch: 9 	Batch: 80 	Training Loss: 3.967938
Epoch: 9 	Batch: 120 	Training Loss: 3.934165
Epoch: 9 	Batch: 160 	Training Loss: 3.917138
Epoch: 9 	Batch: 200 	Training Loss: 3.910391
Epoch: 9 	Batch: 240 	Training Loss: 3.909857
Epoch: 9 	Batch: 280 	Training Loss: 3.907439
Epoch: 9 	Batch: 320 	Training Loss: 3.901893
Epoch: 9 	Batch: 40 	Validation Loss: 3.826410
Epoch: 9 	Training Loss: 3.903304 	Validation Loss: 3.824970
Epoch: 10 	Batch: 40 	Training Loss: 3.910845
Epoch: 10 	Batch: 80 	Training Loss: 3.910709
Epoch: 10 	Batch: 120 	Training Loss: 3.915924
Epoch: 10 	Batch: 160 	Training Loss: 3.900949
Epoch: 10 	Batch: 200 	Training Loss: 3.883792
Epoch: 10 	Batch: 240 	Training Loss: 3.882242
Epoch: 10 	Batch: 280 	Training Loss: 3.875963
Epoch: 10 	Batch: 320 	Training Loss: 3.858594
Epoch: 10 	Batch: 40 	Validation Loss: 3.638186
Epoch: 10 	Training Loss: 3.861175 	Validation Loss: 3.647465
Validation loss decreased (3.779312 --> 3.647465).  Saving model ...
<All keys matched successfully>

測試模型

在狗圖片的測試數據集中試用你的模型。使用下面的代碼單元計算和打印測試loss和準確性。確保您的測試精度大于10%。

步驟 4: 使用CNN遷移學習來區分狗的品種

使用遷移學習（Transfer Learning）的方法，能幫助我們在不損失準確率的情況下大大減少訓練時間。在以下步驟中，你可以嘗試使用遷移學習來訓練你自己的CNN。

模型架構

指定損失函數和優化器

訓練和驗證模型

# train the model
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = Truedef train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):"""returns trained model"""# initialize tracker for minimum validation lossvalid_loss_min = np.Inf for epoch in range(1, n_epochs+1):# initialize variables to monitor training and validation losstrain_loss = 0.0valid_loss = 0.0#################### train the model ####################model.train()for batch_idx, (data, target) in enumerate(loaders['train']):# move to GPUif use_cuda:data, target = data.cuda(), target.cuda()optimizer.zero_grad()    ## find the loss and update the model parameters accordinglyoutput = model(data)loss = criterion(output, target)loss.backward()optimizer.step()## record the average training loss, using something like## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))train_loss += ((1 / (batch_idx + 1)) * (loss.data - train_loss))# print training/validation statistics if (batch_idx+1) % 40 == 0:print('Epoch: {} \tBatch: {} \tTraining Loss: {:.6f}'.format(epoch, batch_idx + 1, train_loss))######################    # validate the model #######################model.eval()for batch_idx, (data, target) in enumerate(loaders['valid']):# move to GPUif use_cuda:data, target = data.cuda(), target.cuda()## update the average validation lossoutput = model(data)loss = criterion(output, target)valid_loss += ((1 / (batch_idx + 1)) * (loss.data - valid_loss))# print training/validation statistics if (batch_idx+1) % 40 == 0:print('Epoch: {} \tBatch: {} \tValidation Loss: {:.6f}'.format(epoch, batch_idx + 1, valid_loss))# print training/validation statistics print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(epoch, train_loss,valid_loss))## TODO: save the model if validation loss has decreasedif valid_loss <= valid_loss_min:print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(valid_loss_min,valid_loss))torch.save(model.state_dict(), save_path)valid_loss_min = valid_loss    # return trained modelreturn modeln_epochs = 2
model_transfer = train(n_epochs, loaders_transfer, model_transfer, optimizer_transfer, criterion_transfer, use_cuda, 'model_transfer.pt')# load the model that got the best validation accuracy (uncomment the line below)
model_transfer.load_state_dict(torch.load('model_transfer.pt'))

結果：

測試模型

用模型預測狗的品種

編寫一個函數，以圖像路徑作為輸入，并返回您的模型預測的狗的品種

步驟 5: 完成你的算法

實現一個算法，它的輸入為圖像的路徑，它能夠區分圖像是否包含一個人、狗或兩者都不包含，然后：

如果從圖像中檢測到一只狗，返回被預測的品種。
如果從圖像中檢測到人，返回最相像的狗品種。
如果兩者都不能在圖像中檢測到，輸出錯誤提示。

我們非常歡迎你來自己編寫檢測圖像中人類與狗的函數，你可以隨意地使用上方完成的?face_detector?和?dog_detector?函數。你需要在步驟5使用你的CNN來預測狗品種。

下面提供了算法的示例輸出，但你可以自由地設計自己的模型！

步驟 6: 測試你的算法

在這個部分中，你將嘗試一下你的新算法！算法認為你看起來像什么類型的狗？如果你有一只狗，它可以準確地預測你的狗的品種嗎？如果你有一只貓，它會將你的貓誤判為一只狗嗎？

結果：

This is a human picture who looks like Norwich terrier

This is a human picture who looks like Curly-coated retriever

This is a dog picture and it's English cocker spaniel

This is a dog picture and it's Bloodhound

This is a dog picture and it's Lowchen

總結

以上是生活随笔為你收集整理的优达学城《DeepLearning》项目2：犬种分类器的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： AI视频行为分析系统项目复盘——技术篇2
下一篇： AI视频行为分析系统项目复盘——技术篇3