數(shù)據(jù)庫課程設(shè)計(jì)結(jié)論

Living in the Pacific Northwest, the intertwined issues of salmon survival and river flow are frequently in the news, and the data we have on our salmon populations is a key piece in the conversation.

生活在西北太平洋地區(qū)，鮭魚生存和河流流量這兩個(gè)相互交織的問題經(jīng)常成為新聞，而我們關(guān)于鮭魚種群的數(shù)據(jù)是談話中的關(guān)鍵部分。

I wanted to see if deep learning — object detection in particular — was up to the challenge of assisting with the real world problem of counting fish seen passing through a fish ladder.

我想看看深度學(xué)習(xí)(尤其是對(duì)象檢測(cè))是否可以應(yīng)對(duì)現(xiàn)實(shí)世界中對(duì)通過魚梯的魚進(jìn)行計(jì)數(shù)的難題。

The answer is yes, and there are enough free or open source tools available to accomplish all aspects of the task.

答案是肯定的，并且有足夠的免費(fèi)或開源工具可完成任務(wù)的所有方面。

Object Detection for Fish Counting魚類計(jì)數(shù)的目標(biāo)檢測(cè)

This approach can be used to count other objects as well. If you have an idea and a place to grab a few images, you can use this walkthrough to help you get your own model running.

該方法也可以用于計(jì)算其他對(duì)象。 如果您有想法并可以獲取一些圖像，則可以使用本演練來幫助您運(yùn)行自己的模型。

步驟1：一個(gè)有趣的問題 (Step 1: An intriguing issue)

The Pacific Northwest is home to many species of salmon whose lives follow a predictable pattern: hatch in fresh water, migrate to the ocean for the majority of their lives, and then migrate back upstream to their original fresh water hatch sites before they spawn and then die.

西北太平洋地區(qū)生活著許多種類的鮭魚，它們的生活遵循可預(yù)測(cè)的模式：在淡水中孵化，在大部分生命中移至海洋，然后在產(chǎn)卵前向上游遷移回到其原始淡水孵化場(chǎng)，死。

Salmon aren’t the only ones using the rivers though. Hydroelectric dams dot our rivers for hundreds of miles, potentially preventing the upstream migration. To solve this, our dams are all built with something called a ‘fish ladder.’ These constructions allow fish to jump upstream in short segments much as they would in an openly flowing river.

鮭魚并不是唯一使用河流的魚類。 水力發(fā)電大壩在我們的河流上遍布數(shù)百英里，有可能阻止上游遷移。 為了解決這個(gè)問題，我們的水壩全部建有所謂的“魚梯”。 這些結(jié)構(gòu)使魚類可以像在一條開放的河流中那樣，在短段內(nèi)向上游跳躍。

Hydroelectric dams provide fish ladders (foreground)水電大壩提供魚梯(前景)

Salmon ladders with viewing windows also provide us with the means to witness this migration and to collect data. Data on the number of salmon returning upstream can then be used to determine the length of the fishing season, set limits on fish per day, adjust the flow at the dam, and help us understand how well we’re doing in this balancing act.

帶有觀察窗的鮭魚梯也為我們提供了見證遷移和收集數(shù)據(jù)的方法。 然后，可以將有關(guān)返回上游的鮭魚數(shù)量的數(shù)據(jù)用于確定捕魚季節(jié)的長度，設(shè)置每天的魚類限制，調(diào)整水壩的流量，并幫助我們了解我們?cè)谶@種平衡行動(dòng)中的狀況。

Counting fish is currently performed by trained experts either as real time counts or by reviewing video recordings. Given how labor intensive this is, an assist from an expert tool could provide a nice boost to increase the amount of data taken, the speed at which it is available, or the number of locations where the data is recorded.

目前，計(jì)數(shù)魚類是由訓(xùn)練有素的專家進(jìn)行的，既可以實(shí)時(shí)計(jì)數(shù)，也可以通過查看錄像來進(jìn)行。 考慮到這是多么勞動(dòng)密集的工作，專家工具的幫助可以極大地提高所獲取的數(shù)據(jù)量，可用數(shù)據(jù)的速度或記錄數(shù)據(jù)的位置數(shù)量。

步驟2：圖像和預(yù)處理 (Step 2: Images and pre-processing)

Whichever project you choose, you’ll need to start with data. Don’t be dissuaded if you can’t start with ideal data; there’s still plenty to learn. I started with web scraped images of vacation photos taken at fish ladders. The images contained a variety of locations, lighting conditions, silhouettes of children, and possible post-production modifications. The images were so difficult to work with that all I could really label was what constituted a “fish”! I ran the rest of this flow with just that and worked out quite a few bugs, so remember that you can start simple. Once I had a sense of what I needed, I found my way to some better images recorded on a video.

無論選擇哪個(gè)項(xiàng)目，都需要從數(shù)據(jù)開始。 如果您無法從理想的數(shù)據(jù)入手，請(qǐng)不要被勸阻； 還有很多東西要學(xué)。 我從網(wǎng)上抓取了在魚梯上拍攝的度假照片的圖像開始。 圖像包含各種位置，光照條件，兒童剪影以及可能的后期制作修改。 這些圖像很難處理，以至于我只能標(biāo)記出構(gòu)成“魚”的東西！ 我僅以此來運(yùn)行其余的流程，并解決了許多錯(cuò)誤，因此請(qǐng)記住，您可以從簡單開始。 一旦感覺到需要的東西，我便找到了錄制視頻中更好圖像的方法。

For the images in video format, I used the free VLC Media Player tool to run the video and extract 300+ frames. I selected 317 images including 15 images with no fish at all.

對(duì)于視頻格式的圖像，我使用了免費(fèi)的VLC Media Player工具來運(yùn)行視頻并提取300多個(gè)幀。 我選擇了317張圖像，其中包括15張完全沒有魚的圖像。

To prepare these images for deep learning, I also needed to associate a correctly labeled box with each object to be detected. Trained experts label fish by species, mature vs. juvenile, and hatchery vs. wild.

為了準(zhǔn)備這些圖像進(jìn)行深度學(xué)習(xí)，我還需要將正確標(biāo)記的框與要檢測(cè)的每個(gè)對(duì)象相關(guān)聯(lián)。 受過訓(xùn)練的專家按魚種，成熟對(duì)幼魚以及孵化場(chǎng)對(duì)野生魚進(jìn)行標(biāo)記。

Salmon Identification鮭魚識(shí)別

Without fish identification expertise, it was still possible to create 3 classes:

沒有魚類鑒定專業(yè)知識(shí)，仍然有可能創(chuàng)建3個(gè)類別：

• ‘a(chǎn)dipose’ for fish having an intact and visible adipose fin
  “脂肪”用于具有完整可見脂肪鰭的魚
• ‘no_adipose’ for fish having no adipose fin
  沒有脂肪的魚的“ no_adipose”
• ‘unknown’ for fish only partially in the viewing window, or whose adipose fin region is obscured by another fish or artifact
  魚只在部分觀察窗中或魚類的脂肪鰭區(qū)域被另一條魚或人工制品遮蓋，因此“未知”

Adipose Fin Identification脂肪鰭識(shí)別

The free tool labelImg worked well and I had my 317 fish images boxed and labeled in an afternoon.

免費(fèi)工具labelImg運(yùn)作良好，一個(gè)下午裝箱并標(biāo)記了我的317條魚圖像。

One of the reasons I knew thatI could get a model working with only 300 or so images was the idea of augmentation. Augmentation takes the originals and constructs variations so that the model is exposed to a variety of changes such as lighting and orientation. With classification, this is easily performed with a few python libraries, but for object detection, the label boxes also need to be transformed if the image is flipped or rotated.

我知道可以得到一個(gè)僅處理300張左右圖像的模型的原因之一就是增強(qiáng)的想法。 增強(qiáng)采用原始圖像并構(gòu)造變體，以便模型可以暴露于各種變化(例如光照和方向)。 使用分類，可以使用幾個(gè)python庫輕松地執(zhí)行此操作，但是對(duì)于對(duì)象檢測(cè)，如果翻轉(zhuǎn)或旋轉(zhuǎn)圖像，則還需要轉(zhuǎn)換標(biāo)簽框。

Rather than do these transforms by hand, I leveraged the free tools at Roboflow. I uploaded my images and my label file, selected options to create additional images with random amounts of blur, changes in brightness, and horizontal flip. After this step, I had 951 training images.

我沒有手動(dòng)進(jìn)行這些轉(zhuǎn)換，而是利用了Roboflow上的免費(fèi)工具。 我上傳了圖像和標(biāo)簽文件，并選擇了一些選項(xiàng)來創(chuàng)建具有隨機(jī)模糊量，亮度變化和水平翻轉(zhuǎn)的其他圖像。 經(jīng)過這一步，我得到了951張訓(xùn)練圖像。

步驟3：選擇并訓(xùn)練模型 (Step 3: Select and train a model)

While it’s possible to build a home-grown object detection model, I opted to start with a known model for my project as a baseline before doing any tailoring. All of this can be done from the model library at Roboflow, and it’s possible to try out more than one.

盡管可以構(gòu)建自家的對(duì)象檢測(cè)模型，但在進(jìn)行任何剪裁之前，我選擇以項(xiàng)目的已知模型作為基準(zhǔn)。 所有這些都可以從Roboflow的模型庫中完成 ，并且可以嘗試多個(gè)。

“You Only Look Once”. YOLO is a popular object detection machine learning model introduced in 2015 by a group of researchers at the University of Washington. Rather than pass an image classifier multiple times over an image to see if there was, say, a dog at the upper left, or maybe at the upper right, this new approach replaced the final layers of an image classifier with additional convolutional layers that allowed it to find all instances in one pass. The immediate improvement in speed was a major leap forward for computer vision and object detection. Since the original paper, the model has been improved several times and a new model built on this earlier foundation was released in June 2020 as YOLO v5. See the repository at https://github.com/ultralytics/yolov5 for more details on the model.

“你只看一次”。 YOLO是由華盛頓大學(xué)的一組研究人員于2015年引入的一種流行的對(duì)象檢測(cè)機(jī)器學(xué)習(xí)模型。 這種新方法不是在圖像上多次傳遞圖像分類器以查看是否在左上方或右上方有一條狗，而是用附加的卷積層替換了圖像分類器的最后一層，從而允許一次找到所有實(shí)例。 速度的立即提高是計(jì)算機(jī)視覺和物體檢測(cè)的重大飛躍。 自原始論文以來，該模型已進(jìn)行了多次改進(jìn)，并且在此較早基礎(chǔ)上構(gòu)建的新模型于2020年6月作為YOLO v5發(fā)布。 有關(guān)該模型的更多詳細(xì)信息，請(qǐng)參見https://github.com/ultralytics/yolov5上的存儲(chǔ)庫。

Given the popularity, speed, and accuracy of YOLO, and the ease of leveraging the tools at Roboflow, trying out the YOLO v5 model was an obvious choice. Starting with a Google Colaboratory template that configured the environment and built the model, I customized this by uploading a new training set experimenting with various epochs and thresholds.

考慮到Y(jié)OLO的受歡迎程度，速度和準(zhǔn)確性，以及在Roboflow上利用工具的便利性 ，嘗試YOLO v5模型是一個(gè)顯而易見的選擇。 從配置環(huán)境并構(gòu)建模型的Google Colaboratory模板開始，我通過上傳新的訓(xùn)練集來進(jìn)行自定義，該訓(xùn)練集嘗試了各種時(shí)期和閾值。

步驟4：探索模型結(jié)果 (Step 4: Explore model results)

The results were impressive and informative.

結(jié)果令人印象深刻，內(nèi)容豐富。

Impressive — the YOLO v5 model trained for 500 epochs in about an hour on the 900+ images, ran inference (prediction) on a new image in about 12 msec, and achieved mAP@0.5 of about 70%.

令人印象深刻 – YOLO v5模型在900幅以上的圖像上訓(xùn)練了大約一個(gè)小時(shí)的500個(gè)歷元，在大約12毫秒內(nèi)對(duì)新圖像進(jìn)行了推斷(預(yù)測(cè))，并達(dá)到了70％的mAP@0.5。

Model correctly labels fish in simple images模型可以在簡單圖像中正確標(biāo)記魚

Individually viewing and rating the model’s success on test images would be a bit of a chore, though, so this is where the mAP@0.5 metric is valuable.

但是，單獨(dú)查看模型并在測(cè)試圖像上對(duì)模型的成功進(jìn)行評(píng)分會(huì)比較麻煩，因此這是mAP@0.5指標(biāo)很有價(jià)值的地方。

mAP@0.5

mAP@0.5

This metric contains two pieces. First, ‘mAP’ indicates the mean Average Precision or correctness of each of the 3 labels taking into account all labels. Second, ‘@0.5’ sets a threshold for how much of the predicted fish bounding box overlaps the original annotation. This second part is a key metric in object detection; it prevents the model from getting credit for guessing the correct fish but drawing a box around some other artifact (like a shadow) instead of an actual fish.

該指標(biāo)包含兩部分。 首先，“ mAP”表示考慮了所有標(biāo)簽的3個(gè)標(biāo)簽中的每個(gè)標(biāo)簽的平均平均精度或正確性。 第二，“ @ 0.5”為預(yù)測(cè)的魚邊界框與原始注釋重疊的閾值進(jìn)行設(shè)置。 第二部分是對(duì)象檢測(cè)中的關(guān)鍵指標(biāo)。 它會(huì)阻止模型因猜測(cè)正確的魚而獲得信譽(yù)，但會(huì)在其他一些人工物體(例如陰影)周圍繪制方框而不是實(shí)際的魚。

mAP@0.5 is a common metric for evaluating object detectionmAP@0.5是用于評(píng)估對(duì)象檢測(cè)的常用指標(biāo)

This model achieved a mAP@0.5 of 70% —but is that good or bad? For some applications, it’s more than enough. In this particular application, there’s more to it than the label assigned to a single image, and ideally the results of a full solution need to be compared to an estimated error in our current fish counting methods.

該模型的mAP@0.5為70％，但這是好是壞？ 對(duì)于某些應(yīng)用程序，這已經(jīng)綽綽有余。 在這個(gè)特定的應(yīng)用程序中，除了分配給單個(gè)圖像的標(biāo)簽之外，還有更多其他功能，并且理想情況下，需要將完整解決方案的結(jié)果與我們當(dāng)前的魚類計(jì)數(shù)方法中的估計(jì)誤差進(jìn)行比較。

Informative — altered lighting conditions, shadows, and crowded conditions can result in both under and over counts. In the image below, the confidence threshold was intentionally set low at 0.4 to expose corner cases or images that provide a challenge for the model. Keep in mind that the goal is to see if we’re ready to use this for a real-life challenge, and that means looking for problems!

信息化 -改變照明條件，陰影和擁擠的條件下可能會(huì)導(dǎo)致這兩個(gè)下和超過計(jì)數(shù)。 在下圖中，將置信度閾值故意設(shè)置為0.4低，以暴露對(duì)模型提出挑戰(zhàn)的極端情況或圖像。 請(qǐng)記住，目標(biāo)是查看我們是否準(zhǔn)備好將其用于現(xiàn)實(shí)生活中的挑戰(zhàn)，這意味著尋找問題！

Model adds an extra fish with a low confidence score of 0.42模型添加了一個(gè)額外的魚，且魚的可信度低至0.42

In the actual application, when the objects are tracked from one frame of video to the next, shadows often move and the label disappears. This next level of challenge will be to address the object tracking from frame to frame so that there is exactly 1 count for each fish, no matter how much time it remains in the viewing window.

在實(shí)際應(yīng)用中，當(dāng)將對(duì)象從視頻的一幀跟蹤到下一幀時(shí)，陰影經(jīng)常移動(dòng)并且標(biāo)簽消失。 接下來的挑戰(zhàn)將是解決從一個(gè)幀到另一個(gè)幀的對(duì)象跟蹤問題，以便每條魚的精確計(jì)數(shù)為1，無論它在觀察窗口中保留了多少時(shí)間。

步驟5：考慮后續(xù)步驟 (Step 5: Consider next steps)

Now the most important step — what did I learn?

現(xiàn)在最重要的一步-我學(xué)到了什么？

• Images with excellent lighting are required
  需要具有出色照明的圖像
• Viewing window height and width are not critical, but the depth needs to be carefully selected to reduce the number of fish that can obscure other fish
  查看窗口的高度和寬度不是關(guān)鍵，但是需要仔細(xì)選擇深度以減少可能使其他魚遮擋的魚的數(shù)量
• Correct species labels are required for training a model to separate sockeye, chinook, and coho in addition to other species. Not being a fish identification expert, there is the possibility that I mistook a scar for a small adipose fin. Correcting mislabeled images is another way to improve the model.
  訓(xùn)練模型以分離其他種類的紅大馬，chinook和coho時(shí)，需要正確的物種標(biāo)簽。 如果不是魚類鑒定專家，我可能會(huì)誤以為是小脂肪鰭的傷痕。 校正貼錯(cuò)標(biāo)簽的圖像是改善模型的另一種方法。
• Salmon swimming upstream in a fish ladder pause to rest for varying amounts of time. In some cases, they will swim slowly and maintain position, and at other times they will slow to the point that they drift backward with the current. This adds an additional level of complexity that will require an advanced system to track objects (fish) from one video frame to the next.
  在魚梯上游游的鮭魚會(huì)停下來休息不同的時(shí)間。 在某些情況下，它們會(huì)緩慢游動(dòng)并保持姿勢(shì)，而在另一些情況下，它們會(huì)減速到隨著電流向后漂移的程度。 這增加了額外的復(fù)雜度，這將需要高級(jí)系統(tǒng)來跟蹤從一個(gè)視頻幀到下一個(gè)視頻幀的對(duì)象(魚)。

Most importantly, while the metrics above are from the better images, most of this learning happened on the first set of images, underscoring the point that excellent images are not required to make progress in trying out a project with deep learning and object detection. In other words, not finding ideal images shouldn’t hold you up from getting started!

最重要的是，盡管上述指標(biāo)來自更好的圖像，但大多數(shù)學(xué)習(xí)是在第一組圖像上進(jìn)行的，這突出說明了在嘗試進(jìn)行深度學(xué)習(xí)和對(duì)象檢測(cè)的項(xiàng)目時(shí)，不需要出色的圖像即可取得進(jìn)展。 換句話說，沒有找到理想的圖像不應(yīng)阻止您入門！

結(jié)論 (Conclusion)

Based on the project so far, I think it’s fair to say that machine learning / AI / deep learning are ready to be taken out of the lab and applied to real world projects.

基于到目前為止的項(xiàng)目，我認(rèn)為可以說機(jī)器學(xué)習(xí)/人工智能/深度學(xué)習(xí)已經(jīng)可以從實(shí)驗(yàn)室中帶出并應(yīng)用于實(shí)際項(xiàng)目中了。

This is good news — as an indicator species, salmon help us assess the health of our environment. Accurate and timely fish counts are critical to ensuring their survival, and ours.

這是一個(gè)好消息-鮭魚作為指示物種，可以幫助我們?cè)u(píng)估環(huán)境的健康。 準(zhǔn)確及時(shí)的魚類計(jì)數(shù)對(duì)于確保它們以及我們的生存至關(guān)重要。

Contact me! I’m open to new opportunities and actively looking for my next exciting data science gig.

聯(lián)絡(luò)我！ 我樂于接受新的機(jī)遇，并積極尋找下一個(gè)令人興奮的數(shù)據(jù)科學(xué)演出。

Link to Repository

鏈接到存儲(chǔ)庫

The notebooks used for this project are posted in github. Keep in mind that these are designed to be used on Google Colab and will not run locally. The 3rd notebook contains a pared down version that runs inference only using saved model weights.

用于該項(xiàng)目的筆記本發(fā)布在github中。 請(qǐng)記住，這些文件只能在Google Colab上使用，不能在本地運(yùn)行。 第三個(gè)筆記本包含精簡版，僅使用保存的模型權(quán)重運(yùn)行推斷。

https://github.com/jshaffer94247/Counting-Fish

https://github.com/jshaffer94247/Counting-Fish

翻譯自: https://medium.com/@jshaffer94247_86066/applying-deep-learning-to-environmental-issues-73a861f3a4ec

數(shù)據(jù)庫課程設(shè)計(jì)結(jié)論

總結(jié)

以上是生活随笔為你收集整理的数据库课程设计结论_结论的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。