當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

iMX8MPlus和iMX8QM机器学习框架eIQ性能对比

發布時間：2023/12/10 编程问答 32 豆豆

生活随笔收集整理的這篇文章主要介紹了 iMX8MPlus和iMX8QM机器学习框架eIQ性能对比小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

By Toradex?胡珊逢

機器學習算法對算力要求較高，通常會采用?GPU?，或者專用的處理器如?NPU?進行加速運算。NXP?先后推出的兩款處理器iMX8QuadMax ?和?iMX8M Plus?分別可以采用?GPU?和?NPU?對常用的機器學習算法例如?TensorFlow Lite?等進行加速。文章將使用?NXP eIQ?框架在兩個處理器上測試不同算法的性能。

這里我們將使用?Toradex?的?Apalis iMX8QM 4GB WB IT V1.1C?和?Verdin iMX8M Plus Quad 4GB WB IT V1.0B?兩個模塊。BSP?為?Linux BSP?V5.3?。eIQ?采用?zeus-5.4.70-2.3.3?版本。Toradex?默認?Yocto Project?編譯環境并沒有直接集成??eIQ?軟件，可以參考這里添加?meta-ml layer?并進行編譯。然后修改??meta-ml/recipes-devtools/python/python3-pybind11_2.5.0.bb?中的Python?版本為?3.8?。最后可以生成??multimedia image。

-------------------------------------

EXTRA_OECMAKE = "-DPYBIND11_TEST=OFF \?
-DPYTHON_EXECUTABLE=${RECIPE_SYSROOT_NATIVE}/usr/bin/python3-native/python3.8 \ "

-------------------------------------

使用?Toradex Easy Installer?將生成的鏡像安裝到??Apalis iMX8QM 4GB WB IT V1.1C?和?Verdin iMX8M Plus Quad 4GB WB IT V1.0B?兩個模塊上。

測試的內容參考?NXP?的?i.MX_Machine_Learning_User's_Guide?文檔進行，包括?TensorFlow Lite、Arm NN、ONNX、PyTorch。由于目前??OpenCV?還只能運行在?iMX8QuadMax ?和?iMX8M Plus?的?CPU?上，無法使用?GPU?或者?NPU?加速，所以本次不做測試。另外，在使用?Arm NN?測試?Caffe?模型時有兩個限制。第一，batch size?必須為?1。例如??deploy.prototxt?文件修改為

-------------------------------------

name: "AlexNet"

layer {

??name: "data"

??type: "Input"

??top: "data"

??input_param { shape: { dim: 1 dim: 3 dim: 227 dim: 227 } }

}

-------------------------------------

第二，?Arm NN?不支持所有的?Caffe?語法，一些老的神經網絡模型文件需要更新到最新的??Caffe?語法。下面是?PC?上用于轉換的?Python3?腳本。

-------------------------------------

import caffe

net = caffe.Net('lenet.prototxt', 'lenet_iter_9000-orignal.caffemodel', caffe.TEST)

net.save('lenet_iter_9000.caffemodel')

-------------------------------------

在兩個模塊上測試結果如下。

TensorFlow Lite

l?Apalis iMX8QM

label_image

-------------------------------------

root@apalis-imx8:/usr/bin/tensorflow-lite-2.4.0/examples# USE_GPU_INFERENCE=1 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1

INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
INFO: Applied NNAPI delegate.
INFO: invoked
INFO: average time: 12.407 ms?
INFO: 0.784314: 653 military uniform
INFO: 0.105882: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.0117647: 466 bulletproof vest
INFO: 0.00784314: 668 mortarboard

-------------------------------------

benchmark_model

-------------------------------------

root@apalis-imx8:/usr/bin/tensorflow-lite-2.4.0/examples# ./benchmark_model --graph=mobilenet_v1_1.0_224_quant.tflite --use_nnapi=true

STARTING!

Log parameter values verbosely: [0]
Graph: [mobilenet_v1_1.0_224_quant.tflite]
Use NNAPI: [1]
NNAPI accelerators available: [vsi-npu]
Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: Created TensorFlow Lite delegate for NNAPI.
Explicitly applied NNAPI delegate, and the model graph will be completely executed by the
delegate.
The input model file size (MB): 4.27635
Initialized session in 16.746ms.
Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150
seconds.
count=17 first=305296 curr=12471 min=12299 max=305296 avg=29650 std=68911
Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.
count=81 first=12417 curr=12430 min=12294 max=12511 avg=12405.6 std=39
Inference timings in us: Init: 16746, First inference: 305296, Warmup (avg): 29650, Inference (avg): 12405.6
Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
Peak memory footprint (MB): init=1.85938 overall=55.1406

-------------------------------------

l?Verdin iMX8M Plus

label_image

-------------------------------------

root@verdin-imx8mp:/usr/bin/tensorflow-lite-2.4.0/examples# USE_GPU_INFERENCE=0 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
INFO: Applied NNAPI delegate.
INFO: invoked
INFO: average time: 2.835 ms?
INFO: 0.768627: 653 military uniform
INFO: 0.105882: 907 Windsor tie
INFO: 0.0196078: 458 bow tie
INFO: 0.0117647: 466 bulletproof vestINFO: 0.00784314: 835 suit

-------------------------------------

benchmark_model

-------------------------------------

root@verdin-imx8mp:/usr/bin/tensorflow-lite-2.4.0/examples# ./benchmark_model --graph=mobilenet_v1_1.0_224_quant.tflite --use_nnapi=true?
STARTING!?
Log parameter values verbosely: [0]?
Graph: [mobilenet_v1_1.0_224_quant.tflite]?
Use NNAPI: [1]?
NNAPI accelerators available: [vsi-npu]?
Loaded model mobilenet_v1_1.0_224_quant.tflite?
INFO: Created TensorFlow Lite delegate for NNAPI.?
Explicitly applied NNAPI delegate, and the model graph will be completely executed by the delegate.?
The input model file size (MB): 4.27635?
Initialized session in 16.79ms.?
Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.?
count=1 curr=6664535?
Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.?
count=367 first=2734 curr=2646 min=2624 max=2734 avg=2650.05 std=16?
Inference timings in us: Init: 16790, First inference: 6664535, Warmup (avg): 6.66454e+06, Inference (avg): 2650.05?
Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.?
Peak memory footprint (MB): init=1.79297 overall=28.5117?

-------------------------------------

Arm NN

l?Apalis iMX8QM

CaffeAlexNet-Armnn

-------------------------------------

root@apalis-imx8:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeAlexNet-Armnn --data-dir=data --model-dir=models
Info: ArmNN v22.0.0
Info: Initialization time: 0.14 ms
Info: Network parsing time: 1397.76 ms
Info: Optimization time: 195.13 ms
Info: = Prediction values for test #0
Info: Top(1) prediction is 2 with value: 0.706226
Info: Top(2) prediction is 0 with value: 1.26573e-05
Info: Total time for 1 test cases: 0.264 seconds
Info: Average time per test case: 263.701 ms
Info: Overall accuracy: 1.000
Info: Shutdown time: 56.83 ms

-------------------------------------

CaffeMnist-Armnn

-------------------------------------

root@apalis-imx8:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeMnist-Armnn --data-dir=data --model-dir=models

Info: ArmNN v22.0.0
Info: Initialization time: 0.09 ms
Info: Network parsing time: 8.70 ms
Info: Optimization time: 2.67 ms?
Info: = Prediction values for test #0?
Info: Top(1) prediction is 7 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #1?
Info: Top(1) prediction is 2 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #5?
Info: Top(1) prediction is 1 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #8?
Info: Top(1) prediction is 5 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #9?
Info: Top(1) prediction is 9 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: Total time for 5 test cases: 0.015 seconds?
Info: Average time per test case: 2.927 ms?
Info: Overall accuracy: 1.000?
Info: Shutdown time: 1.56 ms?

-------------------------------------

CaffeVGG-Armnn

-------------------------------------

root@apalis-imx8:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeVGG-Armnn --data-dir=data --model-dir=models

Info: ArmNN v22.0.0?
Info: Initialization time: 0.08 ms?
Info: Network parsing time: 1452.35 ms?
Info: Optimization time: 491.98 ms?
Info: = Prediction values for test #0?
Info: Top(1) prediction is 2 with value: 0.692014?
Info: Top(2) prediction is 0 with value: 9.80887e-07?
Info: Total time for 1 test cases: 2.723 seconds?
Info: Average time per test case: 2722.846 ms?
Info: Overall accuracy: 1.000?
Info: Shutdown time: 115.74 ms

-------------------------------------

l?Verdin iMX8M Plus

CaffeAlexNet-Armnn

-------------------------------------

root@verdin-imx8mp:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeAlexNet-Armnn --data-dir=data --model-dir=models

Info: ArmNN v22.0.0?
Info: Initialization time: 0.12 ms?
Info: Network parsing time: 1250.55 ms?
Info: Optimization time: 141.40 ms?
Info: = Prediction values for test #0?
Info: Top(1) prediction is 2 with value: 0.706225?
Info: Top(2) prediction is 0 with value: 1.26573e-05?
Info: Total time for 1 test cases: 0.110 seconds?
Info: Average time per test case: 110.124 ms?
Info: Overall accuracy: 1.000?
Info: Shutdown time: 15.04 ms

-------------------------------------

CaffeMnist-Armnn

-------------------------------------

root@verdin-imx8mp:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeMnist-Armnn --data-dir=data --model-dir=models

Info: ArmNN v22.0.0?
Info: Initialization time: 0.11 ms?
Info: Network parsing time: 8.96 ms?
Info: Optimization time: 3.01 ms?
Info: = Prediction values for test #0?
Info: Top(1) prediction is 7 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #1?
Info: Top(1) prediction is 2 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #5?
Info: Top(1) prediction is 1 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #8?
Info: Top(1) prediction is 5 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: = Prediction values for test #9?
Info: Top(1) prediction is 9 with value: 1?
Info: Top(2) prediction is 0 with value: 0?
Info: Total time for 5 test cases: 0.008 seconds?
Info: Average time per test case: 1.608 ms?
Info: Overall accuracy: 1.000?
Info: Shutdown time: 1.69 ms?

-------------------------------------

CaffeVGG-Armnn

-------------------------------------

root@verdin-imx8mp:/usr/bin/armnn-20.08/ArmnnTests# ../CaffeVGG-Armnn --data-dir=data --model-dir=modelsInfo: ArmNN v22.0.0

Info: Initialization time: 0.15 ms?
Info: Network parsing time: 2842.95 ms?
Info: Optimization time: 316.74 ms?
Info: = Prediction values for test #0?
Info: Top(1) prediction is 2 with value: 0.692015?
Info: Top(2) prediction is 0 with value: 9.8088e-07?
Info: Total time for 1 test cases: 1.098 seconds?
Info: Average time per test case: 1097.593 ms?
Info: Overall accuracy: 1.000?
Info: Shutdown time: 130.65 ms?

-------------------------------------

ONNX

l?Apalis iMX8QM

onnx_test_runner

-------------------------------------

root@apalis-imx8:~# time onnx_test_runner -j 1 -c 1 -r 1 -e vsi_npu ./mobilenetv2-7/

result: ?
Models: 1?
Total test cases: 3?
?Succeeded: 3?
?Not implemented: 0?
?Failed: 0?
Stats by Operator type:?
?Not implemented(0): ?
?Failed:?
Failed Test Cases:?
?
real?0m0.643s?
user?0m1.513s?
sys?0m0.111s

-------------------------------------

l?Verdin iMX8M Plus

onnx_test_runner

-------------------------------------

root@verdin-imx8mp:~# time onnx_test_runner -j 1 -c 1 -r 1 -e vsi_npu ./mobilenetv2-7/

-------------------------------------

PyTorch

l?Apalis iMX8QM

pytorch_mobilenetv2.py

-------------------------------------

root@apalis-imx8:/usr/bin/pytorch/examples# time python3 pytorch_mobilenetv2.py

('tabby, tabby cat', 46.348018646240234)?
('tiger cat', 35.17843246459961)?
('Egyptian cat', 15.802857398986816)?
('lynx, catamount', 1.161122441291809)?
('tiger, Panthera tigris', 0.20774582028388977)?
?
real?0m8.806s?
user?0m7.440s?
sys?0m0.593s?

-------------------------------------

l?Verdin iMX8M Plus

pytorch_mobilenetv2.py

-------------------------------------

root@verdin-imx8mp:/usr/bin/pytorch/examples# time python3 pytorch_mobilenetv2.py

-------------------------------------

匯總對比

根據具體測試應用不同，兩者之間的性能差距大小不一。總體來看常用機器學習算法在?Verdin iMX8M Plus?的?NPU?上的表現會優于?Apalis iMX8QM?的?GPU。

總結

機器學習是較為復雜的應用，除了硬件處理器外，影響算法性能表現的還包括對模型本身的優化。尤其是對嵌入式系統有限的處理能力來講，直接將?PC?上現成的模型拿過來用通常會表現不佳。同時根據項目需求選擇合適計算機模塊，畢竟?Verdin iMX8M Plus?和?Apalis iMX8QM?的用途側重點不同。

總結

以上是生活随笔為你收集整理的iMX8MPlus和iMX8QM机器学习框架eIQ性能对比的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。