This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

[参考译文] PROCESSOR-SDK-AM62A:错误:正在运行 TIDL 图形...失败!!!

Guru**** 2407630 points
Other Parts Discussed in Thread: PROCESSOR-SDK-AM62A

请注意,本文内容源自机器翻译,可能存在语法或其它翻译错误,仅供参考。如需获取准确内容,请参阅链接中的英语原文或自行翻译。

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1488751/processor-sdk-am62a-error-running-tidl-graph-failed

器件型号:PROCESSOR-SDK-AM62A

工具与软件:

我使用 edgeai-tidl-tools 在主机 Linux 机器上的 PROCESSOR-SDK-AM62A 10.00.00.08上编译了物体检测模型(Yolox)。

在 AM62A-EVM 上运行推理并构建10.00.00.08映像时、收到错误:

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['YolovXs6']


Running_Model :  YolovXs6  

libtidl_onnxrt_EP loaded 0x54d0d00 
Final number of subgraphs created are : 1, - Offloaded Nodes - 254, Total Nodes - 254 
APP: Init ... !!!
   179.495780 s: MEM: Init ... !!!
   179.495855 s: MEM: Initialized DMA HEAP (fd=5) !!!
   179.496065 s: MEM: Init ... Done !!!
   179.496139 s: IPC: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio0.rpmsg_chrdev.-1.13
   179.514077 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_1] !!!
   179.514100 s: IPC: Init ... Done !!!
APP: ERROR: IPC init failed !!!
REMOTE_SERVICE: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio2.rpmsg_chrdev.-1.21
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio0.rpmsg_chrdev.-1.21
REMOTE_SERVICE: Init ... Done !!!
   179.524210 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   179.537258 s:  VX_ZONE_INIT:Enabled
   179.537293 s:  VX_ZONE_ERROR:Enabled
   179.537316 s:  VX_ZONE_WARNING:Enabled
   179.538577 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
   179.538788 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
   179.538927 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
   179.539073 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
   179.539093 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   179.543202 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
   179.576330 s:  VX_ZONE_ERROR:[ownIpcSendMsg:141] OpenVX send notification failed
   179.576376 s:  VX_ZONE_ERROR:[ownObjDescSend:315] ownIpcSendMsg failed
   179.576392 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
   179.576584 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
   179.576605 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
   179.576621 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:1:2 ... failed !!!
   179.576668 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
   179.576680 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
NodeArg(name='input', type='tensor(float)', shape=[1, 3, 640, 640]) [1, 3, 640, 640] True
   179.849837 s:  VX_ZONE_ERROR:[ownIpcSendMsg:141] OpenVX send notification failed
   179.849887 s:  VX_ZONE_ERROR:[ownObjDescSend:315] ownIpcSendMsg failed
   179.849908 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
   179.849921 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
   179.849933 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
   179.849949 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:1:2 ... failed !!!
   179.849965 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
   179.849977 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
   179.850096 s:  VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:919] graph is not in a state required to be scheduled
   179.850110 s:  VX_ZONE_ERROR:[vxProcessGraph:844] schedule graph failed
   179.850121 s:  VX_ZONE_ERROR:[vxProcessGraph:849] wait graph failed
ERROR: Running TIDL graph ... Failed !!!
2025-03-18 05:54:33.086648216 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Status Message: TIDL Compute Invoke Failed.
Traceback (most recent call last):
  File "/opt/edgeai-gst-apps/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep_modified.py", line 557, in <module>
    run_model(model, mIdx)
  File "/opt/edgeai-gst-apps/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep_modified.py", line 395, in run_model
    imgs, output, proc_time, sub_graph_time, height, width = infer_image(sess, input_images, config)
                                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/edgeai-gst-apps/edgeai-tidl-tools/examples/osrt_python/ort/onnxrt_ep_modified.py", line 210, in infer_image
    output = list(sess.run(None, {input_name: input_data}))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 200, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Status Message: TIDL Compute Invoke Failed.
   179.941587 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
   179.946413 s:  VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
   179.946490 s: IPC: ERROR: Failed to close created endpoint !!!
   179.946503 s: IPC: ERROR: Failed to close created endpoint !!!
REMOTE_SERVICE: Deinit ... Done !!!
   179.946523 s: IPC: Deinit ... !!!
Segmentation fault (core dumped)

  • 请注意,本文内容源自机器翻译,可能存在语法或其它翻译错误,仅供参考。如需获取准确内容,请参阅链接中的英语原文或自行翻译。

    您好!

    我们将找到该问题的解决方案。  

    我在您的日志中看到、  

    _rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio0.rpmsg_chrdev.-1.13

    这表明内核之间的 IPC (包括来自 A53和 AI 加速器(也称为 C7xMMA))无法正确设置驱动程序。

    您是否可以重新启动、保存控制台日志(使用 dmesg 实用程序访问)、然后重试应用程序?

    我们还可以确保远程内核通过以下方式运行:  

    cat /sys/class/remoteproc/remoteproc*/state

    全部应"已连接"或"正在运行"。 如果没有、请分享有关哪一个的详细信息(请在上面的命令中使用"名称"、而不是"状态")

    BR、
    Reese