This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

系统跑着跑着出现异常,网络使用不了,down在了gen_pool_free函数

Other Parts Discussed in Thread: AM3356

我们的板子使用的是AM3356,网络芯片使用LAN8710。使用的是CPSW网口,之前用着都是正常的。最近系统老是出现错误信息,然后网络就用不了了。我尝试使用ifconfig eth0 down命令关闭该网口,然后系统就死机了。困扰了好久,一直找不到原因,希望各位大神指点指点。出现的错误信息如下:

[ 1350.242463] ------------[ cut here ]------------
[ 1350.242476] Kernel BUG at c02af514 [verbose debug info unavailable]
[ 1350.242485] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
[ 1350.242539] Modules linked in: interrupt(O) musb_dsps musb_hdrc udc_core usbcore ecb prueth sha256_generic wkup_m3_ipc hmac md5 wkup_m3_rproc sha1_generic sha1_arm_neon sha1_arm omap_aes_driver omap_sham pru_rproc pruss_intc pruss remoteproc virtio virtio_ring omap_rng rng_core musb_am335x omap_wdt sch_fq_codel
[ 1350.242553] CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G           O    4.4.32-rt41 #120
[ 1350.242557] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 1350.242562] task: cf050d00 ti: cf060000 task.ti: cf060000
[ 1350.242580] PC is at gen_pool_free+0xd8/0xdc
[ 1350.242584] LR is at gen_pool_free+0xd8/0xdc
[ 1350.242590] pc : [<c02af514>]    lr : [<c02af514>]    psr: 60000013
               sp : cf061db8  ip : cf061db8  fp : cf061ddc
[ 1350.242593] r10: c001a3f0  r9 : 10000000  r8 : 00000020
[ 1350.242598] r7 : 00000005  r6 : 00000001  r5 : d083c0e0  r4 : cf42b914
[ 1350.242602] r3 : 00000000  r2 : cf050d00  r1 : d083c0e0  r0 : cf42b914
[ 1350.242609] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[ 1350.242615] Control: 10c5387d  Table: 8a5c4019  DAC: 00000051
[ 1350.242619] Process ksoftirqd/0 (pid: 3, stack limit = 0xcf060208)
[ 1350.242624] Stack: (0xcf061db8 to 0xcf062000)
[ 1350.242632] 1da0:                                                       d083c0e0 cf3a7f10
[ 1350.242643] 1dc0: cf42b890 ca5890b8 0000043e 10000000 cf061e14 cf061de0 c03abb2c c02af448
[ 1350.242653] 1de0: 00000000 c000945c cf1390c0 cf3a7f10 10000000 d083c0e0 cf3a7f30 cf42b890
[ 1350.242663] 1e00: 0000043e 4a1020e0 cf061e4c cf061e18 c03abc94 c03abab0 cf061e50 d083c000
[ 1350.242673] 1e20: c006b654 00000000 cf3a7f10 00000040 c0767980 0000012c cf061ea8 00019a42
[ 1350.242683] 1e40: cf061e6c cf061e50 c03abcf8 c03abb68 cf4290c0 00000001 00000040 c0767980
[ 1350.242693] 1e60: cf061e8c cf061e70 c03b35d4 c03abcc0 cf4290c0 00000001 00000040 c0767980
[ 1350.242703] 1e80: cf061ee4 cf061e90 c0441fb0 c03b35c4 c06dd7bc c06e11d4 c06dd4ac c075a778
[ 1350.242713] 1ea0: c079eb29 c079e7c0 cf429040 cf429040 cf061eb0 cf061eb0 c0765c80 cf060000
[ 1350.242723] 1ec0: 0000000c c0763a8c 00000060 04208040 00000003 00000000 cf061f24 cf061ee8
[ 1350.242733] 1ee0: c0034604 c0441dcc 5f866b22 0000013a 5f864efa c079f8c0 60000013 ffffe000
[ 1350.242743] 1f00: cf015880 00000001 c0763a24 ffffe000 00000000 00000000 cf061f3c cf061f28
[ 1350.242753] 1f20: c0034910 c0034494 cf060000 cf015880 cf061f64 cf061f40 c004fa98 c00348e8
[ 1350.242762] 1f40: 00000000 cf0158c0 cf015880 c004f93c 00000000 00000000 cf061fac cf061f68
[ 1350.242772] 1f60: c004c8c4 c004f948 00000000 00000001 00000000 cf015880 00000000 cf061f7c
[ 1350.242782] 1f80: cf061f7c 00000000 cf061f88 cf061f88 cf0158c0 c004c7e8 00000000 00000000
[ 1350.242792] 1fa0: 00000000 cf061fb0 c000fac0 c004c7f4 00000000 00000000 00000000 00000000
[ 1350.242801] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1350.242810] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 1350.242814] Backtrace:
[ 1350.242836] [<c02af43c>] (gen_pool_free) from [<c03abb2c>] (__cpdma_chan_free+0x88/0xb8)
[ 1350.242850]  r9:10000000 r8:0000043e r7:ca5890b8 r6:cf42b890 r5:cf3a7f10 r4:d083c0e0
[ 1350.242860] [<c03abaa4>] (__cpdma_chan_free) from [<c03abc94>] (__cpdma_chan_process+0x138/0x158)
[ 1350.242874]  r10:4a1020e0 r9:0000043e r8:cf42b890 r7:cf3a7f30 r6:d083c0e0 r5:10000000
[ 1350.242877]  r4:cf3a7f10
[ 1350.242888] [<c03abb5c>] (__cpdma_chan_process) from [<c03abcf8>] (cpdma_chan_process+0x44/0x5c)
[ 1350.242899]  r10:00019a42 r9:cf061ea8 r8:0000012c r7:c0767980 r6:00000040 r5:cf3a7f10
[ 1350.242903]  r4:00000000
[ 1350.242920] [<c03abcb4>] (cpdma_chan_process) from [<c03b35d4>] (cpsw_tx_poll+0x1c/0x7c)
[ 1350.242929]  r7:c0767980 r6:00000040 r5:00000001 r4:cf4290c0
[ 1350.242950] [<c03b35b8>] (cpsw_tx_poll) from [<c0441fb0>] (net_rx_action+0x1f0/0x2f0)
[ 1350.242959]  r7:c0767980 r6:00000040 r5:00000001 r4:cf4290c0
[ 1350.242978] [<c0441dc0>] (net_rx_action) from [<c0034604>] (do_current_softirqs+0x17c/0x22c)
[ 1350.242989]  r10:00000000 r9:00000003 r8:04208040 r7:00000060 r6:c0763a8c r5:0000000c
[ 1350.242993]  r4:cf060000
[ 1350.243003] [<c0034488>] (do_current_softirqs) from [<c0034910>] (run_ksoftirqd+0x34/0x64)
[ 1350.243015]  r10:00000000 r9:00000000 r8:ffffe000 r7:c0763a24 r6:00000001 r5:cf015880
[ 1350.243018]  r4:ffffe000
[ 1350.243033] [<c00348dc>] (run_ksoftirqd) from [<c004fa98>] (smpboot_thread_fn+0x15c/0x288)
[ 1350.243039]  r5:cf015880 r4:cf060000
[ 1350.243055] [<c004f93c>] (smpboot_thread_fn) from [<c004c8c4>] (kthread+0xdc/0xf4)
[ 1350.243067]  r9:00000000 r8:00000000 r7:c004f93c r6:cf015880 r5:cf0158c0 r4:00000000
[ 1350.243083] [<c004c7e8>] (kthread) from [<c000fac0>] (ret_from_fork+0x14/0x34)
[ 1350.243092]  r7:00000000 r6:00000000 r5:c004c7e8 r4:cf0158c0
[ 1350.243103] Code: 1afffffa ebf705eb e89dabf0 ebf705e9 (e7f001f2)
[ 1350.692458] ---[ end trace 0000000000000002 ]---
[ 1366.078048] NOHZ: local_softirq_pending 08
[ 1366.078321] NOHZ: local_softirq_pending 08
[ 1366.078459] NOHZ: local_softirq_pending 08
[ 1366.079250] NOHZ: local_softirq_pending 08
[ 1366.079324] NOHZ: local_softirq_pending 08
[ 1366.079651] NOHZ: local_softirq_pending 08
[ 1366.080306] NOHZ: local_softirq_pending 08
[ 1366.080641] NOHZ: local_softirq_pending 08
[ 1366.081306] NOHZ: local_softirq_pending 08
[ 1366.081639] NOHZ: local_softirq_pending 08
[ 1529.838570] BUG: scheduling while atomic: irq/159-4802200/815/0x00000002
[ 1529.838644] Modules linked in: interrupt(O) musb_dsps musb_hdrc udc_core usbcore ecb prueth sha256_generic wkup_m3_ipc hmac md5 wkup_m3_rproc sha1_generic sha1_arm_neon sha1_arm omap_aes_driver omap_sham pru_rproc pruss_intc pruss remoteproc virtio virtio_ring omap_rng rng_core musb_am335x omap_wdt sch_fq_codel
[ 1529.838658] CPU: 0 PID: 815 Comm: irq/159-4802200 Tainted: G      D    O    4.4.32-rt41 #120
[ 1529.838662] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 1529.838668] Backtrace:
[ 1529.838706] [<c0013494>] (dump_backtrace) from [<c0013690>] (show_stack+0x18/0x1c)
[ 1529.838716]  r7:c0547254 r6:c0765c80 r5:00000000 r4:00000000
[ 1529.838727] [<c0013678>] (show_stack) from [<c0295b0c>] (dump_stack+0x24/0x28)
[ 1529.838743] [<c0295ae8>] (dump_stack) from [<c0050b1c>] (__schedule_bug+0x54/0x64)
[ 1529.838761] [<c0050ac8>] (__schedule_bug) from [<c0547144>] (__schedule+0x490/0x540)
[ 1529.838767]  r5:00000000 r4:ca213400
[ 1529.838776] [<c0546cb4>] (__schedule) from [<c0547254>] (schedule+0x60/0xf8)
[ 1529.838787]  r10:00000000 r9:c079e9ab r8:c006b194 r7:cf11d3c0 r6:00000001 r5:ca213400
[ 1529.838791]  r4:ca5b4000
[ 1529.838807] [<c05471f4>] (schedule) from [<c006b434>] (irq_thread+0xa4/0x1d8)
[ 1529.838813]  r5:ca5b4000 r4:cf635e40
[ 1529.838830] [<c006b390>] (irq_thread) from [<c004c8c4>] (kthread+0xdc/0xf4)
[ 1529.838841]  r9:00000000 r8:00000000 r7:c006b390 r6:cf635e40 r5:ca46bd00 r4:00000000
[ 1529.838852] [<c004c7e8>] (kthread) from [<c000fac0>] (ret_from_fork+0x14/0x34)
[ 1529.838861]  r7:00000000 r6:00000000 r5:c004c7e8 r4:ca46bd00

  • 请问有没有尝试换一块板子吗? 只有网口跑死?其他都好的?
  • 除了网口,其他都正常,换了其他板子没问题。在这个打印信息出来之前,有这么一句话:
    [ 40.014183] edma 49000000.edma: dma_ccerr_handler: Error interrupt without error event!
    之后才会出现上面提到的打印信息:
    [ 40.014183] edma 49000000.edma: dma_ccerr_handler: Error interrupt without error event!
    [ 1350.242463] ------------[ cut here ]------------
    [ 1350.242476] Kernel BUG at c02af514 [verbose debug info unavailable]
    [ 1350.242485] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
    [ 1350.242539] Modules linked in: interrupt(O) musb_dsps musb_hdrc udc_core usbcore ecb prueth sha256_generic wkup_m3_ipc hmac md5 wkup_m3_rproc sha1_generic sha1_arm_neon sha1_arm omap_aes_driver omap_sham pru_rproc pruss_intc pruss remoteproc virtio virtio_ring omap_rng rng_core musb_am335x omap_wdt sch_fq_codel
    [ 1350.242553] CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G O 4.4.32-rt41 #120
    [ 1350.242557] Hardware name: Generic AM33XX (Flattened Device Tree)
    [ 1350.242562] task: cf050d00 ti: cf060000 task.ti: cf060000
    [ 1350.242580] PC is at gen_pool_free+0xd8/0xdc
    [ 1350.242584] LR is at gen_pool_free+0xd8/0xdc
    [ 1350.242590] pc : [<c02af514>] lr : [<c02af514>] psr: 60000013
    sp : cf061db8 ip : cf061db8 fp : cf061ddc
    [ 1350.242593] r10: c001a3f0 r9 : 10000000 r8 : 00000020
    [ 1350.242598] r7 : 00000005 r6 : 00000001 r5 : d083c0e0 r4 : cf42b914
    [ 1350.242602] r3 : 00000000 r2 : cf050d00 r1 : d083c0e0 r0 : cf42b914
    [ 1350.242609] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
  • 如果同样的驱动程序能在其他同样的板子上能跑的话,硬件稳定性方面的问题更大些。
    请先一下PHY电路,看一下PHY时钟的稳定性。DSP 这边的话把DDR降频跑试试。
  • 板间差异检查一下硬件的稳定性

  • 这里有个和你类似的错误,你可以看看:e2e.ti.com/.../649686
  • 这个链接讲的根本不是我这个问题...
  • 问题是不一样,里面的提示的错误类似