求助,启动的时候出现rcu_preempt self-detected stall on cpu

启动讨论 grub/grub2/syslinux/grub4dos/Lilo
回复
chengshuyi
帖子: 20
注册时间: 2018-10-18 10:18
系统: ubuntu16

求助,启动的时候出现rcu_preempt self-detected stall on cpu

#1

帖子 chengshuyi » 2019-06-25 11:13

下面是详细的运行日志,大概每60秒会出现rcu_preempt self-detected stall on cpu

从日志上面的看,大概有两类Call trace,两类Exception stack。

Call trace 1. rcu进程的调用堆栈

代码: 全选

[   21.791774] rcu_preempt     I    0     8      2 0x00000020
[   21.797149] Call trace:
[   21.799596] [<ffff000008085cf4>] __switch_to+0x94/0xd8
[   21.804643] [<ffff000008da2e7c>] __schedule+0x19c/0x5e8
[   21.809763] [<ffff000008da3300>] schedule+0x38/0xa0
[   21.814552] [<ffff000008da674c>] schedule_timeout+0x74/0x320
[   21.820098] [<ffff00000812c79c>] rcu_gp_kthread+0x4dc/0x790
[   21.825560] [<ffff0000080edaac>] kthread+0x12c/0x130
[   21.830428] [<ffff000008084ed8>] ret_from_fork+0x10/0x18
Call trace 2. swapper/0的调用堆栈

代码: 全选

[   21.838825] swapper/0       R  running task        0     1      0 0x00000022
[   21.845722] Call trace:
[   21.848161] [<ffff000008089c48>] dump_backtrace+0x0/0x3c8
[   21.853455] [<ffff00000808a024>] show_stack+0x14/0x20
[   21.858410] [<ffff0000080f8c2c>] sched_show_task+0x14c/0x180
[   21.863956] [<ffff0000080f9c38>] dump_cpu_task+0x40/0x50
[   21.869162] [<ffff00000812e948>] rcu_dump_cpu_stacks+0x94/0xd4
[   21.874873] [<ffff00000812d78c>] rcu_check_callbacks+0x64c/0x958
[   21.880758] [<ffff000008132564>] update_process_times+0x2c/0x58
[   21.886551] [<ffff000008141950>] tick_sched_handle.isra.4+0x30/0x50
[   21.892681] [<ffff0000081419b0>] tick_sched_timer+0x40/0x90
[   21.898142] [<ffff0000081331e0>] __hrtimer_run_queues+0xe8/0x168
[   21.904023] [<ffff000008133460>] hrtimer_interrupt+0xa8/0x230
[   21.909653] [<ffff000008ac85b8>] arch_timer_handler_virt+0x28/0x48
[   21.915698] [<ffff000008120400>] handle_percpu_devid_irq+0x80/0x138
[   21.921830] [<ffff00000811ac1c>] generic_handle_irq+0x24/0x38
[   21.927458] [<ffff00000811b29c>] __handle_domain_irq+0x5c/0xb8
[   21.933168] [<ffff000008081960>] gic_handle_irq+0x78/0x174

Exception stack 1. 这个异常堆栈应该是处理完irq返回

代码: 全选

[   21.938544] Exception stack(0xffff000008003d90 to 0xffff000008003ed0)
[   22.026074] [<ffff000008083230>] el1_irq+0xb0/0x124
[   22.030860] [<ffff000008081b0c>] __do_softirq+0xac/0x228
[   22.036071] [<ffff0000080d35c8>] irq_exit+0xc8/0x100
[   22.040941] [<ffff00000811b2a0>] __handle_domain_irq+0x60/0xb8
[   22.046654] [<ffff000008081960>] gic_handle_irq+0x78/0x174
Exception stack 2. 这个堆栈应该是内核初始化的流程调用函数过程,不知道为什么最后压入el1_irq

代码: 全选

[   22.052028] Exception stack(0xffff00000802b530 to 0xffff00000802b670)
[   22.139556] [<ffff000008083230>] el1_irq+0xb0/0x124
[   22.144350] [<ffff0000081f1c90>] new_slab+0x58/0x538
[   22.149219] [<ffff0000081f43c4>] ___slab_alloc+0x474/0x5e8
[   22.154594] [<ffff0000081f455c>] __slab_alloc.isra.23+0x24/0x38
[   22.160393] [<ffff0000081f4dec>] kmem_cache_alloc+0x1b4/0x1f8
[   22.166018] [<ffff000008291a84>] __kernfs_new_node+0x54/0x178
[   22.171646] [<ffff000008292d34>] kernfs_new_node+0x24/0x48
[   22.177021] [<ffff000008294b88>] __kernfs_create_file+0x38/0xe0
[   22.182817] [<ffff0000082955d0>] sysfs_add_file_mode_ns+0x98/0x1a8
[   22.188865] [<ffff000008296470>] internal_create_group+0xe8/0x2e0
[   22.194829] [<ffff0000082967f0>] sysfs_create_groups+0x48/0xa8
[   22.200544] [<ffff0000086d3120>] device_add_groups+0x10/0x18
[   22.206088] [<ffff0000086d7d14>] bus_add_device+0x3c/0x138
[   22.211462] [<ffff0000086d5e24>] device_add+0x354/0x5b8
[   22.216590] [<ffff00000863d71c>] serdev_controller_add+0x24/0x160
[   22.222550] [<ffff00000863df00>] serdev_tty_port_register+0x90/0x100
[   22.228768] [<ffff0000085fac04>] tty_port_register_device_attr_serdev+0x3c/0x88
[   22.235916] [<ffff0000086169a0>] uart_add_one_port+0x208/0x4c8
[   22.241629] [<ffff00000862e280>] serial_imx_probe+0x420/0x618
[   22.247255] [<ffff0000086da948>] platform_drv_probe+0x58/0xb8
[   22.252884] [<ffff0000086d8d28>] driver_probe_device+0x210/0x2d0
[   22.258762] [<ffff0000086d8ea4>] __driver_attach+0xbc/0xc0
[   22.264138] [<ffff0000086d6e84>] bus_for_each_dev+0x4c/0x98
[   22.269601] [<ffff0000086d8640>] driver_attach+0x20/0x28
[   22.274809] [<ffff0000086d8190>] bus_add_driver+0x1b8/0x228
[   22.280268] [<ffff0000086d9840>] driver_register+0x60/0xf8
[   22.285643] [<ffff0000086da898>] __platform_driver_register+0x40/0x48
[   22.291945] [<ffff0000093fcf28>] imx_serial_init+0x38/0x5c
[   22.297319] [<ffff000008083c90>] do_one_initcall+0x38/0x128
[   22.302779] [<ffff0000093b0d0c>] kernel_init_freeable+0x188/0x22c
[   22.308742] [<ffff000008da15b8>] kernel_init+0x10/0x108
[   22.313866] [<ffff000008084ed8>] ret_from_fork+0x10/0x18

因为个人的经验不是很足,完全分析不出来哪里除了问题,所有请大家帮忙看下,我的问题是:

1. Call trace和 Exception stack的区别,我的理解是call trace是正常进程调用的流程, Exception stack是出现异常的堆栈
2. Exception stack 2 可以看出是内核初始化的调用流程,而且出错应该和串口有关,但是不知道为什么el1_irq会压入堆栈,因为el1_irq是异常向量表,由硬件触发的,不是调用的。
3. 这个可否看出是哪里出了问题?


完整的启动日志放在楼下
chengshuyi
帖子: 20
注册时间: 2018-10-18 10:18
系统: ubuntu16

Re: 求助,启动的时候出现rcu_preempt self-detected stall on cpu

#2

帖子 chengshuyi » 2019-06-25 11:15

字符超过6000了,我放个链接 https://paste.ubuntu.com/p/5stdGcQZDS/
回复