Skip to content

i2c: npcm: disable interrupt enable bit before devm_request_irq#433

Merged
maxdog988 merged 1 commit into
Nuvoton-Israel:NPCM-6.1-OpenBMCfrom
warp5tw:NPCM-6.1-OpenBMC
Jun 26, 2024
Merged

i2c: npcm: disable interrupt enable bit before devm_request_irq#433
maxdog988 merged 1 commit into
Nuvoton-Israel:NPCM-6.1-OpenBMCfrom
warp5tw:NPCM-6.1-OpenBMC

Conversation

@warp5tw
Copy link
Copy Markdown
Collaborator

@warp5tw warp5tw commented Jun 26, 2024

The customer reports that there is a soft lockup issue related to the i2c driver. After checking, the i2c module was doing a tx transfer and the bmc machine reboots in the middle of the i2c transaction, the i2c module keeps the status without being reset.

Due to such an i2c module status, the i2c irq handler keeps getting triggered since the i2c irq handler is registered in the kernel booting process after the bmc machine is doing a warm rebooting.
The continuous triggering is stopped by the soft lockup watchdog timer.

Disable the interrupt enable bit in the i2c module before calling devm_request_irq to fix this issue since the i2c relative status bit is read-only.

Here is the soft lockup log.
[ 28.176395] watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [swapper/0:1]
[ 28.183351] Modules linked in:
[ 28.186407] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.120-yocto-s-dirty-bbebc78 #1
[ 28.201174] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 28.208128] pc : __do_softirq+0xb0/0x368
[ 28.212055] lr : __do_softirq+0x70/0x368
[ 28.215972] sp : ffffff8035ebca00
[ 28.219278] x29: ffffff8035ebca00 x28: 0000000000000002 x27: ffffff80071a3780
[ 28.226412] x26: ffffffc008bdc000 x25: ffffffc008bcc640 x24: ffffffc008be50c0
[ 28.233546] x23: ffffffc00800200c x22: 0000000000000000 x21: 000000000000001b
[ 28.240679] x20: 0000000000000000 x19: ffffff80001c3200 x18: ffffffffffffffff
[ 28.247812] x17: ffffffc02d2e0000 x16: ffffff8035eb8b40 x15: 00001e8480000000
[ 28.254945] x14: 02c3647e37dbfcb6 x13: 02c364f2ab14200c x12: 0000000002c364f2
[ 28.262078] x11: 00000000fa83b2da x10: 000000000000b67e x9 : ffffffc008010250
[ 28.269211] x8 : 000000009d983d00 x7 : 7fffffffffffffff x6 : 0000036d74732434
[ 28.276344] x5 : 00ffffffffffffff x4 : 0000000000000015 x3 : 0000000000000198
[ 28.283476] x2 : ffffffc02d2e0000 x1 : 00000000000000e0 x0 : ffffffc008bdcb40
[ 28.290611] Call trace:
[ 28.293052] __do_softirq+0xb0/0x368
[ 28.296625] __irq_exit_rcu+0xe0/0x100
[ 28.300374] irq_exit+0x14/0x20
[ 28.303513] handle_domain_irq+0x68/0x90
[ 28.307440] gic_handle_irq+0x78/0xb0
[ 28.311098] call_on_irq_stack+0x20/0x38
[ 28.315019] do_interrupt_handler+0x54/0x5c
[ 28.319199] el1_interrupt+0x2c/0x4c
[ 28.322777] el1h_64_irq_handler+0x14/0x20
[ 28.326872] el1h_64_irq+0x74/0x78
[ 28.330269] __setup_irq+0x454/0x780
[ 28.333841] request_threaded_irq+0xd0/0x1b4
[ 28.338107] devm_request_threaded_irq+0x84/0x100
[ 28.342809] npcm_i2c_probe_bus+0x188/0x3d0
[ 28.346990] platform_probe+0x6c/0xc4
[ 28.350653] really_probe+0xcc/0x45c
[ 28.354227] __driver_probe_device+0x8c/0x160
[ 28.358578] driver_probe_device+0x44/0xe0
[ 28.362670] __driver_attach+0x124/0x1d0
[ 28.366589] bus_for_each_dev+0x7c/0xe0
[ 28.370426] driver_attach+0x28/0x30
[ 28.373997] bus_add_driver+0x124/0x240
[ 28.377830] driver_register+0x7c/0x124
[ 28.381662] __platform_driver_register+0x2c/0x34
[ 28.386362] npcm_i2c_init+0x3c/0x5c
[ 28.389937] do_one_initcall+0x74/0x230
[ 28.393768] kernel_init_freeable+0x24c/0x2b4
[ 28.398126] kernel_init+0x28/0x130
[ 28.401614] ret_from_fork+0x10/0x20
[ 28.405189] Kernel panic - not syncing: softlockup: hung tasks
[ 28.411011] SMP: stopping secondary CPUs
[ 28.414933] Kernel Offset: disabled
[ 28.418412] CPU features: 0x00000000,00000802
[ 28.427644] Rebooting in 20 seconds..

The customer reports that there is a soft lockup issue related to
the i2c driver. After checking, the i2c module was doing a tx transfer
and the bmc machine reboots in the middle of the i2c transaction, the i2c
module keeps the status without being reset.

Due to such an i2c module status, the i2c irq handler keeps getting triggered
since the i2c irq handler is registered in the kernel booting process
after the bmc machine is doing a warm rebooting.
The continuous triggering is stopped by the soft lockup watchdog timer.

Disable the interrupt enable bit in the i2c module before calling
devm_request_irq to fix this issue since the i2c relative status bit
is read-only.

Here is the soft lockup log.
[   28.176395] watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [swapper/0:1]
[   28.183351] Modules linked in:
[   28.186407] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.120-yocto-s-dirty-bbebc78 Nuvoton-Israel#1
[   28.201174] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   28.208128] pc : __do_softirq+0xb0/0x368
[   28.212055] lr : __do_softirq+0x70/0x368
[   28.215972] sp : ffffff8035ebca00
[   28.219278] x29: ffffff8035ebca00 x28: 0000000000000002 x27: ffffff80071a3780
[   28.226412] x26: ffffffc008bdc000 x25: ffffffc008bcc640 x24: ffffffc008be50c0
[   28.233546] x23: ffffffc00800200c x22: 0000000000000000 x21: 000000000000001b
[   28.240679] x20: 0000000000000000 x19: ffffff80001c3200 x18: ffffffffffffffff
[   28.247812] x17: ffffffc02d2e0000 x16: ffffff8035eb8b40 x15: 00001e8480000000
[   28.254945] x14: 02c3647e37dbfcb6 x13: 02c364f2ab14200c x12: 0000000002c364f2
[   28.262078] x11: 00000000fa83b2da x10: 000000000000b67e x9 : ffffffc008010250
[   28.269211] x8 : 000000009d983d00 x7 : 7fffffffffffffff x6 : 0000036d74732434
[   28.276344] x5 : 00ffffffffffffff x4 : 0000000000000015 x3 : 0000000000000198
[   28.283476] x2 : ffffffc02d2e0000 x1 : 00000000000000e0 x0 : ffffffc008bdcb40
[   28.290611] Call trace:
[   28.293052]  __do_softirq+0xb0/0x368
[   28.296625]  __irq_exit_rcu+0xe0/0x100
[   28.300374]  irq_exit+0x14/0x20
[   28.303513]  handle_domain_irq+0x68/0x90
[   28.307440]  gic_handle_irq+0x78/0xb0
[   28.311098]  call_on_irq_stack+0x20/0x38
[   28.315019]  do_interrupt_handler+0x54/0x5c
[   28.319199]  el1_interrupt+0x2c/0x4c
[   28.322777]  el1h_64_irq_handler+0x14/0x20
[   28.326872]  el1h_64_irq+0x74/0x78
[   28.330269]  __setup_irq+0x454/0x780
[   28.333841]  request_threaded_irq+0xd0/0x1b4
[   28.338107]  devm_request_threaded_irq+0x84/0x100
[   28.342809]  npcm_i2c_probe_bus+0x188/0x3d0
[   28.346990]  platform_probe+0x6c/0xc4
[   28.350653]  really_probe+0xcc/0x45c
[   28.354227]  __driver_probe_device+0x8c/0x160
[   28.358578]  driver_probe_device+0x44/0xe0
[   28.362670]  __driver_attach+0x124/0x1d0
[   28.366589]  bus_for_each_dev+0x7c/0xe0
[   28.370426]  driver_attach+0x28/0x30
[   28.373997]  bus_add_driver+0x124/0x240
[   28.377830]  driver_register+0x7c/0x124
[   28.381662]  __platform_driver_register+0x2c/0x34
[   28.386362]  npcm_i2c_init+0x3c/0x5c
[   28.389937]  do_one_initcall+0x74/0x230
[   28.393768]  kernel_init_freeable+0x24c/0x2b4
[   28.398126]  kernel_init+0x28/0x130
[   28.401614]  ret_from_fork+0x10/0x20
[   28.405189] Kernel panic - not syncing: softlockup: hung tasks
[   28.411011] SMP: stopping secondary CPUs
[   28.414933] Kernel Offset: disabled
[   28.418412] CPU features: 0x00000000,00000802
[   28.427644] Rebooting in 20 seconds..

Signed-off-by: Tyrone Ting <kfting@nuvoton.com>
@maxdog988 maxdog988 merged commit 7617e2d into Nuvoton-Israel:NPCM-6.1-OpenBMC Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants