[BUG][CFL]ipc timed out for 0x50020000 size 108 after system boot up

[YvonneYang2]

Describe the bug
sudo reboot to do system reboot, ipc timed out for 0x50020000 size 108 after system boot up.

To Reproduce
sudo reboot to do system reboot

Expected behavior
Bootup without any issues.

Impact
ipc tx: 0x50020000: GLB_COMP_MSG: GET_VALUE
IPC timed out

Reproduce rate
2/10

Test recipe
linux (topic/sof-dev): 7271e69
sof (master): b73b9a6
topology: sof-hda-generic.tplg
Platform: CFL laptop with ALC3254 codec

dmesg

[   50.689651] sof-audio-pci 0000:00:1f.3: ipc tx: 0x50020000: GLB_COMP_MSG: GET_VALUE
[   50.992745] sof-audio-pci 0000:00:1f.3: error: ipc timed out for 0x50020000 size 108
[   50.992767] sof-audio-pci 0000:00:1f.3: status: fw entered - code 00000005
[   50.993033] sof-audio-pci 0000:00:1f.3: error: unexpected fault 0x00000000 trace 0x00004000
[   50.993045] sof-audio-pci 0000:00:1f.3: error: host status 0x80000000 dsp status 0x00000000 mask 0x00000003
[   50.993049] sof-audio-pci 0000:00:1f.3: error: waking up any trace sleepers
[   50.993054] sof-audio-pci 0000:00:1f.3: error: set/get ctrl ipc comp 2
[   50.993060] sof-audio-pci 0000:00:1f.3: error: volume get failed to idle -16
[   50.993128] sof-audio-pci 0000:00:1f.3: ipc tx: 0x50020000: GLB_COMP_MSG: GET_VALUE
[   51.296938] sof-audio-pci 0000:00:1f.3: error: ipc timed out for 0x50020000 size 108
[   51.296956] sof-audio-pci 0000:00:1f.3: status: fw entered - code 00000005
[   51.297221] sof-audio-pci 0000:00:1f.3: error: unexpected fault 0x00000000 trace 0x00004000
[   51.297233] sof-audio-pci 0000:00:1f.3: error: host status 0x80000000 dsp status 0x00000000 mask 0x00000003
[   51.297237] sof-audio-pci 0000:00:1f.3: error: waking up any trace sleepers
[   51.297242] sof-audio-pci 0000:00:1f.3: error: set/get ctrl ipc comp 61
[   51.297248] sof-audio-pci 0000:00:1f.3: error: volume get failed to idle -16
[   51.297313] sof-audio-pci 0000:00:1f.3: ipc tx: 0x50020000: GLB_COMP_MSG: GET_VALUE
[   51.601084] sof-audio-pci 0000:00:1f.3: error: ipc timed out for 0x50020000 size 108
[   51.601102] sof-audio-pci 0000:00:1f.3: status: fw entered - code 00000005
[   51.601396] sof-audio-pci 0000:00:1f.3: error: unexpected fault 0x00000000 trace 0x00004000
[   51.601418] sof-audio-pci 0000:00:1f.3: error: host status 0x80000000 dsp status 0x90020000 mask 0x00000003
[   51.601422] sof-audio-pci 0000:00:1f.3: error: waking up any trace sleepers
[   51.601426] sof-audio-pci 0000:00:1f.3: error: set/get ctrl ipc comp 67
[   51.601433] sof-audio-pci 0000:00:1f.3: error: volume get failed to idle -16

boot.log
boot_dmesg.log

[michalgrodzicki]

@YvonneYang2 please provide detailed reproduce step.

[lgirdwood]

dmesg shows DSP has completed IPC.

host status 0x80000000 dsp status 0x00000000 mask 0x00000003

[YvonneYang2]

Reproduced steps:

1. Update the test recipe on the platform, do system reboot,check the dmesg.
2. Do aplay or arecord, check the dmesg.
3. Then do system reboot, dmesg shows this ipc-timeout issue

The ipc timeout issue only shows twice during my yesterday test.

[lgirdwood]

@YvonneYang2 can you redo this with trace disabled. @mengdonglin can advise on disabling trace.

[YvonneYang2]

Test total 20 times after disabled trace. Issue not reproduced.

Environment
Firmware: 3e423f4 https://github.com/thesofproject/sof/tree/master
Kernel: e3ec7cf https://github.com/thesofproject/linux/commits/topic/sof-dev
Topology: file: sof-hda-generic-2ch.tplg same as firmware
platform ：CFL

[ranj063]

@YvonneYang2 this test does not include @lyakh's patch for fixing the race 042d6db.

Can you please re-test this with the latest sof-dev.

[wenqingfu]

mark as fixed by thesofproject/linux#943. please reopen if you reproduce this again.