can't get any FW error logs once dsp is powered off

[keyonjie]

once we power off the Audio DSP (tplg load failed or DSP panic or any other causes that will lead to powering off audio dsp cores), we can't get any error logs from sof-logger, this make debugging to this kind of firmware issues(e.g. ipc time out) very difficult, we need to fix this issue ASAP.

[   56.330869] sof-audio-pci 0000:00:0e.0: error : DSP panic!
[   56.330875] sof-audio-pci 0000:00:0e.0: panic: dsp_oops_offset 788480 offset 788480
[   56.330887] sof-audio-pci 0000:00:0e.0: status: fw entered - code 00000005

root@test-apl:~# sof-logger -l sof-apl.ldc
 CORE  LEVEL      COMP_ID                TIMESTAMP            DELTA                FILE_NAME    CONTENT
root@test-apl:~#

[keyonjie]

I believe we have the same issue with topic/sof-dev branch, @ranj063 can you take a look?

[ranj063]

@keyonjie is this really a regression? If I remember correctly @dabekjakub confirmed that he can read the exception debugfs entry in case of a panic

[xiulipan]

@ranj063
checked and try to reproduce a lot failing cases in CI. Let me give some explanation here and later I will a fix for it and have some test later.

kconfig for ENABLE_DEBUGFS_CACHE is set.
Otherwise we can only get more error in demsg

Situation here:

1. No error. Every thing is right.
   etrace is empty as expected.
   trace have logs
2. tplg load error, IPC failed
   etrace is empty, because hda_dsp_core_reset_power_down is called to power memory bank off. memory window is un-readable
   trace is also empty, because of DSP is powered off, no more update can be done to host.
3. panic happen/check on @keyonjie device
   etrace should have some logs with part of dma traces.
   not sure if the cache handled the memory window cache read rightly.
   trace have some old logs

@ranj063 could you share the PR that hda_dsp_core_reset_power_down is enabled for fail cases?
I can see about 5-6 times it is calls when I got some tplg failed.

I need to check why the power off is done so quick in some fail cases.

[ranj063]

@xiulipan DSP is powered off is when topology widgets are unloaded which would be the case when a topology loading error occurs.

[ranj063]

ASoC: topology: free created components in tplg load error

This patch takes care of unloading the component when tplg loading fails.

[keyonjie]

mark it to P1 as it blocks debugging for critical FW issue.

[RanderWang]

I tested on two different Whiskylake today based on latest kernel and FW, and found there is no such issue now. @keyonjie what is your platform ?