[BUG] DSP panic when testing signal stop/start on DMIC pipeline #6020
Description
Describe the bug
This is a sof regression and happens on a lot of our models(Please refer to our daily test:http://sof-ci.sh.intel.com/#/result/planresultdetail/14058).
The bisecting result points to:
9d9ba73
To Reproduce
~/sof-test/test-case/check-signal-stop-start.sh -m capture -c 50- This issue can also be reproduced with
~/sof-test/test-case/multiple-pipeline.sh -f c -l 50on some platforms.
Reproduction Rate
100%
dmesg
[ 3050.891163] kernel: snd_sof:ipc3_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x60040000: GLB_STREAM_MSG: TRIG_START
[ 3050.893324] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[ 3050.893335] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: DSP panic!
[ 3050.893340] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_BOOT_COMPLETE (6)
[ 3050.893366] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x00000005: module: ROM, state: FW_ENTERED, running
[ 3050.893393] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: status code: 0xdead006 (unknown)
[ 3050.893537] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: reason: runtime exception (0x6)
[ 3050.893542] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: trace point: 0x00004000
[ 3050.893547] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: panic at :0
[ 3050.893552] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: DSP Firmware Oops
[ 3050.893556] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: Exception Cause: InstrPIFDataErrorCause, PIF data error during instruction fetch
[ 3050.893560] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EXCCAUSE 0x0000000c EXCVADDR 0x00000000 PS 0x00060725 SAR 0x00000000
[ 3050.893568] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC1 0xbe02e197 EPC2 0xbe02d612 EPC3 0x00000000 EPC4 0x00000000
[ 3050.893575] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPC5 0x00000000 EPC6 0x00000000 EPC7 0x00000000 DEPC 0x00000000
[ 3050.893580] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS2 0x00060120 EPS3 0x00000000 EPS4 0x00000000 EPS5 0x00000000
[ 3050.893586] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: EPS6 0x00000000 EPS7 0x00000000 INTENABL 0x00000000 INTERRU 0x00000222
[ 3050.893592] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: stack dump from 0xbe0a5c90
[ 3050.893600] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5c90: 00000000 00000000 00000000 00000000
[ 3050.893607] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5ca0: fe057ce8 be0a5ce0 be0a5d10 0000000c
[ 3050.893612] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5cb0: be0a5c90 00000000 00000000 00000000
[ 3050.893617] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5cc0: 00000000 00000000 00000000 00000000
[ 3050.893622] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5cd0: 40000000 be0a5d10 9e0de100 00000000
[ 3050.893626] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5ce0: be02e197 00000000 00000000 00000000
[ 3050.893631] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5cf0: be0a60a0 00000000 be046c10 be0b0140
[ 3050.893636] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0xbe0a5d00: be037678 be0a5f10 be0a6070 c0000000
[ 3050.893640] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[ 3050.893645] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state change: 6 -> 7
[ 3050.893796] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: trace IO error
[ 3051.398335] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: trigger stream 6 dir 1 cmd 0
[ 3051.398555] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x60050000 failed: -19
[ 3051.398593] kernel: snd_sof_intel_hda_common:hda_dsp_stream_trigger: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0xa0]=0x20240000 successful
[ 3051.398620] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at soc_component_trigger on 0000:00:1f.3: -19
[ 3051.398628] kernel: DMIC: ASoC: trigger FE cmd: 0 failed: -19
[ 3051.398727] kernel: snd_sof:sof_pcm_hw_free: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: free stream 6 dir 1
[ 3051.398735] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x60030000 failed: -19
[ 3051.398902] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30020000 failed: -19
[ 3051.398908] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget DMIC0.IN
[ 3051.398916] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30210000 failed: -19
[ 3051.398921] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget BUF10.2
[ 3051.398929] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30020000 failed: -19
[ 3051.398933] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget EQIIR10.0
[ 3051.398941] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30210000 failed: -19
[ 3051.398946] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget BUF10.1
[ 3051.398953] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30020000 failed: -19
[ 3051.398959] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget Dmic0
[ 3051.398964] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30210000 failed: -19
[ 3051.398969] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget BUF10.0
[ 3051.398974] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30020000 failed: -19
[ 3051.398978] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget PCM6C
[ 3051.398983] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc3_tx_msg_unlocked: ipc message send for 0x30110000 failed: -19
[ 3051.398987] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: failed to free widget PIPELINE.10.DMIC0.IN
[ 3051.398992] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Failed to free connected widgets
[ 3051.399003] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at snd_soc_pcm_component_hw_free on 0000:00:1f.3: -19
[ 3051.399620] kernel: snd_sof:sof_pcm_close: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: close stream 6 dir 1
[ 3051.416590] kernel: snd_sof:sof_pcm_open: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: open stream 7 dir 1
[ 3051.416593] kernel: snd_sof:sof_pcm_open: sof-audio-pci-intel-tgl 0000:00:1f.3: period min 192 max 16384 bytes
[ 3051.416595] kernel: snd_sof:sof_pcm_open: sof-audio-pci-intel-tgl 0000:00:1f.3: period count 2 max 16
[ 3051.416596] kernel: snd_sof:sof_pcm_open: sof-audio-pci-intel-tgl 0000:00:1f.3: buffer max 65536 bytes
[ 3051.416693] kernel: snd_sof:sof_pcm_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: hw params stream 7 dir 1
[ 3051.416716] kernel: snd_sof_intel_hda_common:hda_dsp_stream_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0xa0]=0x40000 successful
[ 3051.416735] kernel: snd_sof_intel_hda_common:hda_dsp_stream_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0xa0]=0x40000 successful
[ 3051.416738] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: period_bytes:0x3e80
[ 3051.416739] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: periods:4
[ 3051.416787] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Failed to setup widget PIPELINE.11.DMIC1.IN
[ 3051.416788] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Failed to set up connected widgets
[ 3051.416790] kernel: snd_sof:sof_ipc3_pcm_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: stream_tag 2
[ 3051.416792] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: HW params ipc failed for stream 2
[ 3051.416793] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at snd_soc_pcm_component_hw_params on 0000:00:1f.3: -19
[ 3051.416813] kernel: DMIC16kHz: ASoC: __soc_pcm_hw_params() failed (-19)
[ 3051.416819] kernel: DMIC16kHz: ASoC: dpcm_fe_dai_hw_params failed (-19)
[ 3051.416824] kernel: snd_sof:sof_pcm_hw_free: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: free stream 7 dir 1
[ 3051.417019] kernel: snd_sof:sof_pcm_close: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: close stream 7 dir 1
[ 3053.445995] kernel: snd_sof_intel_hda_common:hda_dsp_core_reset_enter: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x4]=0x9010f0f successful
[ 3053.446030] kernel: snd_sof_intel_hda_common:hda_dsp_core_power_down: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x4]=0xf0f successful
[ 3053.446053] kernel: snd_sof_intel_hda_common:hda_dsp_core_is_enabled: sof-audio-pci-intel-tgl 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask 1
[ 3053.446163] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000010 at 00000044
[ 3053.446190] kernel: snd_sof_intel_hda:hda_codec_i915_display_power: sof-audio-pci-intel-tgl 0000:00:1f.3: Turning i915 HDAC power 0
[ 3053.446199] kernel: snd_sof_intel_hda_common:hda_dsp_state_log: sof-audio-pci-intel-tgl 0000:00:1f.3: Current DSP power state: D3
[ 3053.539154] kernel: snd_sof_intel_hda:hda_codec_i915_display_power: sof-audio-pci-intel-tgl 0000:00:1f.3: Turning i915 HDAC power 1
[ 3053.543919] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000000 at 00000044
[ 3053.546840] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000040 at 00000048
[ 3053.546868] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIW: 00000000 at 00000048
[ 3053.550086] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000000 at 00000048
[ 3053.550114] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIW: 00000040 at 00000048
[ 3053.551951] kernel: snd_sof_intel_hda:hda_codec_i915_display_power: sof-audio-pci-intel-tgl 0000:00:1f.3: Turning i915 HDAC power 0
[ 3053.551989] kernel: snd_sof_intel_hda_common:hda_dsp_state_log: sof-audio-pci-intel-tgl 0000:00:1f.3: Current DSP power state: D0I0
[ 3053.551995] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state change: 7 -> 1
[ 3053.551999] kernel: snd_sof:snd_sof_load_firmware: sof-audio-pci-intel-tgl 0000:00:1f.3: loading firmware
[ 3053.552003] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state change: 1 -> 2
[ 3053.552030] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000002 at 00000048
[ 3053.552053] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIW: 00000000 at 00000048
[ 3053.552077] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIR: 00000000 at 00000044
[ 3053.552096] kernel: snd_sof:snd_sof_pci_update_bits_unlocked: sof-audio-pci-intel-tgl 0000:00:1f.3: Debug PCIW: 00000004 at 00000044
[ 3053.552099] kernel: snd_sof:snd_sof_run_firmware: sof-audio-pci-intel-tgl 0000:00:1f.3: booting DSP firmware
[ 3053.552245] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: period_bytes:0x0
[ 3053.552249] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: periods:1
[ 3053.552405] kernel: snd_sof_intel_hda_common:hda_dsp_stream_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x160]=0x40000 successful
[ 3053.552436] kernel: snd_sof_intel_hda_common:hda_dsp_stream_hw_params: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x160]=0x40000 successful
[ 3053.552448] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: period_bytes:0x0
[ 3053.552451] kernel: snd_sof_intel_hda_common:hda_dsp_stream_setup_bdl: sof-audio-pci-intel-tgl 0000:00:1f.3: periods:1
[ 3053.552585] kernel: snd_sof_intel_hda_common:hda_dsp_cl_boot_firmware: sof-audio-pci-intel-tgl 0000:00:1f.3: Attempting iteration 0 of Core En/ROM load...
[ 3053.552600] kernel: snd_sof_intel_hda_common:hda_dsp_core_power_up: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x4]=0xf010f0f successful
[ 3053.552627] kernel: snd_sof_intel_hda_common:hda_dsp_core_reset_leave: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x4]=0xf010f0e successful
[ 3053.552638] kernel: snd_sof_intel_hda_common:hda_dsp_core_run: sof-audio-pci-intel-tgl 0000:00:1f.3: unstall/run core: core_mask = 1
[ 3053.552651] kernel: snd_sof_intel_hda_common:hda_dsp_core_is_enabled: sof-audio-pci-intel-tgl 0000:00:1f.3: DSP core(s) enabled? 1 : core_mask 1
[ 3053.553342] kernel: snd_sof_intel_hda_common:cl_dsp_init: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0xd4]=0x80000000 successful
[ 3053.611686] kernel: snd_sof_intel_hda_common:cl_dsp_init: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x80000]=0x5000001 successful
[ 3056.611681] kernel: snd_sof_intel_hda_common:hda_cl_copy_fw: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x80000]=0x5000001 timedout
[ 3056.611692] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: hda_cl_copy_fw: timeout with rom_status_reg (0x80000) read
[ 3056.612347] kernel: snd_sof_intel_hda_common:hda_dsp_stream_trigger: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x160]=0x140002 timedout
[ 3056.612361] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: hda_dsp_stream_trigger: cmd 0 on -- (Playback, stream_tag: 1): timeout on STREAM_SD_OFFSET read
[ 3056.612366] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: DMA trigger stop failed
[ 3056.612370] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[ 3056.612372] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware download failed
[ 3056.612374] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_BOOT_IN_PROGRESS (2)
[ 3056.612385] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x05000001: module: ROM, state: INIT_DONE, waiting for: DMA_BUFFER_FULL, running
[ 3056.612412] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: extended rom status: 0x5000001 0x0 0x0 0x0 0x0 0x0 0x2560521 0x0
[ 3056.612414] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[ 3056.612457] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Failed to start DSP
[ 3056.612460] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to boot DSP firmware after resume -110
[ 3056.612464] kernel: snd_sof:sof_set_fw_state: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state change: 2 -> 3
[ 3056.612479] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: enabling device failed: -110
[ 3056.612482] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Filter update failed: -110
[ 3056.641881] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: enabling device failed: -22
[ 3056.641883] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Filter update failed: -22