[BUG] IMX8 QEMU boot failed - qemu-check.sh regression after qemu commit f71c955 in sof-4.2 branch

[fredoh9]

@marc-hb EDIT: this is a QEMU/IMX regression, Docker is only the messenger.
This is blocking any qemu progress like

• scripts/qemu-check.sh: Add qemu check for mt8195 platform #5274

Describe the bug
SOF Docker images were upgraded to Ubuntu 20.04 and proxy-free changes.

• [skip-ci] Remove proxy setting and upgrade to Ubuntu 20.04 for SOF docker images #4172

after the upgrade, IMX QEMU boot failed

Run ./scripts/docker-qemu.sh ../sof.git/scripts/qemu-check.sh ${PLATFORM}
+ '[' -z 2.0 ']'
+ timeout --foreground 2.0 /home/sof/qemu/xtensa-softmmu/qemu-system-xtensa -cpu imx8 -M adsp_imx8 -nographic -kernel /home/sof/sof.git/build_imx8_gcc/src/arch/xtensa/sof-imx8.ri
qemu-system-xtensa: terminating on signal 15 from pid 16 (timeout)
Error: Process completed with exit code 1.

To Reproduce
for new sofqemu docker image : thesofproject/sofqemu:20210511_2
for stable sofqemu docker image : thesofproject/sofqemu:20200422

As I restored back to known stable image, need to use new image to reproduce the problem.

# Use latest sof doker image as this image is great
docker pull thesofproject/sof
docer tag thesofproject/sof sof

# build SOF FWs...

# Use the problematic sofqemu image instead of latest
docker pull thesofproject/sofqemu:20210511_2
docker tag thesofproject/sofqemu:20210511_2 sofqemu

cd SOF_BUILD_ROOT
./scripts/docker-qemu.sh ../sof.git/scripts/qemu-check.sh imx8

Reproduction Rate
10/10, 100%

[marc-hb]

@fredoh9 to save some time extracting them from https://github.com/thesofproject/sof/blob/main/.github/workflows/pull-request.yml and since you just been through them, would you mind adding some reproduction steps to the description, especially the docker commands to switch to the newer image? I mean just from memory, I mean they don't have to be perfectly tested.

Not everyone is familiar with docker and most docker commands take a long time.

cc: @greg-intel

[fredoh9]

This is my first build of sofqemu image with my setup, last known build is more than year ago, I'm not sure this is any environment setup issue. Same recipe with previous ubuntu 18.04 also fails to me. I need to approach step by step. Need to focus the other task now, will resume later today.

After reading build logs, there was multiple complains saying flex and bison. But surprisingly this is not fatal errors. This is not the root-cause either but will be fixed.

make[1]: Leaving directory '/home/sof/qemu/slirp'
...
make[1]: flex: Command not found
         DEP srcpos.c
         BISON dtc-parser.tab.c
make[1]: bison: Command not found
         LEX dtc-lexer.lex.c
make[1]: flex: Command not found
...
         DEP util.c
         LEX convert-dtsv0-lexer.lex.c
make[1]: flex: Command not found
         BISON dtc-parser.tab.c
make[1]: bison: Command not found
         LEX dtc-lexer.lex.c
make[1]: flex: Command not found
        CHK version_gen.h
         LEX convert-dtsv0-lexer.lex.c
make[1]: flex: Command not found
         BISON dtc-parser.tab.c
make[1]: bison: Command not found
         LEX dtc-lexer.lex.c
make[1]: flex: Command not found
...
  GEN     hw/core/trace.c
make[1]: Entering directory '/home/sof/qemu/slirp'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/sof/qemu/slirp'
        CHK version_gen.h
         LEX convert-dtsv0-lexer.lex.c
make[1]: flex: Command not found
         BISON dtc-parser.tab.c
make[1]: bison: Command not found
         LEX dtc-lexer.lex.c
make[1]: flex: Command not found
  CC      tests/qemu-iotests/socket_scm_helper.o
  LINK    tests/qemu-iotests/socket_scm_helper

[fredoh9]

Good news and bad news.

Good news is I'm able to build my own thesofproject/sofqemu:20200422 and IMX QEMU test passes. So I don't have any setup/environment issue.

Bad news is there are about 25 commits after 20200422. I need to do git bisect. I will discuss with @lrgirdwo if I find 'bad' commit.

[fredoh9]

@lrgirdwo From qemu repo, this is 'bad' commit
f71c955 hw: mu: Add mu-write implementation

[fredoh9]

@dbaluta for imx8, looks just return error silently, but imx8x and imx8m throw more logs.

imx8m_error.txt
imx8x_error.txt