scripts: xtensa-build-all.sh: Refactor and improve script

[crazoes]

• Use getopts for simpler, robust argument parsing
• Exit if an unknown platform is specified
• Derive $ROOT from $HOST instead of explicit assignment
• Use switch case instead of if else for testing string

Signed-off-by: Shreeya Patel shreeya.patel23498@gmail.com

[crazoes]

Furthers #2631

[crazoes]

Jenkins probably uses ./scripts/xtensa-build-all.sh -j $PLATFORM as well, although can't seem to find any jenkinsfile in the repo.

[lgirdwood]

SOFCI TEST

[marc-hb]

typo

[dbaluta]

I have also seen this typo, but since we are just now refactoring the old code this can be done in a separate PR. But now that you pointed it I think we can add a separate patch in this PR.
@crazoes Remeber that in a PR like this fixes come first, then the actual refactoring.

[marc-hb]

I like to treat typos the same as indentation/whitespace: don't obscure actual changes with them. However if that line changes ANYWAY in git for some other and good reason then it's a good opportunity to fix typos at the same time.

[crazoes]

@marc-hb @dbaluta I have maintained the order here, fixing first and then moving the code around.

In the process I have noticed that when we order moving of code after fixes, we will always end up with merge conflicts after editing the fix commit. So what is the rationale behind ordering fixes first and refactors later?

[marc-hb]

These are associative arrays in disguise. Maybe we could add a dependency on bash version 4 in the future (not in this PR which is already fixing many other things).

[crazoes]

Yeah, I just checked out associative arrays, this whole thing can be made into a nested associative array, with keys platform names, and values as another associative array containing PLATFORM, ARCH, etc..

[marc-hb]

Using PLATFORMS[@] as an array when it's not is really bad and should be fixed and tested very first thing before refactoring anything[*]. Either make PLATFORMS an actual array and fix all the missing quotes, or make it a string with whitespace. Please not a bit of both any more. I suspect actually making it an array means fewer changed lines. It's a bit overkill because I don't think platform names will ever whitespace in them but it's more consistent with SUPPORTED_PLATFORMS[@].

[*] Moving code that works by coincidence is like moving a large pile of glassware.

[marc-hb]

It's a miracle this works considering PLATFORMS is not an array. This works by combining two bugs that cancel each other out:

• To list each element of the array it should be for platform in "${PLATFORMS[@]}" with double quotes
• but PLATFORMS is not an array but a string with whitespace

https://mywiki.wooledge.org/BashGuide/Arrays
Demo, ./scripts/xtensa-build-all.sh byt ABC apl with this:

--- a/scripts/xtensa-build-all.sh
+++ b/scripts/xtensa-build-all.sh
@@ -90,6 +90,19 @@ else
        done
 fi
 
+# Here's an actual array
+declare -p SUPPORTED_PLATFORMS
+printf 'SPi=%s\n' "${SUPPORTED_PLATFORMS[@]}"
+
+# Not an array, just pretending
+declare -p PLATFORMS
+# yet bash expands this a string with whitespace, amazing
+printf 'Pi=%s\n' "${PLATFORMS[@]}"
+printf 'Pi=%s\n' "${PLATFORMS[@]}"
+
+# now something even weirder
+printf '?#?=%s\n' "${#PLATFORMS[@]}"
+
 # check target platform(s) have been passed in
 if [ ${#PLATFORMS[@]} -eq 0 ];
 then
@@ -97,6 +110,8 @@ then
        exit 1
 fi
 
+exit 0
+
 if [ "x$USE_PRIVATE_KEY" == "xyes" ]

[crazoes]

I don't know why but https://github.com/thesofproject/sof/pull/2711/commits doesn't show the right order of commit, rather ordered by dates. Making it difficult to read the actual sequence of the commits. I have checked the reviews again and verified I haven't missed any comment. However, there have been many amending and merge conflicts here so please let me know if I missed anything or if something might have lost due to wrongful merge conflict resolution on my part.

[marc-hb]

I think you meant: if type xtensa-bxt-elf-gcc; then.
This explains the Travis failure.

[crazoes]

Either way HOST is never set to xtensa-bxt-elf-gcc which is what is being set as the compiler. Currently unable to get the docker image because of network issues so cannot reproduce locally. Anyways, I will try removing the ! and see if it works

  The CMAKE_C_COMPILER:

    xtensa-bxt-elf-gcc

  is not a full path and was not found in the PATH.

[crazoes]

huh, this does fix it. But doesn't work well with what I said earlier. Maybe the CMAKE_C_COMPILER is not derived from TOOLCHAIN which is what is using the HOST value?

[marc-hb]

bxt and apl were technically different but have been used somewhat interchangeably in some contexts. If you look at commit c7edd73 you can see this logic is only backward compatibility to support who built a toolchain with the older bxt name. All SOF documentation now refers to apl

[marc-hb]

I don't know why but https://github.com/thesofproject/sof/pull/2711/commits doesn't show the right order of commit, rather ordered by dates. Making it difficult to read the actual sequence of the commits.
I have checked the reviews again and verified I haven't missed any comment. However, there have been many amending and merge conflicts here so please let me know if I missed anything or if something might have lost due to wrongful merge conflict resolution on my part.

Wow, I think you discovered a new github limitation with force pushes, thanks! I added it to the list:
zephyrproject-rtos/zephyr#14444 (comment)

In the process I have noticed that when we order moving of code after fixes, we will always end up with merge conflicts after editing the fix commit. So what is the rationale behind ordering fixes first and refactors later?

Yes, re-ordering commits that touch the same parts of the code will always cause painful conflicts. The rationale is that version history is sometimes painfully written once by one person (you), but then it's immutable and read, bisected, backported and used in many other ways by many other people and many tools. This "clean history" rationale is a very common rationale among git users, apparently much less common among github users, github seems to care less about history.

ordering fixes first and refactors later?

It's not always like this, it depends. In other cases a zero-functional change refactor is a prelude to shorten and clarify a bug fix coming immediately next and making the fix much easier to revert in case it regresses something else. In this PR the hopefully small fix(es) were completely unrelated to the refactor, so better not increase and obscure the distance between the bugs and their fixes by shuffling code in between.

PS: besides root-causing the Travis failure, I haven't reviewed the latest version yet.

[lgirdwood]

SOFCI TEST

[xiulipan]

@crazoes What the change about the -j option? Why you would like to remove it now?

[crazoes]

@crazoes What the change about the -j option? Why you would like to remove it now?

@xiulipan Explained in another discussion here : #2711 (comment)

[xiulipan]

@crazoes I will try to remove the -j option in our Jenkins CI scripts.

[lgirdwood]

@xiulipan do you need to update CI before or after this is merged ?

[lgirdwood]

CI known issues.

[marc-hb]

This PR missed one $j -> $platform replacement:

$ shellcheck scripts/xtensa-build-all.sh 
In scripts/xtensa-build-all.sh line 273:

       case $platform in
       ...

		TOOLCHAIN=$HOST
		PATH=$pwd/../$HOST/bin:$OLDPATH
		COMPILER="gcc"

		case $j in
                     ^-- SC2154: j is referenced but not assigned.

			byt|cht|cnl|sue) DEFCONFIG_PATCH="_gcc";;
			*)	     DEFCONFIG_PATCH="";;
		esac

I think this means the wrong _defconfig files have been used for byt|cht|cnl|sue for the last 3- months?

(unrelated but large shellcheck noise reduced in unrelated PR #3163 )

[crazoes]

@marc-hb you are right. I will send a fix.

(Done in PR #3167)