[Feature Request] BREAKING change: Protect `namePrefix` value to further mitigate pipeline failures due to existing globally unique names

[eriqua]

Description

Context

Managing the namePrefix token through the settings.json mitigated the risk of losing ownership on the globally unique resource names such as keyvaults.
It turns out that it is not enough since those names keep on being used by others, most likely carml users who forget to change the namePrefix value before running the pipelines. That is causing related pipelines to fail in the CARML testing environment due to resource names being already taken.
We currently need to change those names and further regenerate readmes containing 1:1 module samples to solve corresponding pipelines failures.
This is expected to get worse with the increase of the CARML usage and with the introduction of the new dependencies approach, since also dependencies will be removed and the number of failing pipelines is likely to increase.

Suggestion

• Create a new GH secret/ADO variable to host the namePrefix
• Keep the possibility to manage the token via the settings.json
• Implement a logic to give priority to the settings.json token if set, default to the secret otherwise
• In CARML comment out the namePrefix token in the settings.json and remove our value for the CI environment to leverage the env variable.
• Update the test module locally scripts accordingly
• Update documentation

cc: @ahmadabdalla, @MrMCake

[AlexanderSehr]

I approve this message.

[ahmadabdalla]

Happy to support with this feature 😃

[ahmadabdalla]

@eriqua @MrMCake Thinking about this

• If we default to the file unless a GitHub Secret namePrefix exists, we do not have to comment out the settings and rely on the secret instead.

---

Options

Option 1

Add the namePrefix token as a GitHub secret to be available as an environment variable. Add it on each workflow, and modify the actions to create an exception for the namePrefix token to be processed differently when the local tokens are being searched like the below:

if(($token.name -eq 'namePrefix') -and '${{ env.NAMEPREFIX }}'){
   $token.value = '${{ env.NAMEPREFIX }}'
}

• quickest option but is not scalable as we have to do a similar thing for other use cases (if they ever come up, so maybe if they do we develop option2?)

Option 2

Add the namePrefix token as a GitHub secret to be available as an environment variable. Migrate tokens and config to the global variables YAML, if it gets provided in the YAML, then it supersedes the one coming in from the GitHub Secret. This uses the existing setEnvironmentVariables

• A lot more work involved, and needs proper testing as well

CC: @MariusStorhaug

[ahmadabdalla]

@MariusStorhaug @eriqua @MrMCake need your inputs please on the proposal/options above.

[MariusStorhaug]

I vote option 2, with the added value of achieving what originally intended in the issue (#953) regarding simplifying settings and variables.

[ahmadabdalla]

@MrMCake @eriqua @MariusStorhaug

Just some updates. The following PoC works from a 'pipeline' perspective:

• Removed the validateModuleDeployment dependency on the settings.json and embedded its content into the globalvariables.yml.
• Supported the namePrefix token to be prioritized from a GitHub secret IF provided, otherwise will default to the local one.

Some complications:

• Our global tests Pester depends on the settings.json to perform the tokens parameter file tests (i.e. if someone didn't replace a token in their parameter file these tests detect it). Replacing this mechanism needs more work as we need to rely on the convert YAML to object script to perform a similar function, or need to think of a better way to achieve this.
• Our test module locally wrapper also depends on the settings.json. Similar to the above.

I'm keen to demo the PoC soon and get some feedback.

[MariusStorhaug]

I left my update on the matter in the other issue that Ahmad has (#1363):

#1363 (comment)