Tests not belonging to the changed extension are run

[jiasli]

In PR's CIs, we have seen tests not belonging to the changed extension are run.

This is because we use

azure-cli-extensions/azure-pipelines.yml

Line 118 in b93f528

ADO_PULL_REQUEST_LATEST_COMMIT: $(System.PullRequest.SourceCommitId)

which is then used in git diff:

azure-cli-extensions/scripts/ci/test_source.py

Line 39 in dacf087

cmd_tpl = 'git --no-pager diff --name-only origin/{commit_start} {commit_end} -- {code_dir}'

Using SourceCommitId is wrong, because this is not the commit after merging with main. Changes that are on main but not on SourceCommitId will be identified by git diff, resulting in tests for these changes being run.

Instead, HEAD is the commit of merging SourceCommitId with target branch main.

On the other hand, main repo https://github.com/Azure/azure-cli uses HEAD for git diff, so doesn't have this problem:

https://github.com/Azure/azure-cli/blob/a130dfe12ba3393a4a48afaf833ce0aa4a76c559/.azure-pipelines/templates/automation_test.yml#L46

azdev test --no-exitfirst --repo=./ --src=HEAD --tgt=origin/$(System.PullRequest.TargetBranch) --cli-ci --profile ${{ parameters.profile }} --verbose --series

BTW, System.PullRequest.SourceCommitId is not documented (https://github.com/MicrosoftDocs/azure-devops-docs/issues/4588).

[jiasli]

To verify HEAD is the merged commit, simply checkout 2675aeab8b20ab818765b5518d99c4200d94528e as shown in

https://dev.azure.com/azure-sdk/public/_build/results?buildId=1457077&view=results

https://dev.azure.com/azure-sdk/public/_build/results?buildId=1457077&view=logs&j=5ef8dbec-7f7c-5b30-10e0-52c3d859343c&t=98abb23f-96ba-5562-ab7b-869ac2f9fa03&l=33

git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin --depth=1 +2675aeab8b20ab818765b5518d99c4200d94528e:refs/remotes/origin/2675aeab8b20ab818765b5518d99c4200d94528e

and check the log:

$ git fetch origin 2675aeab8b20ab818765b5518d99c4200d94528e
$ git checkout 2675aeab8b20ab818765b5518d99c4200d94528e
$ git log --graph --pretty=oneline
*   2675aeab8b20ab818765b5518d99c4200d94528e (HEAD) Merge 1cfe15b72a4aaebf7971cdf8e3bffdeb28cb7fdf into b401e2c4ecda4966957bc2b3179d45a2aaaea6f6
|\
| * 1cfe15b72a4aaebf7971cdf8e3bffdeb28cb7fdf azdev linter
| * 6bd111d0eecbc3d6b481291c4fde222ff065894f fetch
* | b401e2c4ecda4966957bc2b3179d45a2aaaea6f6 (origin/dev, dev) [Packaging] Bump Python image to `3.10.3-alpine3.15` (#21688)
|/
* 37a504f8a1cc7095a842859549da9083466e71a0 26.1.0 (#21761)

[navba-MSFT]

Discussed with @jiasli offline and removing the CXP Attention label from this.