Skip to content

Conversation

@bebound
Copy link
Contributor

@bebound bebound commented Nov 24, 2023

Description

Close #22462
Release zip package so that CLi can be used without admin privilege.

Difference between MSI build

  • Only keep az.cmd entry script
  • Rename wbin to bin
  • x64 only

Package size

The uncompressed folder is 283MB
zip with deflate algorithm: 90MB
zip with LZMA algorithm: 77MB (Windows File Explorer does not support it)

MSI is packed to a cab file and zipped with LZX algorighm: 67MB.

If I compressed it to 7z, it's only 37MB, which is due to the Solid compression. If I disable solid compression, the size become 72MB.

Unfortunately, zip does not support solid compression. Since it can be opened without installing any software, let's keep using zip.

7z is much faster than Compress-Archive and Expand-Archive

Compress-Archive and Expand-Archive take 5 and 10 minutes respectively. 7z is much faster, completing the task in just several seconds.

Doc is MicrosoftDocs/azure-docs-cli#4045

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Nov 24, 2023

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9

@azure-client-tools-bot-prd
Copy link

Hi @bebound,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Nov 24, 2023

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Nov 24, 2023

Support Windows ZIP package


pushd %BUILDING_DIR%
%BUILDING_DIR%\python.exe %REPO_ROOT%\scripts\compact_aaz.py
if %errorlevel% neq 0 goto ERROR
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the error code of last command.
compact_aaz.py may fails because of #27923

Copy link
Contributor Author

@bebound bebound Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

azps.ps1 and az are not included in zip as they cause other problems. See #26682

msbuild /t:rebuild /p:Configuration=Release /p:Platform=%ARCH% %REPO_ROOT%\build_scripts\windows\azure-cli.wixproj
) else (
echo Building ZIP...
ren %BUILDING_DIR% "Azure CLI"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Azure CLI as root folder of zip package.
PS: The MSI installation path is C:\Program Files\Microsoft SDKs\Azure\CLI2.

) else (
echo Building ZIP...
ren %BUILDING_DIR% "Azure CLI"
"%ProgramFiles%\7-Zip\7z.exe" a -tzip "%OUTPUT_DIR%\Microsoft Azure CLI.zip" "%ZIP_DIR%"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compress-Archive and Expand-Archive take 5 and 10 minutes respectively. 7z is much faster, completing the task in just several seconds.

robocopy %PYTHON_DIR% %BUILDING_DIR% /s /NFL /NDL

set CLI_SRC=%REPO_ROOT%\src
%BUILDING_DIR%\python.exe -m pip install --no-warn-script-location --force-reinstall pycparser==2.18
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pycparser==2.19 is installed later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%BUILDING_DIR%\python.exe -m pip install --no-warn-script-location --requirement %CLI_SRC%\azure-cli\requirements.py3.windows.txt

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also see #27196 (comment)

REM Rename zip root folder to "Azure CLI"
ren %BUILDING_DIR% "Azure CLI"
REM Use LZMA compression to reduce the size of the zip file.
"%ProgramFiles%\7-Zip\7z.exe" a -tzip -m0=LZMA "%OUTPUT_DIR%\Microsoft Azure CLI.zip" "%ZIP_DIR%"
Copy link
Member

@jiasli jiasli Jan 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to 7-Zip's manual C:\Program Files\7-Zip\7-zip.chm, only 7z type supports the {N}={MethodID}[:param1][:param2] ... [:paramN] notation:

image

For Zip, it should be m={MethodID}:

image

Copy link
Contributor Author

@bebound bebound Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right.
According to the doc, -m0 should only works in 7z and I should use -mm.
However, it works.
I've created a bug report for 7z: https://sourceforge.net/p/sevenzip/bugs/2441/

@jiasli
Copy link
Member

jiasli commented Jan 16, 2024

Windows File Explorer doesn't support extracting LZMA-compressed ZIP and fails with 0x80004005:

image

This ZIP file is created by

& "C:\Program Files\7-Zip\7z.exe" a -tzip -mm=LZMA msi_compressed.zip msi

If you use the Windows's built-in "Send to -> Compressed (zipped) folder" to create the ZIP file:

image

You can see it uses Deflate:

& "C:\Program Files\7-Zip\7z.exe" l -slt msi.zip
...
Path = msi\Microsoft Azure CLI.msi
...
Method = Deflate

You can also see this from 7-Zip File Manager:

image

Reference: https://stackoverflow.com/questions/6896487/how-to-determine-compression-method-of-a-zip-rar-file

@jiasli
Copy link
Member

jiasli commented Jan 16, 2024

PowerShell-7.4.1-win-x64.zip downloaded from https://github.com/PowerShell/PowerShell/releases/tag/v7.4.1 also uses Deflate:

& "C:\Program Files\7-Zip\7z.exe" l -slt .\PowerShell-7.4.1-win-x64.zip | Select-Object -First 30
...
Method = Deflate

@bebound
Copy link
Contributor Author

bebound commented Jan 17, 2024

Windows File Explorer doesn't support extracting LZMA-compressed ZIP and fails with 0x80004005:

Well spotted!
Windows File Explorer can list the content of LZMA zip but not able to uncompress it. (That's weird...)
We have to revert to Deflate.

@bebound bebound changed the title {Packaging} Release Windows ZIP package {Packaging} Support Windows ZIP package Jan 17, 2024
mkdir %ARTIFACTS_DIR%
set TEMP_SCRATCH_FOLDER=%ARTIFACTS_DIR%\cli_scratch
set BUILDING_DIR=%ARTIFACTS_DIR%\cli
set ZIP_ROOT_FOLDER="AzureCLI"
Copy link
Member

@jiasli jiasli Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy %REPO_ROOT%\build_scripts\windows\scripts\azps.ps1 %BUILDING_DIR%\wbin\
copy %REPO_ROOT%\build_scripts\windows\scripts\az %BUILDING_DIR%\wbin\
) else (
copy %REPO_ROOT%\build_scripts\windows\scripts\az_zip.cmd %BUILDING_DIR%\wbin\az.cmd
Copy link
Member

@jiasli jiasli Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other Microsoft-owned apps mentioned in #27911 (comment) put their executables such as Code.exe, pwsh.exe under the root folder. Perhaps we can do the same and put az.cmd under the root folder?

But this approach has its own problem: adding Azure CLI's folder to PATH will also put python.exe under PATH. If a user accidentally executes python.exe, that may corrupt Azure CLI's virtual environment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to maintain the current structure to avoid adding other executables into PATH.
PS: Other CLI tools also place their executables to dedicated folder. For example: C:\Program Files\Docker\Docker\resources\bin

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use wbin, instead of bin?

Copy link
Member

@jiasli jiasli Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. The explanation is right in the comment:

echo Creating the wbin (Windows binaries) folder that will be added to the path...
mkdir %BUILDING_DIR%\wbin

It was added by #2655.

I initially thought it means

A Unicode version with the letter "W" used to indicate "wide"
-- https://learn.microsoft.com/en-us/windows/win32/intl/unicode-in-the-windows-api

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this package is for Windows, w seems redundant.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other Microsoft-owned apps mentioned in #27911 (comment) put their executables such as Code.exe, pwsh.exe under the root folder. Perhaps we can do the same and put az.cmd under the root folder?

But this approach has its own problem: adding Azure CLI's folder to PATH will also put python.exe under PATH. If a user accidentally executes python.exe, that may corrupt Azure CLI's virtual environment.

Adding python in the path could also create instability if the platform (for example a custom build agent) is using python for other purposes.

I think that wbin will be confusing and the information it adds is not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dcaro I can change the folder to bin in zip package.
Should we also do this in MSI? We never mention wbin in doc, and it's automatically added into PATH during MSI installation. This change should not affect users unless they use full path to run az.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's leave the folder name in the msi as is for now, as you mention, it is automatically added to path and transparent to users.

copy %REPO_ROOT%\build_scripts\windows\scripts\azps.ps1 %BUILDING_DIR%\wbin\
copy %REPO_ROOT%\build_scripts\windows\scripts\az %BUILDING_DIR%\wbin\
if "%TARGET%"=="msi" (
REM Creating the wbin (Windows binaries) folder that will be added to the path...
Copy link
Contributor Author

@bebound bebound Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I use echo instead of REM, the (Windows binaries) folder will be parsed and folder was unexpected at this time. error will be raised. It seems there are some special rules when parentheses block is executed.
Repro script:

echo () ok
if 'kk'=='kk' (
    echo () ok2
)
() ok
ok2 was unexpected at this time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you truly need to use echo, you may quote the sentence:

echo "Creating the wbin (Windows binaries) folder that will be added to the path..."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. (Generally, echo is able to print multiple word without the quote: https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/echo)

In this instance, since the line functions more as a comment, so I changed it to REM.

@bebound bebound merged commit 17283d1 into Azure:dev Jan 30, 2024
@bebound bebound deleted the zip branch January 30, 2024 06:14
@bebound bebound changed the title {Packaging} Support Windows ZIP package [Packaging] Support Windows ZIP package Jan 30, 2024
@jiasli
Copy link
Member

jiasli commented Feb 6, 2024

Downloaded from https://azcliprod.blob.core.windows.net/zip/azure-cli-2.57.0-x64.zip and the ZIP package works like a charm! 🎉

@jiasli
Copy link
Member

jiasli commented Feb 6, 2024

Another great benefit of the ZIP package is that it allows much easier edge build release.

Currently, in order to install an MSI edge build, I have to

  1. Uninstall the official MSI
  2. Install the edge buld MSI

But with ZIP, I can simply extract it and run bin/az.cmd!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot Packaging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Release Azure CLI as ZIP packages

5 participants