Helix SDK should support artifact directories with unix permissions

[davidwrighton]

• This issue is causing unreasonable pain

The Helix sdk uses Zip files to package up contents when passing a directory into Helix. Unfortunately on Unix platforms this causes issues when there are native executables in the payload. In particular, the permissions on the various files are lost when packaged as a zip file. The helix runtime supports handling a tarball, which would be more suitable than the current Zip file approach.

Recently as part of (dotnet/runtime#42049), I ran into this issue, and while the suggested solution was to write a script which ran chmod to fix the various executables I instead chose to manually launch tar from the msbuild logic that was invoking the sdk, and then used the PayloadArchive feature to send the payload instead of PayloadDirectory. However, looking through our usage of Helix this problem is common, and there are many ad hoc solutions involving chmod. It would be much more reasonable to have some notation that caused the SDK to use tar, and sidestep this entire problem for future infrastructure needs.

[MattGal]

Thanks. Does using tar still solve this problem if the .tar / .tar.gz files are made on a Windows box?

[davidwrighton]

Well, the tar file would need to have the appropriate permissions in it, which aren't likely to happen on a Windows box. I don't think we should remove the current logic which magically puts +x permissions on scripts in the root of the payload, but for problematic scenarios which aren't script based, it is unlikely that helix would be launched from a Windows box in any case. For instance, I would expect this problem to be likely for payloads which have native Linux or OSX binaries, and it is unlikely those will be crosscompiled from a Windows machine and then sent to a non-Windows machine. Given our current infrastructure, and expected future infrastructure, I expect we would be launching Helix from a Unix machine in those cases.

[davidwrighton]

It also seems like it would be worth documenting in the Helix SDK, the PayloadArchive feature, as well as the detail about how permissions are handled, as I was surprised by some of this.

[alexperovich]

I believe we can do this easily. Base .net doesn't have tar support, but we could just use sharpziplib or something. I doubt it would ever work when sending work from windows to linux though.

[davidwrighton]

Another thought is to just shell out to tar, which is available on recent builds of Windows as well as Linux. Feel free to take a quick look at my PR, and in particular the change to src/coreclr/tests/helixpublishwitharcade.proj

Note, I marked this issue as unreasonable pain, not because it was all that painful to solve, just because it was an speedbump I really didn't need, that really didn't need to exist.

[garath]

After looking at this in the Triage meeting, the general feeling is that the value isn't there for the additional complexity and new dependencies that this would require. Improvements in documentation around how to handle this situation would certainly help, and we can look at that.