AppX deployment fails for really large packages

[WilliamsJason]

Looks like MemoryStream isn't large enough. I tried swapping to a FileStream backed by a temporary file and that also failed, but that may be a server issue. We'll need to get that investigated/fixed before it's worth addressing the MemoryStream fix (FileStream will probably be slower and use up more storage space but may be the easiest solution).

[WilliamsJason]

Just to add a couple more details--MemoryStream fails with an exception about 'stream too long' while the FileStream got an OperationCanceledException much further along in the process.

[david-c-kline]

can you define "really large"? :)

[hpsin]

4GB + aka video games.
You can guess why with a number like that (currently we haven't determined
why, but it's pretty auspicious).
Note that this may be an issue with Device Portal itself (TBD) and also an
issue in browsers.

On Aug 10, 2016 5:07 PM, "David Kline" notifications@github.com wrote:

can you define "really large"? :)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#141 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABltO082SDrSrWHzP8TRkkNdQUPZl5Sfks5qemezgaJpZM4Jgljx
.

[WilliamsJason]

4 GB is probably the magic number for WDP. WDPW will actually choke on a smaller size due to the MemoryStream implementation (possibly 256 MB, maybe as high as 2 GB if resources are plentiful, see this stack overflow thread for more details).

[WilliamsJason]

Just an update here--The underlying WDP issues should all be resolved shortly but the MemoryStream implementation in the wrapper project still needs to be updated.

[dotMorten]

Suggestion: Instead of loading all those files into memory, post an HttpMultipartContent that consists of HttpStreamContent items. It would GREATLY reduce the code in InstallApplicationAsync, and it would also reduce memory consumption to the size of the buffer that's used to stream from the files to the network (which would only be a few kb).
All that code with the boundaries etc are completely overkill, since that is already supported by the HttpContent classes: https://github.com/Microsoft/WindowsDevicePortalWrapper/blob/master/WindowsDevicePortalWrapper/WindowsDevicePortalWrapper.Shared/Core/AppDeployment.cs#L122%2DL170
Could be reduced to

var content = new HttpMultipartContent();
foreach (string dependencyFile in dependencyFileNames)
    content.Add(new HttpStreamContent(File.OpenRead(dependencyFile).AsInputStream()));

[WilliamsJason]

This sounds like exactly the solution I was looking for, thank you! I suspect this will be MUCH more performant as well which I was also concerned about.

I'm not sure when I'll get time to make this change, if anyone else wants to make it feel free and I'll review the PR. Otherwise I'll get to it when I am able.

[david-c-kline]

There was an issue (need to investigate to see if it is still there) with how the Device Portal parsed multipart content that required the code currently in the wrapper.

If I can get some time, I'll check if the issue still exists.

[david-c-kline]

Closed by mistake... sorry

[dotMorten]

I'm guessing it's because of this bug: https://insider.windows.com/FeedbackHub/fb?contextid=519&feedbackid=19a5af49-38f4-409a-b464-e66f80679545&form=1

We could work around that by using a custom HttpContent object and write to the output stream file by file in a buffered manner. I can take a stab a that.

[dotMorten]

Took a first stab at it (see PR linked above). It would need some serious testing