Image memory usage regression

[marlenecota]

If an Image's width and height have been set, then we can presumably set the desiredMaxSize of the LoadedImageSurface when loading the resource.

Caveats:

• maxWidth and maxHeight take priority
• (max)Width and (max)Height have to be constants. If they're using a % and then resize the window to make it bigger, then the image wouldn't grow.
• Do both values (width and height) have to be set? I think that if only one of the values in the pair has been set, we can do this: StartLoadFromStream(stream, {width, 0}) and get what we want.

[stecrain]

I just wanted to chime in and say that this can make a big difference when over-sized assets are used.

In the app I am working on I have an app bug resulting in me requesting 4k images in a 280 x 150 size image on screen.
Now I definitely should not request 4k images to display them so small on screen.
Luckily I have the benefit of just adding some url params and requesting a smaller image.

But for developers who don't have access to a resizing service this is a big deal.

I changed this line in ReactImage.cpp

winrt::LoadedImageSurface::StartLoadFromStream(memoryStream) :

to

winrt::LoadedImageSurface::StartLoadFromStream(memoryStream, m_brush->AvailableSize()) :

The result was a 100MB reduction in App Total Commit for the same page.

This definitely seems like something that should be included in the framework.
I know Xaml already will deduce a decode size when using the element, but I don't know the details of how to achieve that while still supporting all the effects.

@kmelmon looking forward to hearing your thoughts

[kmelmon]

@stecrain this is a good catch. With the change to use LoadedImageSurface, we lost "right-size-decoding".

After thinking about this and talking it over with @codendone here's what I think we should do:

1. Only use LoadedImageSurface for repeating images. In this scenario I think we're likely safe just decoding to the image's natural size as it's supposed to tile. We can further optimize this case if necessary.
2. For all other cases, use a BitmapImage instead, which will give us back the right-size-decoding optimization. There is a fairly significant amount of complexity that we'll get for free if we do this, namely:
   a) Right-size-decoding computes the actual size on screen in pixels, which includes all the render transforms in the tree.
   b) The size to decode to is computed during the RenderWalk which handles all the weird timing concerns that come up with this (layout has to run first, properties can change multiple times, etc).
   b) When the computed size on screen changes, right-size-decoding is smart enough to only decode again if the size gets larger.

Another observation: For images coming from http/https, we're currently downloading the image ourselves into a stream and we're not doing any caching of this stream. This means we'll download/decode/store the same image multiple times if the same URI is used across multiple images.
XAML has caching smarts if we pass the URI into XAML. XAML doesn't fully support all the features needed (such as the authorization header), but we should investigate using the URI in more cases as we'll get caching benefits for free. Alternatively we could do caching ourselves.

[stecrain]

Those are all great points @kmelmon
Regarding the caching, I think we can probably only download manually when auth (or other headers) are present. At least that way we get the image caching.

I might be missing something though, because to me it seems like using BitmapImage conditionally would require quite a bit of code, and refactoring. (Unless you can use a BitmapImage as the source on a Canvas?)