Fix memory overhead

[Therzok]

No description provided.

[Therzok]

I'll provide numbers soon and give a shout when it's ready to merge.

[Therzok]

Ok, so, running the whole thing in VS diagnostics:

100 times iteration on a 65 * 1000 * 1000 bytes, gc.collect after each run - worst case possible.

• Old version, peak usage 167mb, 14.03s runtime
  
• Old version, but with memorystream initialization set - 145mb, 8.78s runtime.
  
• New version as in this PR, 74mb peak memory, 5.75s runtime
  

Without GCs:

• Old - 259mb memory usage, 16.93s runtime
  
• Old with init - 351mb peak, 13.20s runtime. (one object isn't collected, so we have multiple streams alive, but it should be way lower in actual setups).
  
• New version - 76mb peak, 7.19s runtime.
  

I'd definitely say this is an improvement. It also seems that the byte arrays get reused (rare collections), so this is a big +.

These numbers are just for crunching and not meant to say anything more than this solution is an improvement. Overall speed and memory usage should be improved by this PR. Marking as ready to merge.

cc @kjpou1

[Therzok]

Tests done with this code. File created in cmd/powershell in the working directory of the application with fsutil createnew myfile.txt 65000000.

[Therzok]

Also tested them for integrity.

[Therzok]

Up for theoretical case from #295:

The asset stream is X bytes.

With the old version, there would be at least ... bytes allocated:

• X bytes for the asset stream.
• 1 + 2 + 4 + 8 + ... + Y, where Y=2^k, Y>X
• X bytes for the byte array.

With the init version:

• X bytes for the asset stream.
• X bytes for the memorystream.
• X bytes for the bytearray.

With the new version:

• X bytes for the asset stream.
• X bytes for the bytearray.

[Therzok]

Just a further note. It may be useful to abstract this into an internal extension method on Streams, since I've found multiple instances copyto()/toarray() calls on streams.

[kjpou1]

Thanks Marius