support lossless .ARW recompression

[StephanBusch]

Sony uses the .ARW format for their raw photographs.
For many years those files were hardly compressible but now they can be decoded losslessly.
The sources can be found here

https://github.com/FLIF-hub/FLIF/tree/master/raw-tools/sony_arw

Could this format please be supported by Precomp?

[schnaader]

Sounds good, especially the author's intent to get a "truly lossless" algorithm, not only pixel-perfect. I'll have to contact the author due to the GPLv3 license, though. Apache 2.0 and GPLv3 are compatible in only one direction - integrating Apache 2.0 code into a GPLv3 project is OK, but the other way round is not, so I'll need some solution for this (for example author's permission to use it).

[goblin]

Yeah sure, go ahead, I don't want licensing to get in the way of opensource, perhaps I should add MIT as well ;-) I've updated the README to include Apache 2.0 as an option so you should have no trouble with that now.

However please note that dcraw.c uses GPLv2 and some other options, do make sure that it's fine with Dave Coffin. I think it should be OK to remove the "RESTRICTED" functions from dcraw.c as I only really use a small portion of dcraw - this should make it easier to comply with (a) or (c) options of dcraw license.

I'm not a lawyer and this is not legal advice etc.

[goblin]

Wow, I wasn't aware of packJPG, this stuff looks like dark magic! It could probably be used to further compress the header of the ARW, which contains a JPG preview.

[schnaader]

Cool, thanks for the quick note and permission! I'll have a look into dcraw.c.

packJPG is close to dark magic, yeah, the key to understanding it is that JPG has a lossy part and, after that, a lossless part. The lossless part is essentially huffman coding, it can be reverted and replaced with a better compression (arithmetic coding). There are some other tricks used (correlation between coefficients) to boost compression further.

[StephanBusch]

would it help you if I contribute a windows version of Goblin's ARW recompressor?
What exactly would help you the most so that this feature can be added into Precomp?

[schnaader]

Sorry for the late reply. At the moment, I just don't have much spare time to work on this. I saw that Shelwien did some promising work on encode.ru and will have a look into that.

But at the moment, other issues do have higher priority and doing all the stuff the scripts do in a way that doesn't rely on external tools is much work, so don't expect any progress before a Precomp 0.4.6 release.

Issue at FLIF (for reference)