Changes buffer size for writing header in fwrite

[philippechataignon]

This PR is an attempt to Resolve #5048.

Actually, buffer size for writing header only uses headerLen. headerLen is computed accurately and deflateBound returns a buffer size witch guarantees an one pass compression. But that's not true with OpenBSD which uses on old zlib version.

Here, we use by default the buffer size used for rows from the buffer size parameter (buffMB) and, if we have a very long header, we use headerLen.

[mattdowle]

Many thanks! If possible it would be great if you could raise an issue to the OpenBSD team (maybe they are not aware it is using a 16-year version of zlib) and then we can place a link to that question/report in our news item so that readers of our news can be reassured that it has been reported to OpenBSD team.

[philippechataignon]

Thanks Matt to merge this PR without waiting my answer. I confirm this PR fixes the problem. I will send a issue to OpenBSD team.

[philippechataignon]

Here is report on openbsd-misc list:
https://marc.info/?l=openbsd-misc&m=162445581925627&w=1

[mattdowle]

Thanks @philippechataignon. But if I were them I would be unclear reading that. What I had in mind was more like:
"Hi. Is it intentional that OpenBSD 6.9, released May 2021 uses a 16 year old version of zlib (v1.2.3; July 2005) when the latest version v1.2.11 (Jan 2017) is available?"
Could you post a follow a follow up please and we'll link to your follow up post. Although, perhaps OpenBSD team will miss the reply if it's in the same thread. So a new thread would avoid that problem.

[philippechataignon]

It's an old version but they applied regularly patches as you can see here : https://cvsweb.openbsd.org/src/lib/libz/?sortby=date#dirlist : it's not a real 1.2.3. For example of asking a new zlib version, you can see this answer : https://marc.info/?l=openbsd-misc&m=141066380405272&w=2. OpenBSD system is very conservative and I'm not sure they will jump to 1.2.11.
Perhaps, if you disagree, you can send your message above as lead maintainer in a new thread and link it in news. Not a problem for me.

[mattdowle]

It's an old version but they applied regularly patches as you can see here : https://cvsweb.openbsd.org/src/lib/libz/?sortby=date#dirlist : it's not a real 1.2.3.

It's sort of a fork of zlib then but with no new name?

For example of asking a new zlib version, you can see this answer : https://marc.info/?l=openbsd-misc&m=141066380405272&w=2

The subject was "no zlib from zlib.net" (where the word 'no' is the first word) so after reading that exchange a few times I suspect the responder may have been frustrated by the unhelpful and incorrect subject and the comparison made to Ubuntu and then responded bluntly to correct the incorrect 'no' and the reply ended up along the lines of "we in fact have v1.2.3 from 2005". I guess that was frustrating to the OP because they knew that and it was 2014. When busy people in a dev team scan a mailing list, each subject conveys how much time each OP took to make a good subject and therefore how much respect each OP has for the responders they are seeking a reply from.

OpenBSD system is very conservative and I'm not sure they will jump to 1.2.11. Perhaps, if you disagree, you can send your message above as lead maintainer in a new thread and link it in news. Not a problem for me.

Essentially, I have complained about OpenBSD in our news item, and that's what I'm concerned about. I want to get the messaging right before release and get the facts right. It currently reads as follows :

OpenBSD 6.9 released May 2021 apparently uses a 16 year old version of zlib (v1.2.3 from 2005) which induces Compress gzip error: -9 from fwrite(), #5048. Thanks to Philippe Chataignon for investigating and for the PR which attempts a solution.

I'm glad I used the word 'apparently' as indeed it seems 'it's not really 1.2.3'. We'll continue to revise this news item before release. It is frustrating to me that we are spending time fixing a problem caused by a new release of an operating system (OpenBSD 6.9 was released last month May 2021) that uses a 16 year old version of a wildly popular and depended on library such as zlib. I have hope that it was either not intentional, there's a good reason such as maybe zlib's license changed after 1.2.3, or there's something else I didn't know I didn't know.

Here's my attempt then : https://marc.info/?l=openbsd-misc&m=162455479311886&w=1

• version numbers in title
• "16 year old" in title
• "please" in title
• First words of body are "is it", a question.
• link to background (this PR where I'm writing now)
• very short post; as short as I could make it. I spent time on making it short and removing superfluous information.
• "intentional or is there any good reason" is truly what I'm asking.

[rdm]

This looks like it had been a protocol / format change, though one which apparently was obscure enough that many people did not notice.

But that makes it worth careful inspection, and the change in the compressed representation should be well documented.