Improved precision encoding support for float32 and float64

[MattNewberry]

Adjustment to sprintf format to better handle more precise float values, as noted in #196

[johto]

(Moving discussion here)

@MattNewberry wrote:

According to http://en.wikipedia.org/wiki/IEEE_floating_point, float precision should be as follows:

When using a decimal floating point format the decimal representation will be preserved using:
7 decimal digits for decimal32
16 decimal digits for decimal64
34 decimal digits for decimal128

Shouldn't we be looking at the values for binary32 and binary64? So 8 and 16 digits for float32 and float64, respectively.

[MattNewberry]

I'll have to defer to your knowledge on that, but am happy to update the request accordingly.

[msakrejda]

Given that Postgres is using 18 (extra_float_digits=3) for pg_dump / pg_restore, I'm curious why there is a disparity...

[johto]

I'll have to defer to your knowledge on that, but am happy to update the request accordingly.

Oh please don't. I'm quite clueless about floats; I only know not to trust one when I see it. As evidenced just now: I believe I was reading the wrong table. Near the bottom (which I believe you were reading before) it says:

The original binary value will be preserved by converting to decimal and back again using:
    9 decimal digits for binary32
    17 decimal digits for binary64

So I think we still need to +1 both of these. Sorry about that.

[msakrejda]

@johto Yeah, I think that's right. See this e-mail prompting the bump of extra_float_digits to 3 (I guess the 18 is an artifact of how Postgres handles this).

[johto]

Given that Postgres is using 18 (extra_float_digits=3) for pg_dump / pg_restore, I'm curious why there is a disparity...

Yeah, looks like the numbers postgres is using are 9 and 18. Maybe the 18 isn't really necessary, and the extra digits needs to be 3 just to make float4 work correctly? shrug

I guess this would be where someone dug up the discussion where the project last changed extra_float_digits to see what their reasoning was..

[johto]

@johto Yeah, I think that's right. See this e-mail prompting the bump of extra_float_digits to 3 (I guess the 18 is an artifact of how Postgres handles this).

Yeah, I had a vague recollection of that conversation taking place around that time. I think we have an agreement on 9 and 17 digits for float4 and float8, respectively?

[msakrejda]

Yeah, I had a vague recollection of that conversation taking place around that time. I think we have an agreement on 9 and 17 digits for float4 and float8, respectively?

Yes.

[MattNewberry]

I'm happy to submit a new pull request with the final commit if needed to prevent needless overlap.

[msakrejda]

@MattNewberry Nah, it's fine. I'll squash and commit. Thanks!

[msakrejda]

Squashed, rebased, added test, and added you to the contributors list. Thanks!

[MattNewberry]

Good deal, thank you!