fix floating point parsing precision in some rare cases#4463
fix floating point parsing precision in some rare cases#4463
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4463 +/- ##
==========================================
- Coverage 99.38% 99.38% -0.01%
==========================================
Files 76 76
Lines 14490 14489 -1
==========================================
- Hits 14401 14400 -1
Misses 89 89
Continue to review full report at Codecov.
|
|
You could drop the lower half of the table (thus removing (may be off-by-one) |
… it was distracting to see the number as a column name too
|
I checked that the new test failed for me also before this PR. |
|
Interesting that the other 12 out of 100,000 also straddle by 0.0...6 too. Building on your example: > for (i in 0:99999) {
s = sprintf("0.80606%05d", i)
r = eval(parse(text=s))
f = fread(text=s)$V1
if (!identical(r, f))
cat(s, sprintf("%1.17f", r), sprintf("%1.17f", f), "\n")
}
0.8060603509 0.80606035089999994 0.80606035090000006
0.8060614740 0.80606147399999994 0.80606147400000006
0.8060623757 0.80606237569999994 0.80606237570000006
0.8060629084 0.80606290839999994 0.80606290840000006
0.8060632774 0.80606327739999994 0.80606327740000006
0.8060638101 0.80606381009999994 0.80606381010000006
0.8060647118 0.80606471179999994 0.80606471180000006
0.8060658349 0.80606583489999994 0.80606583490000006
0.8060667366 0.80606673659999994 0.80606673660000006 # Gabe's pick
0.8060672693 0.80606726929999994 0.80606726930000006
0.8060676383 0.80606763829999994 0.80606763830000006
0.8060681710 0.80606817099999994 0.80606817100000006
0.8060690727 0.80606907269999994 0.80606907270000006 |
|
@MichaelChirico better? |
|
perfect! much improved. |
Closes #4461
Solution actually easier than expected, h/t https://stackoverflow.com/a/4937591/3576984 for the inspiration.