test_precise_conversion failure on 32bit boxes

[yarikoptic]

======================================================================
FAIL: test_precise_conversion (pandas.io.tests.test_parsers.TestCParserLowMemory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/buildd/pandas-0.15.0/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_parsers.py", line 3204, in test_precise_conversion
    self.assertTrue(sum(precise_errors) <= sum(normal_errors))
AssertionError: False is not true

[yarikoptic]

forgot to mention - -that was 0.15.0 release

[jreback]

hmm we had this same test fail on windows 32-bit as well
thought we turned it off

[jreback]

xref #8357

cc @mdmueller

thoughts?

[mdmueller]

Hm, I do remember this from Windows 32-bit, and I'm still unsure why it's happening--double should still be 64 bits. I'm not sure how to debug this myself, but maybe it would help if @yarikoptic or someone else could try adding print(zip(precise_errors, normal_errors)) before this assertion and posting the output?

[yarikoptic]

smells like a precision issue

(Pdb) l
3199                def error(val):
3200                    return abs(Decimal('{0:.100}'.format(val)) - actual_val)
3201                normal_errors.append(error(normal_val))
3202                precise_errors.append(error(precise_val))
3203                self.assertEqual(roundtrip_val, float(text[2:])) # round-trip should match float()
3204 ->         self.assertTrue(sum(precise_errors) <= sum(normal_errors))
3205            self.assertTrue(max(precise_errors) <= max(normal_errors))
3206    
3207        def test_pass_dtype(self):
3208            data = """\
3209    one,two
(Pdb) print sum(precise_errors), sum(normal_errors)
3.086420017447283637523651121E-14 1.665334547787666418403387064E-14
(Pdb) print sum(precise_errors) - sum(normal_errors)
1.421085469659617219120264057E-14

whenever a fix for this gets in, I will package current master and reupload it to Debian -- with other fixes hopefully it would push us forward ;-) are you planing on bugfix release soonish?

[jreback]

@yarikoptic 0.15.1 prob by end of november, will be only bug-fixes. (of course you are welcome to pull the specific bug fix issues)

[yarikoptic]

On Mon, 27 Oct 2014, jreback wrote:

@yarikoptic 0.15.1 prob by end of november, will be only bug-fixes. (of
course you are welcome to pull the specific bug fix issues)

and it will be off the current master (which seems to be collecting bug
fixes so far) or you would branch off the 0.15.0 release picking some
commits into it?

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[jreback]

we will keep merging fixes into master. then release 0.15.1 off of that.

If you need for debian to start with 0.15.0 and cherry-pick go for it (as prob only a couple of tests issues that you have identified).

[yarikoptic]

regarding this issue, I didn't realize that E-14 is not small enough really for you to be interested ;)
here are some of the offenders

(Pdb) [(i, p -n, p, n) for i, (p, n) in enumerate(zip(precise_errors, normal_errors)) if (p-n > 1e-16)]
[(2, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446053853012844920158386E-16'), Decimal('4.602699764072895050048828125E-25')), (8, Decimal('2.220446046071911193430423736E-16'), Decimal('2.220446047661112137138843536E-16'), Decimal('1.5892009437084197998046875E-25')), (14, Decimal('2.220446049250313080847263337E-16'), Decimal('2.220446051469211429357528687E-16'), Decimal('2.218898348510265350341796875E-25')), (20, Decimal('2.220446041304308362305164337E-16'), Decimal('2.220446045277310721576213837E-16'), Decimal('3.97300235927104949951171875E-25')), (26, Decimal('2.220446048920506946742534638E-16'), Decimal('2.220446049085410013794898987E-16'), Decimal('1.64903067052364349365234375E-26')), (32, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446052893509306013584137E-16'), Decimal('3.64319622516632080078125E-25')), (38, Decimal('2.220446044152904115617275238E-16'), Decimal('2.220446046701608598232269287E-16'), Decimal('2.548704482614994049072265625E-25')), (47, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446052038914076983928680E-16'), Decimal('2.78860099613666534423828125E-25')), (53, Decimal('2.220446042443713657557964325E-16'), Decimal('2.220446045847013369202613831E-16'), Decimal('3.403299711644649505615234375E-25')), (54, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446052773519791662693024E-16'), Decimal('3.5232067108154296875E-25')), (59, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446049655112661421298981E-16'), Decimal('4.0479958057403564453125E-26')), (60, Decimal('2.220446043912925086915493012E-16'), Decimal('2.220446046581619083881378174E-16'), Decimal('2.668693996965885162353515625E-25')), (65, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446053463211953639984131E-16'), Decimal('4.212898872792720794677734375E-25')), (66, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446050389718376100063324E-16'), Decimal('1.13940529525279998779296875E-25')), (71, Decimal('2.220446045292309410870075226E-16'), Decimal('2.220446047271311245858669281E-16'), Decimal('1.97900183498859405517578125E-25')), (72, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446054197817668318748474E-16'), Decimal('4.947504587471485137939453125E-25')), (77, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446051079410538077354431E-16'), Decimal('1.829097457230091094970703125E-25')), (78, Decimal('2.220446046761520840227603912E-16'), Decimal('2.220446048005916960537433624E-16'), Decimal('1.2443961203098297119140625E-25')), (83, Decimal('2.220446040524706579744815826E-16'), Decimal('2.220446044887509830296039581E-16'), Decimal('4.3628032505512237548828125E-25')), (84, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446051814016252756118774E-16'), Decimal('2.563703171908855438232421875E-25')), (89, Decimal('2.220446048140905164182186126E-16'), Decimal('2.220446048695609122514724731E-16'), Decimal('5.54703958332538604736328125E-26')), (90, Decimal('2.220446041993918009102344513E-16'), Decimal('2.220446045622115544974803925E-16'), Decimal('3.62819753587245941162109375E-25')), (95, Decimal('2.220446049250313080847263337E-16'), Decimal('2.220446052503708414733409882E-16'), Decimal('3.25339533388614654541015625E-25')), (96, Decimal('2.2
20446049250313080847263336E-16'), Decimal('2.220446049430214837193489075E-16'), Decimal('1.79901756346225738525390625E-26')), (102, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446053238314129412174225E-16'), Decimal('3.988001048564910888671875E-25')), (107, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446050119906999170780182E-16'), Decimal('8.69593918323516845703125E-26')), (108, Decimal('2.220446044842513762414455414E-16'), Decimal('2.220446047046413421630859375E-16'), Decimal('2.203899659216403961181640625E-25')), (114, Decimal('2.220446049250313080847263336E-16'), Decimal('2.220446050854512713849544525E-16'), Decimal('1.60419963300228118896484375E-25')), (120, Decimal('2.220446040074910931289196014E-16'), Decimal('2.220446044662612006068229675E-16'), Decimal('4.587701074779033660888671875E-25')), (129, Decimal('2.220446049250313080847263337E-16'), Decimal('2.220446049999917484819889069E-16'), Decimal('7.49604403972625732421875E-26')),

[yarikoptic]

anyone could take a crack at this? 32bit boxes are still used by some ;)

[mdmueller]

Unfortunately I've been pretty busy lately and still have a couple issues to look at for another organization, but I can try and fix this at some point if it's not too time-critical. Maybe this weekend or the next?

[yarikoptic]

might get critical for m
well, I am now not even sure if 0.15 would allow to enter Debian jessie which now has 0.14.1, and 0.15 didn't propagate due to such little gotchas :-/ so dunno -- the faster the better I guess -- I will at least submit for migration but there is no guarantee it would happen