Fix incorrectly handling quotes

[mikicz]

Basically we came across csvs which had patterns equal to patterns in csv_gp_python/tests/fixtures/quote_and_newline.csv, which are valid (get loaded fine by excel & pandas), but we didn't handle correctly

[blaginin]

now this one fails as invalid

a,b,c
"lll","""",""",
"

[mikicz]

fuuuuuuuuck

[mikicz]

Ok, throw some more cases at me! 😄

[blaginin]

Ok, throw some more cases at me! 😄

well if you're asking 😉

[blaginin]

I think this one is also not correctly handled

a,b,c
1,2 "and three",3

quotes might be part of the cell, and not necessarily wrap the whole cell. This reads correctly by pandas or excel. But your version marks it as incorrect

Not sure if it's the same bug / worth fixing in the same pr tho

[blaginin]

if you're going to fix this here: maybe check that that you won't accidentally make this valid

a,b,c
1,2 "and
three",3

[mikicz]

Those do get detected as invalid, see the test I just added

[blaginin]

but the first one really should be valid?

[mikicz]

Hmmm according to RFC 4180 it's valid but it it's not what we ask for when we ask for csvs - all cells to be quoted and enclosed in a quoted cell

[mikicz]

I messed that sentence up in the rewrite

• all quotes to be escaped, and enclosed in quoted cells