-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Describe the bug
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_1 ... ok
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_2 ... ok
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_3 ... ok
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_4 ... ok
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_5 ... ok
thread 'datasource::file_format::csv::tests::test_csv_parallel_one_col::case_6' panicked at datafusion\core\src\datasource\file_format\csv.rs:1267:9:
assertion left == right failed
left: 30
right: 20
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
test datasource::file_format::csv::tests::test_csv_parallel_one_col::case_6 ... FAILED
failures:
failures:
failures:
failures:
datasource::file_format::csv::tests::test_csv_parallel_one_col::case_6
failures:
datasource::file_format::csv::tests::test_csv_parallel_one_col::case_6
test result: FAILED. 5 passed; 1 failed; 0 ignored; 0 measured; 708 filtered out; finished in 0.11s
error: test failed, to rerun pass --lib
D:\workspace\rust\datafusion\datafusion\core>
failures:
datasource::file_format::csv::tests::test_csv_parallel_one_col::case_6
test result: FAILED. 5 passed; 1 failed; 0 ignored; 0 measured; 708 filtered out; finished in 0.11s
To Reproduce
No response
Expected behavior
No response
Additional context
I notice that, below code, which gets file_size of one_col.csv by target_os. I don't think this is right.
datafusion/datafusion/core/src/datasource/file_format/csv.rs
Lines 1254 to 1258 in 5da7ab3
| let file_size = if cfg!(target_os = "windows") { | |
| 30 // new line on Win is '\r\n' | |
| } else { | |
| 20 | |
| }; |
Because
tests/data/one_col.csv is save as lf instead of crlf, won't apprear \r\n. I got file info in windows machine. according file info, this file is 20bytes.
Directory of D:\workspace\rust\datafusion\datafusion\core\tests\data
2024/06/19 21:40 20 one_col.csv
1 File(s) 20 bytes
0 Dir(s) 4,626,849,792 bytes free