[Rust] Explicitly seeking a BufReader will discard the internal buffer

[asfimport]

This behavior was observed in the Parquet Rust file reader (parquet/src/util/io.rs).

 

Pull request: #6280

 

From the Rust documentation for BufReader:

 

"Seeking always discards the internal buffer, even if the seek position would otherwise fall within it. This guarantees that calling .into_inner() immediately after a seek yields the underlying reader at the same position."

 

https://doc.rust-lang.org/std/io/struct.BufReader.html#impl-Seek

Reporter: Max Burke / @maxburke

Related issues:

• [Rust] FileSource read implementation is seeking for each single byte (fixes)

PRs and other links:

• GitHub Pull Request #6280
• GitHub Pull Request #6949

_{Note: This issue was originally created as ARROW-7681. Please see the migration documentation for further details.}

[asfimport]

Andy Grove / @andygrove:
Deferring this to 1.0.0 due to the concerns about the PR adding further dependencies on unstable Rust features

[asfimport]

Rémi Dettai / @rdettai:
I've proposed a fix in #6949 that uses an inlined modified version of BufReader instead of nightly methods.

 

I've tested it on a large parquet of mine:

• on fast disk, both versions take ~30s to read the column
• on a slow mount, the the old version takes ~160s and the fixed one still takes ~30s (still CPU bounded)

[asfimport]

Chao Sun / @sunchao:
Issue resolved by pull request 6949
#6949