[Rust] [Parquet] Intermittent test failure thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool'

[asfimport]

The following parquet tests intermittently fail

test encodings::encoding::tests::test_bool ... FAILED

thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18

ARROW-10943 tracks an intermittent error in the parquet tests. We think we fixed one error but there is still something wrong.

I can reproduce this locally at ca685a0 by running the following shell script:

#!/bin/bash
# run a command in a loop

cd /Users/alamb/Software/arrow2/rust
export PARQUET_TEST_DATA=`pwd`/../cpp/submodules/parquet-testing/data
export ARROW_TEST_DATA=`pwd`/../testing/data
export RUST_BACKTRACE=1

i=0
while true; do
    echo "iteration $i"
    cargo test -p parquet  -- test_bool
    i=$((i+1))
done

Here is an example of the full stack trace / errors:

test encodings::encoding::tests::test_bool ... FAILED

failures:

---- encodings::encoding::tests::test_bool stdout ----
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
stack backtrace:
   0: std::panicking::begin_panic
             at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:505
   1: <bool as parquet::util::bit_util::FromBytes>::from_ne_bytes
             at ./src/util/bit_util.rs:73
   2: parquet::util::bit_util::from_ne_slice
             at ./src/util/bit_util.rs:32
   3: parquet::encodings::rle::RleDecoder::get_batch
             at ./src/encodings/rle.rs:402
   4: <parquet::encodings::decoding::RleValueDecoder<T> as parquet::encodings::decoding::Decoder<T>>::get
             at ./src/encodings/decoding.rs:322
   5: parquet::encodings::encoding::tests::put_and_get
             at ./src/encodings/encoding.rs:1271
   6: <T as parquet::encodings::encoding::tests::EncodingTester<T>>::test_internal
             at ./src/encodings/encoding.rs:1214
   7: parquet::encodings::encoding::tests::EncodingTester::test
             at ./src/encodings/encoding.rs:1159
   8: parquet::encodings::encoding::tests::test_bool
             at ./src/encodings/encoding.rs:969
   9: parquet::encodings::encoding::tests::test_bool::{{closure}}
             at ./src/encodings/encoding.rs:966
  10: core::ops::function::FnOnce::call_once
             at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
  11: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Reporter: Andrew Lamb / @alamb

Related issues:

• [Rust] Intermittent build failure in parquet encoding (relates to)

_{Note: This issue was originally created as ARROW-11027. Please see the migration documentation for further details.}

[asfimport]

Andrew Lamb / @alamb:
FYI [~gbowyer@fastmail.co.uk]

When tried running with some of the code from #8948 (a previous attempt to figure out what is going on here) the problem still happens on my machine:

(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 16051b53cd3a1eebdba7a50356ee49585b69e152

(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ bash ~/Documents/loop.sh 2>& 1 | tee /tmp/loop.log | grep panic
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
   0: std::panicking::begin_panic
             at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:505

When I picked the second commit from that PR the problem still happens

(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 04f680e121eed9fe524f60b91faa5e7b64fa9921

(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 04f680e121eed9fe524f60b91faa5e7b64fa9921
[master 870ff05f9] [Rust][Hack] Replace murmur2-64a with wyhash
 Author: Greg Bowyer <gbowyer@fastmail.co.uk>
 Date: Wed Dec 16 17:13:24 2020 -0800
 2 files changed, 3 insertions(+), 2 deletions(-)
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ bash ~/Documents/loop.sh 2>& 1 | tee /tmp/loop.log | grep panic
thread 'encodings::encoding::tests::test_bool' panicked at 'assertion failed: `(left == right)`
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
thread 'encodings::encoding::tests::test_bool' panicked at 'assertion failed: `(left == right)`
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
^C

[asfimport]

Andrew Lamb / @alamb:
Migrated to github: apache/arrow-rs#159