Occasional Integration test failure

[pitrou]

On a PR I got what seemed to be an occasional unrelated error in the integration tests between Javascript and C++:
https://github.com/apache/arrow/actions/runs/15113644177/job/42478772147#step:11:20082

If you expand the failure, you get the following logs:

  ################# FAILURES #################
  FAILED TEST: interval_mdn C++ producing,  JS consuming
  <class 'RuntimeError'>: Command failed: /build/js/bin/integration.ts -a /tmp/tmp9z_g9id9/4b71e8dd_generated_interval_mdn.json_as_file -j /tmp/arrow-integration-da8j13w3/generated_interval_mdn.json --mode VALIDATE
  With output:
  --------------
  
   json: /tmp/arrow-integration-da8j13w3/generated_interval_mdn.json
  arrow: /tmp/tmp9z_g9id9/4b71e8dd_generated_interval_mdn.json_as_file
  Error: json and arrow record batches report the same values: fail 
   Error: 4: -1109815901,1503025502,1803493504,261723424 !== -1109815901,1503025502,1803493496,261723424
      at file:///build/js/bin/integration.ts:216:23
      at compareVectors (file:///build/js/bin/integration.ts:219:7)
      at file:///build/js/bin/integration.ts:191:13
      at compareTableIsh (file:///build/js/bin/integration.ts:193:7)
      at validateReaderIntegration (file:///build/js/bin/integration.ts:147:13)
      at validate (file:///build/js/bin/integration.ts:134:5)
      at async file:///build/js/bin/integration.ts:59:21
      at validateReaderIntegration (file:///build/js/bin/integration.ts:150:15)
      at validate (file:///build/js/bin/integration.ts:134:5)
      at async file:///build/js/bin/integration.ts:59:21
  
  --------------
  
  
  FAILED TEST: interval_mdn JS producing,  C++ consuming
  <class 'RuntimeError'>: Command failed: /build/cpp/debug/arrow-json-integration-test --integration --arrow=/tmp/tmp9z_g9id9/847e76a4_generated_interval_mdn.json_as_file --json=/tmp/arrow-integration-da8j13w3/generated_interval_mdn.json --mode=VALIDATE
  With output:
  --------------
  Error message: Invalid: Record batch 1 did not match
  JSON:
  f1:   [
      null,
      null,
      -1233922984M1832700302d3945364585489654316ns,
      -865764767M-1868167704d5995137276854357748ns,
      -1109815901M1503025502d1124093548480635000ns,
      null,
      -1853878757M-470410382d-3482109426864569063ns,
      -274343993M150680692d1713410614578950687ns,
      null,
      1204985559M1793434370d37200152077710886ns
    ]
  
  Arrow:
  f1:   [
      null,
      null,
      -1233922984M1832700302d3945364585489654316ns,
      -865764767M-1868167704d5995137276854357748ns,
      -1109815901M1503025502d1124093548480635008ns,
      null,
      -1853878757M-470410382d-3482109426864569063ns,
      -274343993M150680692d1713410614578950687ns,
      null,
      1204985559M1793434370d37200152077710886ns
    ]

And the explanation seems to be limited floating-point precision when the value -1109815901M1503025502d1124093548480635000ns is read from the JSON file. The nanoseconds components is 1124093548480635000 in the JSON, but Javascript only knows floating-point numbers and they don't have enough precision for this large integer. The nanoseconds component is therefore rounded to 1124093548480635008, and that fails comparing to the data received over Arrow IPC.

[pitrou]

cc @sumimakito @trxcllnt @domoritz

[trxcllnt]

These two lines look fishy:

arrow-js/src/util/interval.ts

Lines 49 to 50 in ea5593a

	data[ai++] = Number(BigInt(nanoseconds) & BigInt(0xFFFFFFFF));
	data[ai++] = Number(BigInt(nanoseconds) >> BigInt(32));

[raulcd]

This keeps failing occasionally. Example on a PR with unrelated changes to integration tests:
https://github.com/apache/arrow/actions/runs/15389273840/job/43297060659?pr=46261

[mbrobbel]

https://github.com/apache/arrow-rs/actions/runs/15997777144/job/45125088356#step:14:649

#
  # Fatal error in , line 0
  # unreachable code
  #
  #
  #
  #FailureMessage Object: 0x7fca6451de60
  ----- Native stack trace -----
  
   1: 0x7fca6bfa0fd7  [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   2: 0x7fca6ccdb91f V8_Fatal(char const*, ...) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   3: 0x7fca6c9e9b8f int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHandle<v8::internal::Isolate> >(unsigned char, v8::internal::SlotAccessorForHandle<v8::internal::Isolate>) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   4: 0x7fca6c9e9d31 v8::internal::Deserializer<v8::internal::Isolate>::ReadObject() [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   5: 0x7fca6c9eb4b6 v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   6: 0x7fca6c9e9545 int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHandle<v8::internal::Isolate> >(unsigned char, v8::internal::SlotAccessorForHandle<v8::internal::Isolate>) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   7: 0x7fca6c9e9d31 v8::internal::Deserializer<v8::internal::Isolate>::ReadObject() [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   8: 0x7fca6c9eb4b6 v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
   9: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  10: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  11: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  12: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  13: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  14: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  15: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  16: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  17: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  18: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  19: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  20: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  21: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  22: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  23: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  24: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  25: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  26: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  27: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  28: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  29: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  30: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  31: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  32: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  33: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  34: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  35: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  36: 0x7fca6c9ea03d int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHeapObject>(unsigned char, v8::internal::SlotAccessorForHeapObject) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  37: 0x7fca6c9eb261 v8::internal::Deserializer<v8::internal::Isolate>::ReadData(v8::internal::Handle<v8::internal::HeapObject>, int, int) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  38: 0x7fca6c9eb5aa v8::internal::Deserializer<v8::internal::Isolate>::ReadObject(v8::internal::SnapshotSpace) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  39: 0x7fca6c9e9545 int v8::internal::Deserializer<v8::internal::Isolate>::ReadSingleBytecodeData<v8::internal::SlotAccessorForHandle<v8::internal::Isolate> >(unsigned char, v8::internal::SlotAccessorForHandle<v8::internal::Isolate>) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  40: 0x7fca6c9e9d31 v8::internal::Deserializer<v8::internal::Isolate>::ReadObject() [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  41: 0x7fca6c9ed796 v8::internal::ObjectDeserializer::Deserialize() [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  42: 0x7fca6c9ed9a1 v8::internal::ObjectDeserializer::DeserializeSharedFunctionInfo(v8::internal::Isolate*, v8::internal::SerializedCodeData const*, v8::internal::Handle<v8::internal::String>) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  43: 0x7fca6c9e03d7 v8::internal::CodeSerializer::Deserialize(v8::internal::Isolate*, v8::internal::AlignedCachedData*, v8::internal::Handle<v8::internal::String>, v8::ScriptOriginOptions, v8::internal::MaybeHandle<v8::internal::Script>) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  44: 0x7fca6c4c9112  [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  45: 0x7fca6c4c94bd v8::internal::Compiler::GetSharedFunctionInfoForScriptWithCachedData(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::ScriptDetails const&, v8::internal::AlignedCachedData*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason, v8::internal::NativesFlag) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  46: 0x7fca6c4279e6 v8::ScriptCompiler::CompileUnboundInternal(v8::Isolate*, v8::ScriptCompiler::Source*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  47: 0x7fca6bf10b8b node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  48: 0x7fca6c46523e  [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  49: 0x7fca6c465560 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  50: 0x7fca6c22d1b6  [/opt/conda/envs/arrow/bin/../lib/libnode.so.115]
  Trace/breakpoint trap (core dumped)
  ======================================================================
  Testing file /tmp/arrow-integration-7l67ks1p/generated_datetime.json
  -- Creating binary inputs
  -- Validating file
  -- Validating stream
  Traceback (most recent call last):
    File "/__w/arrow-rs/arrow-rs/dev/archery/archery/integration/runner.py", line 317, in _run_ipc_test_case
      run_binaries(producer, consumer, test_case)
    File "/__w/arrow-rs/arrow-rs/dev/archery/archery/integration/runner.py", line 359, in _produce_consume
      consumer.stream_to_file(producer_stream_path, consumer_file_path)
    File "/__w/arrow-rs/arrow-rs/dev/archery/archery/integration/tester_js.py", line 73, in stream_to_file
      self.run_shell_command(cmd, cwd=ARROW_JS_ROOT)
    File "/__w/arrow-rs/arrow-rs/dev/archery/archery/integration/tester.py", line 233, in run_shell_command
      subprocess.check_call(cmd, **kwargs)
    File "/opt/conda/envs/arrow/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '/build/js/bin/stream-to-file.ts < /tmp/tmpd8pb16ti/874c61a2_generated_datetime.producer_file_as_stream > /tmp/tmpd8pb16ti/874c61a2_generated_datetime.consumer_stream_as_file' returned non-zero exit status 133.
  ======================================================================
...
################# FAILURES #################
  FAILED TEST: datetime Rust producing,  JS consuming
  <class 'subprocess.CalledProcessError'>: Command '/build/js/bin/stream-to-file.ts < /tmp/tmpd8pb16ti/874c61a2_generated_datetime.producer_file_as_stream > /tmp/tmpd8pb16ti/874c61a2_generated_datetime.consumer_stream_as_file' returned non-zero exit status 133.
  

[pitrou]

@mbrobbel This seems like it could be a different problem, can you open a separate issue for it?

[mbrobbel]

Another one: https://github.com/apache/arrow-rs/actions/runs/16264960698/job/45918450859#step:14:6653

[trxcllnt]

@mbrobbel Is it possible to upload the generated_datetime.producer_file_as_stream as a workflow artifact so we can download and debug why it's failing?