SIGSEGV: invalid memory reference in test_class

[messense]

 RUST_BACKTRACE=1 cargo test --verbose --test test_class
       Fresh quote v0.3.15
       Fresh libc v0.2.23
       Fresh utf8-ranges v1.0.0
       Fresh num-traits v0.1.37
       Fresh log v0.3.7
       Fresh void v1.0.2
       Fresh regex-syntax v0.4.1
       Fresh unicode-xid v0.0.4
       Fresh memchr v1.0.1
       Fresh thread-id v3.1.0
       Fresh unreachable v0.1.1
       Fresh synom v0.11.3
       Fresh aho-corasick v0.6.3
       Fresh thread_local v0.3.3
       Fresh syn v0.11.11
       Fresh regex v0.2.2
       Fresh env_logger v0.4.2
       Fresh pyo3cls v0.1.0 (file:///Users/messense/Projects/PyO3/pyo3cls)
       Fresh pyo3 v0.1.0 (file:///Users/messense/Projects/PyO3)
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `/Users/messense/Projects/PyO3/target/debug/deps/test_class-27aaf9ec0adb8344`

running 26 tests
test binary_arithmetic ... ok
test delitem ... ok
test comparisons ... ok
test callable ... ok
test data_is_dropped ... ok
test context_manager ... ok
test class_with_properties ... ok
test contains ... ok
test empty_class ... ok
test empty_class_in_module ... ok
test empty_class_with_new ... ok
test new_with_two_args ... ok
test inplace_operations ... ok
error: process didn't exit successfully: `/Users/messense/Projects/PyO3/target/debug/deps/test_class-27aaf9ec0adb8344` (signal: 11, SIGSEGV: invalid memory reference)

Caused by:
  process didn't exit successfully: `/Users/messense/Projects/PyO3/target/debug/deps/test_class-27aaf9ec0adb8344` (signal: 11, SIGSEGV: invalid memory reference)

macOS 10.12.5
Python 3.6.1

[fafhrd91]

can you reliably reproduce this?

[messense]

Yes, every time. But seems a little random:

running 26 tests
test callable ... ok
test data_is_dropped ... ok
test contains ... ok
test class_with_properties ... ok
test context_manager ... ok
test binary_arithmetic ... ok
test comparisons ... FAILED
test delitem ... ok
test empty_class ... ok
test empty_class_in_module ... ok
test empty_class_with_new ... ok
test gc_integration ... ok
test inplace_operations ... ok
error: process didn't exit successfully: `/Users/messense/Projects/PyO3/target/debug/deps/test_class-27aaf9ec0adb8344` (signal: 11, SIGSEGV: invalid memory reference)

[messense]

Reproduced in Travis CI, seems related to Python 3.6, Python 3.5 passed.

[fafhrd91]

I restarted job and it passed

[fafhrd91]

something related to ref counters. I can not reproduce this problem reliably

[messense]

rust-lldb ./target/debug/test_class-27aaf9ec0adb8344
(lldb) command source -s 0 '/tmp/rust-lldb-commands.kQLx0x'
Executing commands in '/tmp/rust-lldb-commands.kQLx0x'.
(lldb) command script import "/Users/messense/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/etc/lldb_rust_formatters.py"
(lldb) type summary add --no-value --python-function lldb_rust_formatters.print_val -x ".*" --category Rust
(lldb) type category enable Rust
(lldb) target create "./target/debug/test_class-27aaf9ec0adb8344"
Current executable set to './target/debug/test_class-27aaf9ec0adb8344' (x86_64).
(lldb) l
   1    #![feature(proc_macro, specialization)]
   2    #![allow(dead_code, unused_variables)]
   3
   4    extern crate pyo3;
   5
   6    use pyo3::*;
   7    use std::{mem, isize, iter};
   8    use std::cell::RefCell;
   9    use std::sync::Arc;
   10   use std::sync::atomic::{AtomicBool, Ordering};
(lldb) c
error: invalid process
(lldb) run
Process 47166 launched: './target/debug/test_class-27aaf9ec0adb8344' (x86_64)

running 26 tests
test callable ... ok
test delitem ... ok
test data_is_dropped ... ok
test contains ... ok
test class_with_properties ... ok
test comparisons ... ok
test binary_arithmetic ... ok
test context_manager ... ok
test empty_class ... ok
test empty_class_in_module ... ok
test new_with_one_arg ... ok
test gc_integration ... ok
test inplace_operations ... ok
test_class-27aaf9ec0adb8344 was compiled with optimization - stepping may behave oddly; variables may not be available.
Process 47166 stopped
* thread #10, name = 'instance_method', stop reason = EXC_BAD_ACCESS (code=1, address=0x1ffffffff)
    frame #0: 0x000000010012975b test_class-27aaf9ec0adb8344`_PyObject_Alloc(use_calloc=<unavailable>, ctx=<unavailable>, nelem=<unavailable>, elsize=<unavailable>) at obmalloc.c:1258 [opt]

[messense]

Sometimes:

Process 47580 stopped
* thread #9, name = 'delitem', stop reason = EXC_BAD_ACCESS (code=1, address=0x1ffffffff)
    frame #0: 0x000000010012975b test_class-27aaf9ec0adb8344`_PyObject_Alloc(use_calloc=<unavailable>, ctx=<unavailable>, nelem=<unavailable>, elsize=<unavailable>) at obmalloc.c:1258 [opt]

Process 47624 stopped
* thread #10, name = 'rich_comparisons', stop reason = EXC_BAD_ACCESS (code=1, address=0x1)
    frame #0: 0x00000001001c5595 test_class-27aaf9ec0adb8344`PyArena_Free(arena=<unavailable>) at pyarena.c:175 [opt]

[fafhrd91]

reference to PyMethodDef get destroyed?

[messense]

Any idea how to confirm that?

[fafhrd91]

it is not big deal at the moment.
I am trying to port async-tokio to new system and I am not sure about usability of new system

[messense]

Seems something wrong with inplace_operations test case, after commenting it out, test passes.

Got some log:

test inplace_operations ... ACQUIRE
drop pyptr: 0x10b687d20 12 0x70000bcdfb28
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 50 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5ed0 754 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 51 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5ed0 754 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 52 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5f10 46 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 53 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5ef0 124 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 54 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5ef0 124 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 55 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5ff0 26 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 56 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5f10 46 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop pyptr: 0x10b687d20 12 0x70000bcdfca8
drop pptr: 0x10be7e9d0 57 0x70000bcdfd88
ACQUIRE
RELEASE
drop pyptr: 0x10b6e5f50 28 0x70000bcdf4d8
drop Py: 0x10be11270 5 0x70000bcdf4f8
drop Py: 0x10be11270 4 0x70000bcdf538
drop Py: 0x10be11270 4 0x70000bcdf538
drop pyptr: 0x10b6a0ab8 2302 0x70000bcdfde8
drop pyptr: 0x10c0590d8 1 0x70000bcdff88
DEALLOC: InPlaceOperations, 0x10be11270
drop Py: 0x10be11270 4494266968 0x70000bcdffa8
drop Py: 0x10be11270 4494266967 0x70000bcdffa8
drop Py: 0x10be11270 4494266966 0x70000bcdffa8
drop Py: 0x10be11270 4494266965 0x70000bcdffa8
drop Py: 0x10be11270 4494266964 0x70000bcdffa8
drop Py: 0x10be11270 4494266963 0x70000bcdffa8
drop Py: 0x10be11270 4494266962 0x70000bcdffa8
drop Py: 0x10be11270 4494266961 0x70000bcdffa8
RELEASE
ok
test instance_method ... ACQUIRE
drop pyptr: 0x10b686290 4 0x70000bcdff38
drop pyptr: 0x10b686290 4 0x70000bce00b8
drop Py: 0x10be11270 2 0x70000bce0118
drop pptr: 0x10c085f48 3 0x70000bce01a8
ACQUIRE
RELEASE
error: process didn't exit successfully: `/Users/messense/Projects/PyO3/target/debug/deps/test_class-27aaf9ec0adb8344 --nocapture --test-threads=1` (signal: 11, SIGSEGV: invalid memory reference)

The part below:

DEALLOC: InPlaceOperations, 0x10be11270
drop Py: 0x10be11270 4494266968 0x70000bcdffa8
drop Py: 0x10be11270 4494266967 0x70000bcdffa8
drop Py: 0x10be11270 4494266966 0x70000bcdffa8
drop Py: 0x10be11270 4494266965 0x70000bcdffa8
drop Py: 0x10be11270 4494266964 0x70000bcdffa8
drop Py: 0x10be11270 4494266963 0x70000bcdffa8
drop Py: 0x10be11270 4494266962 0x70000bcdffa8
drop Py: 0x10be11270 4494266961 0x70000bcdffa8

looks suspicious. ref count shouldn't be that high.

[fafhrd91]

good catch!

[fafhrd91]

0x70000bcdffa8 == -1

[messense]

Confirmed a3e6117 fixed it.