cothread will not build using Python 3.7

[willrogers]

Unfortunately something seems to have changed about PyThreadState, so exc_type etc have moved.

The documentation suggests that that information is not public, but I'm not sure how else you are meant to get access to it.

hgs15624@pc0072 ~/c/p/cothread (epicscorelibs)> python3.7 setup.py build
running build
running build_py
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/cothread
copying cothread/catools.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/__init__.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/cadef.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/coserver.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/coselect.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/cosocket.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/cothread.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/dbr.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/input_hook.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/libca_path.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/load_ca.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/poll_win32.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/pv.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/py23.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/version.py -> build/lib.linux-x86_64-3.7/cothread
copying cothread/qt.py -> build/lib.linux-x86_64-3.7/cothread
creating build/lib.linux-x86_64-3.7/cothread/tools
copying cothread/tools/pvtree.py -> build/lib.linux-x86_64-3.7/cothread/tools
copying cothread/tools/__init__.py -> build/lib.linux-x86_64-3.7/cothread/tools
running build_ext
building 'cothread._coroutine' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/context
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/scratch/hgs15624/python/prefix/include/python3.7m -c context/_coroutine.c -o build/temp.linux-x86_64-3.7/context/_coroutine.o -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wundef -Wcast-align -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes
context/_coroutine.c: In function ‘coroutine_wrapper’:
context/_coroutine.c:88:17: error: ‘PyThreadState’ has no member named ‘exc_type’
     thread_state->exc_type = NULL;
                 ^
context/_coroutine.c:89:17: error: ‘PyThreadState’ has no member named ‘exc_value’
     thread_state->exc_value = NULL;
                 ^
context/_coroutine.c:90:17: error: ‘PyThreadState’ has no member named ‘exc_traceback’
     thread_state->exc_traceback = NULL;
                 ^
In file included from /scratch/hgs15624/python/prefix/include/python3.7m/pytime.h:6:0,
                 from /scratch/hgs15624/python/prefix/include/python3.7m/Python.h:68,
                 from context/_coroutine.c:33:
context/_coroutine.c:105:28: error: ‘PyThreadState’ has no member named ‘exc_type’
     Py_XDECREF(thread_state->exc_type);
                            ^
/scratch/hgs15624/python/prefix/include/python3.7m/object.h:860:50: note: in definition of macro ‘Py_XDECREF’
         PyObject *_py_xdecref_tmp = (PyObject *)(op); \
                                                  ^
context/_coroutine.c:106:28: error: ‘PyThreadState’ has no member named ‘exc_value’
     Py_XDECREF(thread_state->exc_value);
                            ^
/scratch/hgs15624/python/prefix/include/python3.7m/object.h:860:50: note: in definition of macro ‘Py_XDECREF’
         PyObject *_py_xdecref_tmp = (PyObject *)(op); \
                                                  ^
context/_coroutine.c:107:28: error: ‘PyThreadState’ has no member named ‘exc_traceback’
     Py_XDECREF(thread_state->exc_traceback);
                            ^
/scratch/hgs15624/python/prefix/include/python3.7m/object.h:860:50: note: in definition of macro ‘Py_XDECREF’
         PyObject *_py_xdecref_tmp = (PyObject *)(op); \
                                                  ^
context/_coroutine.c: In function ‘coroutine_switch’:
context/_coroutine.c:151:42: error: ‘PyThreadState’ has no member named ‘exc_type’
         PyObject *exc_type = thread_state->exc_type;
                                          ^
context/_coroutine.c:152:43: error: ‘PyThreadState’ has no member named ‘exc_value’
         PyObject *exc_value = thread_state->exc_value;
                                           ^
context/_coroutine.c:153:47: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         PyObject *exc_traceback = thread_state->exc_traceback;
                                               ^
context/_coroutine.c:168:21: error: ‘PyThreadState’ has no member named ‘exc_type’
         thread_state->exc_type = exc_type;
                     ^
context/_coroutine.c:169:21: error: ‘PyThreadState’ has no member named ‘exc_value’
         thread_state->exc_value = exc_value;
                     ^
context/_coroutine.c:170:21: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         thread_state->exc_traceback = exc_traceback;
                     ^
error: command 'gcc' failed with exit status 1

[willrogers]

cc @ifmalcuaz

[Araneidae]

It looks as if the exception state has been reorganised in 3.7. It looks like everything we need is still in pystate.h, and we'll probably want to manage exc_state and exc_info in the new version of the PyThreadState struct. I hope to look at this soonish.

Looking at this page: python/cpython#1773 Those fields were moved to a PyExcState object, and there's a PyExcState *_PyErr_GetExcInfo(PyThreadState *tstate) method in errors.c.

I've submitted a pull request: #10

[Araneidae]

An initial fix has been pushed to a working branch fix-python-3.7. However, we are not yet properly taking the chaining of generator exceptions into account ... and this may end up burning us very badly in the future; I have an uneasy feeling about how generators will cope with the stack changing underfoot.

[willrogers]

See also https://bugs.python.org/issue25612

[willrogers]

Fixed in #12