Failure to initialize default stdout and stderr in the InProcessKernel

[rayosborn]

I use the in-process kernel in an Jupyter shell embedded in a PyQt GUI. It is based on the QtConsole RichJupyterWidget. Here is a slimmed-down version of the code to set it up.

from qtconsole.inprocess import QtInProcessKernelManager
from qtconsole.rich_jupyter_widget import RichJupyterWidget

class MainWindow(QtWidgets.QMainWindow):

    def __init__(self, app, tree, settings, config):
        """ Create a MainWindow for the application.
        super(MainWindow, self).__init__()
        mainwindow = QtWidgets.QWidget()
        self.console = RichJupyterWidget(config=self.config)
        self.console.kernel_manager = QtInProcessKernelManager(config=self.config)
        self.console.kernel_manager.start_kernel()
        self.console.kernel_manager.kernel.gui = 'qt'
        self.console.kernel_client = self.console.kernel_manager.client()
        self.console.kernel_client.start_channels()
        self.console.show()

Calls to the QtConsole FrontendWidget function _silent_exec_callback trigger an Empty exception in the Queue module.

2021-07-12 10:53:24,760 - ERROR - Exception in GUI event loop
---------------------------------------------------------------------------
Empty                                     Traceback (most recent call last)
~/Documents/Computing/Repositories/nexpy/src/nexpy/gui/mainwindow.py in update_all_magic_menu(self)
   2037 
   2038         """
-> 2039         self.console._silent_exec_callback('get_ipython().magic("lsmagic")',
   2040                 self.populate_all_magic_menu)
   2041 

~/opt/miniconda3/envs/py38/lib/python3.8/site-packages/qtconsole/frontend_widget.py in _silent_exec_callback(self, expr, callback)
    417         # not the unique request originated from here (can use msg id ?)
    418         local_uuid = str(uuid.uuid1())
--> 419         msg_id = self.kernel_client.execute('',
    420             silent=True, user_expressions={ local_uuid:expr })
    421         self._callback_dict[local_uuid] = callback

~/Documents/Computing/Repositories/ipykernel/ipykernel/inprocess/client.py in execute(self, code, silent, store_history, user_expressions, allow_stdin)
    108                        allow_stdin=allow_stdin)
    109         msg = self.session.msg('execute_request', content)
--> 110         self._dispatch_to_kernel(msg)
    111         return msg['header']['msg_id']
    112 

~/Documents/Computing/Repositories/ipykernel/ipykernel/inprocess/client.py in _dispatch_to_kernel(self, msg)
    178         loop = asyncio.get_event_loop()
    179         loop.run_until_complete(kernel.dispatch_shell(msg_parts))
--> 180         idents, reply_msg = self.session.recv(stream, copy=False)
    181         self.shell_channel.call_handlers_later(reply_msg)
    182 

~/opt/miniconda3/envs/py38/lib/python3.8/site-packages/jupyter_client/session.py in recv(self, socket, mode, content, copy)
    807             socket = socket.socket
    808         try:
--> 809             msg_list = socket.recv_multipart(mode, copy=copy)
    810         except zmq.ZMQError as e:
    811             if e.errno == zmq.EAGAIN:

~/Documents/Computing/Repositories/ipykernel/ipykernel/inprocess/socket.py in recv_multipart(self, flags, copy, track)
     30 
     31     def recv_multipart(self, flags=0, copy=True, track=False):
---> 32         return self.queue.get_nowait()
     33 
     34     def send_multipart(self, msg_parts, flags=0, copy=True, track=False):

~/opt/miniconda3/envs/py38/lib/python3.8/queue.py in get_nowait(self)
    196         raise the Empty exception.
    197         '''
--> 198         return self.get(block=False)
    199 
    200     # Override these methods to implement other queue organizations

~/opt/miniconda3/envs/py38/lib/python3.8/queue.py in get(self, block, timeout)
    165             if not block:
    166                 if not self._qsize():
--> 167                     raise Empty
    168             elif timeout is None:
    169                 while not self._qsize():

Empty: 

I haven't been able to diagnose the issue, but the problem appears to be in the following lines in ipykernel InProcessKernelClient function, _dispatch_to_kernel in the following lines:

        stream = kernel.shell_stream
        self.session.send(stream, msg)
        msg_parts = stream.recv_multipart()
        loop = asyncio.get_event_loop()
        loop.run_until_complete(kernel.dispatch_shell(msg_parts))
        idents, reply_msg = self.session.recv(stream, copy=False)
        self.shell_channel.call_handlers_later(reply_msg)

If I monitor the DummySocket send_multipart and recv_multipart functions, there is a send and receive pair triggered by lines 2 and 3 above, followed by another send and receive pair triggered by lines 5 and 6, during a couple of successful calls to _dispatch_to_kernel when starting the channels. However, the next call, which has silent=True fails because the kernel.dispatch_shell function doesn't send a message in line 5 so the queue is empty in line 6. I don't know if the silent option is the key difference. It might be a problem with using a different handler in dispatch_shell.

I am happy to try and extract whatever information is helpful from my debugger, but I don't know enough about the message passing machinery to know what to look for.

[rayosborn]

I've discovered that the problem is with the InProcessKernel execute_request function. The process never gets through the _redirected_io contextual menu, so the parent execute_request function never gets called. Apart from using async, there is no difference with the old version so I'm not sure what is going on. Has there been a change in how the temporary stdout and stdin are defined?

I should have made clear from the outset that my code has no problems with ipykernel v5.5. This is purely a v6 issue.

[rayosborn]

After a lot of time in the debugger, I now believe that the underlying problem is due to changes in the OutStream class. In particular, the initialization of self.stdout and self.stderr in the InProcessKernel is prevented by an exception in the OutStream function _setup_stream_redirects(self, name). I was able to eliminate the problem by subclassing OutStream and overriding _setup_stream_redirects.

In inprocess/ipkernel.py, I made the following changes to the end of the InProcessKernel class, and added a new InProcessOutStream class.

    @default('stdout')
    def _default_stdout(self):
        return InProcessOutStream(self.session, self.iopub_thread, 'stdout')

    @default('stderr')
    def _default_stderr(self):
        return InProcessOutStream(self.session, self.iopub_thread, 'stderr')


class InProcessOutStream(OutStream):

    def _setup_stream_redirects(self, name):
        pass

After this, self.stdout and self.stdin are correctly initialized and execute_request works. I'm not sure this is ready for a PR, since I don't know what the purpose of the stream redirects are, but they don't seem required by the InProcessKernel.

I have renamed the issue accordingly.

[rayosborn]

A better fix than in the previous comment is just to add the keyword argument watchfd=False to the initialization of the OUtStream for stdout and stderr, i.e.,

    @default('stdout')
    def _default_stdout(self):
        return OutStream(self.session, self.iopub_thread, 'stdout', 
                         watchfd=False)

    @default('stderr')
    def _default_stderr(self):
        return OutStream(self.session, self.iopub_thread, 'stderr', 
                         watchfd=False)

I will submit a PR with this change.