[Python] Bug async api call producing "RuntimeError: cannot join current thread" #712
Description
Description
There's an Python program making some asynchronous API calls by passing async=True to the generated default API methods. Then an exception was thrown at the end of program(no such error if no async calls):
Exception ignored in: <bound method ApiClient.__del__ of <hpc_acm.api_client.ApiClient object at 0x7f5998219b70>>
Traceback (most recent call last):
File "/home/.../hpc_acm/api_client.py", line 79, in __del__
self.pool.join()
File "/usr/lib/python3.5/multiprocessing/pool.py", line 511, in join
self._task_handler.join()
File "/usr/lib/python3.5/threading.py", line 1051, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
openapi-generator version
3.1.2
OpenAPI declaration file content or url
https://raw.githubusercontent.com/EvanCui/hpc-acm/master/src/Swagger/swagger.yaml
Command line used for generation
java -jar ~/openapi-generator-cli.jar -l python -i swagger.yaml -o generated/
Steps to reproduce
I tried to reproduce the problem with a minimum code sample but haven't got it. But the debugging output clearly shows the problem rooted in src/main/resources/python/api_client.mustache
# The following lines are for my debug output
from multiprocessing import util
util.DEFAULT_LOGGING_FORMAT = '[%(levelname)s/%(threadName)s:%(thread)d] %(message)s'
util.log_to_stderr(util.DEBUG)
def __del__(self):
# The following two lines are for my debug output
from multiprocessing import util
util.debug('Api client is closing!')
self.pool.close()
self.pool.join() From the following debug output you can see del was executed on a thread which tried to join itself later, while in my minimum code sample it's on the main thread and then everything is OK. So we can assume that del is called by Python GC and can be run on a random thread besides the main thread.
[DEBUG/MainThread:140428243785472] added worker
[DEBUG/MainThread:140428243785472] added worker
...
[INFO/Dummy-6:140428243785472] process shutting down
[DEBUG/Dummy-6:140428243785472] running all "atexit" finalizers with priority >= 0
[DEBUG/Dummy-6:140428243785472] finalizing pool
[DEBUG/Dummy-6:140428243785472] helping task handler/workers to finish
[DEBUG/Dummy-6:140428243785472] joining worker handler
[DEBUG/Thread-1:140428186494720] worker got sentinel -- exiting
[DEBUG/Thread-1:140428186494720] worker exiting after 48 tasks
[DEBUG/Thread-5:140428152923904] result handler found thread._state=TERMINATE
[DEBUG/Thread-5:140428152923904] result handler exiting: len(cache)=0, thread._state=2
[DEBUG/Thread-2:140428178102016] worker got sentinel -- exiting
[DEBUG/Thread-2:140428178102016] worker exiting after 48 tasks
[DEBUG/Thread-3:140428169709312] worker handler exiting
[DEBUG/Thread-4:140428161316608] task handler got sentinel
[DEBUG/Dummy-6:140428243785472] joining task handler
[DEBUG/Thread-4:140428161316608] task handler sending sentinel to result handler
[DEBUG/Thread-4:140428161316608] task handler sending sentinel to workers
[DEBUG/Thread-4:140428161316608] task handler exiting
[DEBUG/Thread-4:140428161316608] Api client is closing! <-- This is my mark!
[DEBUG/Thread-4:140428161316608] closing pool
[DEBUG/Thread-4:140428161316608] joining pool
Exception ignored in: <bound method ApiClient.__del__ of <hpc_acm.api_client.ApiClient object at 0x7fb7fc518748>>
Traceback (most recent call last):
File "/home/.../api_client.py", line 82, in __del__
self.pool.join()
File "/usr/lib/python3.5/multiprocessing/pool.py", line 511, in join
self._task_handler.join()
File "/usr/lib/python3.5/threading.py", line 1051, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
[DEBUG/Dummy-6:140428243785472] joining result handler
[DEBUG/Dummy-6:140428243785472] running the remaining "atexit" finalizers
Related issues/PRs
Suggest a fix/enhancement
I suggest either try to ignore the exception, or make a normal "close" method to be called by user manually.