st2 python3 interpreter is falling back to importing a python2 lib that is incompatible with python3

[mjtice]

SUMMARY

One of my libraries is trying to import cassandra.cluster. That library is trying to import from concurrent.futures import ThreadPoolExecutor, FIRST_COMPLETED, wait as wait_futures - which, as I understand, is built into python3 (and a separate library for compatibility with python3)

ISSUE TYPE

• Bug Report

STACKSTORM VERSION

# st2 --version
st2 3.0.0, on Python 2.7.6

OS / ENVIRONMENT / INSTALL METHOD

Docker container running Ubuntu 14.04

STEPS TO REPRODUCE

Create a python runner in your python3-specific pack. Inside the runner import cassandra libs and just create an object.

from cassandra.cluster import Cluster
        cluster = Cluster()

EXPECTED RESULTS

I expect the library to import and the object to initialize

ACTUAL RESULTS

st2 python3 falls back to python2 to import the lib and it throws an exception similar to

  File \"/opt/stackstorm/packs/ostk_common/actions/lib/ostkdbs.py\", line 2, in <module>
    from cassandra.cluster import Cluster
  File \"/opt/stackstorm/virtualenvs/ostk_common/lib/python3.5/site-packages/cassandra/cluster.py\", line 23, in <module>
    from concurrent.futures import ThreadPoolExecutor, FIRST_COMPLETED, wait as wait_futures
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/concurrent/futures/__init__.py\", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/concurrent/futures/_base.py\", line 414
    raise exception_type, self._exception, self._traceback
                        ^
SyntaxError: invalid syntax

[Kami]

Thanks for reporting this.

It looks like you installed a pack using --python3 flag?

I belive its related to #4654 (as per my comments there, there are a lot of edge cases in mixed Python 2 and 3 environments and that's why we plan to deprecate --python3 flag in the future when Ubuntu Bionic packages using Python 3 go GA).

[Kami]

I will look into this issue this week, so I would also appreciate if you can share the offending pack with us.

At the very least, if you can't share the whole pack, please share at least requirements.txt file and action in question.

[Kami]

I managed to reproduce and track down the root cause for this.

It's related to the PYTHONPATH we set for Python 3 actions. We include python3.x lib and site-packages directory from pack virtual environment, but we don't include global one (e.g. /usr/lib/python3.5) so it tries to import concurrent package from Python 2.7 site-packages instead of from global Python 3.x lib directory.

I'm working on a fix and end to end test cases which will catch such issues in the future.

[Kami]

#4666 should fix this (I still need to work on end to end tests to ensure it correctly covers all the scenarios).

Thanks for reporting this again.