Handle missing fixtures better in unit tests.

[allmightyspiff]

Given this test

    def test_quick(self):
        result = self.run_command(['call-api', 'SoftLayer_Test', 'getTest'])
        self.assert_no_fail(result)

Running it will give this result

pytest tests/CLI/modules/vs/vs_create_tests.py
================================================================================================= test session starts =================================================================================================
platform darwin -- Python 3.7.0, pytest-3.8.1, py-1.6.0, pluggy-0.7.1
rootdir: /Users/christopher/Code/softlayer-python, inifile: setup.cfg
collected 26 items

tests/CLI/modules/vs/vs_create_tests.py .........................F                                                                                                                                              [100%]

====================================================================================================== FAILURES =======================================================================================================
_____________________________________________________________________________________________ VirtCreateTests.test_quick ______________________________________________________________________________________________
NOTE: Incompatible Exception Representation, displaying natively:

testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 265, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 265, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/christopher/Code/softlayer-python/SoftLayer/transports.py", line 215, in __call__
    proxies=_proxies_dict(self.proxy))
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/requests/sessions.py", line 524, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/requests/sessions.py", line 637, in send
    r = adapter.send(request, **kwargs)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/christopher/Code/softlayer-python/tests/CLI/modules/vs/vs_create_tests.py", line 620, in test_quick
    self.assert_no_fail(result)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/testing/__init__.py", line 152, in assert_no_fail
    raise result.exception
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/testing.py", line 326, in invoke
    cli.main(args=args or (), prog_name=prog_name, **extra)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/christopher/.virtualenvs/py3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/CLI/call_api.py", line 117, in cli
    result = env.client.call(*args, **kwargs)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/API.py", line 263, in call
    return self.transport(request)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/transports.py", line 462, in __call__
    raise call.exception
  File "/Users/christopher/Code/softlayer-python/SoftLayer/transports.py", line 451, in __call__
    call.result = self.transport(call)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/transports.py", line 496, in __call__
    result = self.transport(call)
  File "/Users/christopher/Code/softlayer-python/SoftLayer/transports.py", line 244, in __call__
    raise exceptions.TransportError(0, str(ex))
SoftLayer.exceptions.TransportError: TransportError(0): ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))


------------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------------

========================================================================================= 1 failed, 25 passed in 1.98 seconds =========================================================================================

Yikes. File "softlayer-python/SoftLayer/testing/__init__.py", line 152, in assert_no_fail
that should likely handle that error better. Ideally telling the user their isn't a fixture or something.

[pgajdos]

Probably this is an offtopic, but I am perhaps one of such user. While trying to run the testsuite during rpm package build, I am getting:

[   28s] =================================== FAILURES ===================================
[   28s] _______________________ TestHelpSetup.test_setup_cancel ________________________
[   28s] NOTE: Incompatible Exception Representation, displaying natively:
[   28s] 
[   28s] testtools.testresult.real._StringException: Traceback (most recent call last):
[   28s]   File "/usr/lib/python3.7/site-packages/mock/mock.py", line 1305, in patched
[   28s]     return func(*args, **keywargs)
[   28s]   File "/home/abuild/rpmbuild/BUILD/softlayer-python-5.7.1/tests/CLI/modules/config_tests.py", line 90, in test_setup_cancel
[   28s]     self.assertEqual(result.exit_code, 2)
[   28s]   File "/usr/lib/python3.7/site-packages/testtools/testcase.py", line 411, in assertEqual
[   28s]     self.assertThat(observed, matcher, message)
[   28s]   File "/usr/lib/python3.7/site-packages/testtools/testcase.py", line 498, in assertThat
[   28s]     raise mismatch_error
[   28s] testtools.matchers._impl.MismatchError: 1 != 2

Could you please help me and point me to a solution?

[allmightyspiff]

can you let me know what steps you ran to get to that state?

[pgajdos]

https://build.opensuse.org/public/build/home:pgajdos:fail/openSUSE_Tumbleweed/x86_64/python-softlayer/_log

Here is the build log, which should describe it exhaustively. Let me know if you need more info, though.
Thanks!

[allmightyspiff]

I opened #1131 for this, as it seems to be a bug with the test I think.

[pgajdos]

Thanks!