Host Whitelisting

[kevinkjt2000]

It seems that asynctest requires at least localhost sockets when I ran my test. Would it be possible to selectively enable certain sockets and not others? For example, here I would probably get past this if I could allow localhost sockets.

============================================== FAILURES ===============================================
______________________ TestBot.test__sends_error_message_when_connection_refused ______________________
venv/lib/python3.6/site-packages/asynctest/case.py:272: in run
    self._setUp()
venv/lib/python3.6/site-packages/asynctest/case.py:217: in _setUp
    self._init_loop()
venv/lib/python3.6/site-packages/asynctest/case.py:172: in _init_loop
    loop = self.loop = asyncio.new_event_loop()
/usr/lib64/python3.6/asyncio/events.py:688: in new_event_loop
    return get_event_loop_policy().new_event_loop()
/usr/lib64/python3.6/asyncio/events.py:599: in new_event_loop
    return self._loop_factory()
/usr/lib64/python3.6/asyncio/unix_events.py:56: in __init__
    super().__init__(selector)
/usr/lib64/python3.6/asyncio/selector_events.py:67: in __init__
    self._make_self_pipe()
/usr/lib64/python3.6/asyncio/selector_events.py:129: in _make_self_pipe
    self._ssock, self._csock = self._socketpair()
/usr/lib64/python3.6/asyncio/unix_events.py:60: in _socketpair
    return socket.socketpair()
/usr/lib64/python3.6/socket.py:489: in socketpair
    a = socket(family, type, proto, a.detach())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = (<AddressFamily.AF_UNIX: 1>, <SocketKind.SOCK_STREAM: 1>, 0, 11), kwargs = {}

    def guarded(*args, **kwargs):
>       raise SocketBlockedError()
E       pytest_socket.SocketBlockedError: A test tried to use socket.socket.

venv/lib/python3.6/site-packages/pytest_socket.py:56: SocketBlockedError
================================= 1 failed, 4 passed in 0.41 seconds ==================================
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=False debug=False>>
Traceback (most recent call last):
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 512, in __del__
    self.close()
  File "/usr/lib64/python3.6/asyncio/unix_events.py", line 63, in close
    super().close()
  File "/usr/lib64/python3.6/asyncio/selector_events.py", line 110, in close
    self._close_self_pipe()
  File "/usr/lib64/python3.6/asyncio/selector_events.py", line 120, in _close_self_pipe
    self._remove_reader(self._ssock.fileno())
AttributeError: '_UnixSelectorEventLoop' object has no attribute '_ssock'

To reproduce:
pytest.ini

[pytest]
addopts = --disable-socket

test.py

import asynctest

class TestClass(asynctest.TestCase):
    async def test_something(self):
        pass

commands to enter:

pip install pytest pytest-socket asynctest
pytest test.py

[miketheman]

Hi @kevinkjt2000 !

Thanks for the awesome report.
(minor note, test case should pass async def test_something(self): or I get a TypeError)

Right now, the granularity of the logic is very coarse - it's set at the test case level - so it's not set to distinguish a difference between types of sockets within a test case. So we can selectively enable socket use for the entire test case using fixtures, but then run the risk of the items within the async test case making network calls.

I also was curious about your use of asynctest vs pytest-asyncio - I was able to simplify the case to this code, allowing the async test to pass:

import pytest
import pytest_socket


@pytest.mark.asyncio
async def test_something(socket_enabled):
    pass

Again, this doesn't have the granularity of detecting what happens during the test case, as socket is now enabled.

I was thinking about how that might be implemented, and haven't come up with a decent design - as it would require evaluating within the test case any time socket is called to determine the AddressFamily in use.

[kevinkjt2000]

Well I could enable socket and add self to get the asynctest to pass too xD (I edited my original comment to include self)

As for how to design this, I would suggest some kind of lookup list in the fake disabled socket? If it is allowed, let it use real socket; otherwise, send it to fake socket? I am ignorant as to how blocking all sockets is implemented currently.

[sadams]

I'll have a go if no one else has started...

Before i go too far, @miketheman do you have any views on the following:

1. instead of overriding socket.socket we can intercept socket.socket.connect (to give us the host)
2. optionally provide a list of allowed hosts like this: disable_socket(['127.0.0.1']) or for cli --disable-socket 127.0.0.1,127.0.1.1 (alternatively we could create a separate method called disable_socket_connect(...) if that is better)
3. because of the above, i think ip addresses rather than hostnames will have to be used (i think connect gets the resolved IP address rather than the hostname). However, I can add something to do the docs showing how to use getaddrinfo to help with that.

Cheers

[miketheman]

Hey @sadams! Awesome that you're going for it.

For the questions asked, some thoughts.

1. If overriding the subset of the socket, are there other ways one might use it that don't directly use connect that would circumvent the disable? I guess a look at uses of socket across GitHub might be an indication, but not a perfect sample, since people do some crazy things behind closed repos.

2. The config interface is one I also am struggling with to get a clean approach, since the main desire is to disable sockets, and then another option to selectively enable for some hosts, unless you're thinking that the disable action would only disable the listed hosts?

3. What happens currently if a socket.gethostbyname is called? If DNS resolution works when disabling socket, then that could be a bug, since DNS resolution is a network call, and the point of this plugin was to disable them. So I'm not sure how the hostname/ip address should work just yet.

[sadams]

Hi @miketheman ,
Good point about socket being used in DNS resolution, but I think it's not using python-level sockets at all:

import socket

def foo():
    print('overridden')

socket.socket = foo

print(socket.gethostbyname('google.com'))
# 172.217.23.46

So that is 'technically' a bug in the original, but I think it's probably pretty rare that people are using this to avoid DNS side effects.

[sadams]

On the other points, i ended up doing the code, so i will submit a PR and we can discuss there (basically i added an optional argument to disable_sockets that was a whitelist of hosts).

[miketheman]

I may have closed this prematurely - the solution in #9 appears to work for connections with designated IP addresses, but the repro case here for unix sockets remains unsolved.

[kamikaze]

got same problem... allowing AF_UNIX sockets while disabling AF_INET could be nice to have