[Python] StreamWriter fails to open with socket as sink

[asfimport]

Trying to use pyarrow StreamWriter to write to a socket fails. I've tried the following:

• Use socket directly and get "TypeError: Unable to read from object of type: <class 'socket._socketobject'>"
  looks like pyarrow expects the object to have a "write" attribute

• Using as a file object with: "sock.makefile("wb", 65536)" get the error "pyarrow.error.ArrowIOError: IOError: '_fileobject' object has no attribute 'tell'"
  Not sure why exactly "tell" is required.

• using "os.fdopen(sock.fileno(), "wb", 65536)" get a rather cryptic error that I've been unable to track down:
  "terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid"
  Seems to indicate some kind of std::string construction with a NULL, but I can't find that anywhere

Reporter: Bryan Cutler / @BryanCutler
Assignee: Bryan Cutler / @BryanCutler

Related issues:

• Arrow 0.3 release (blocks)

_{Note: This issue was originally created as ARROW-822. Please see the migration documentation for further details.}

[asfimport]

Bryan Cutler / @BryanCutler:
Some code to reproduce, if needed

    import pyarrow as pa
    import socket, os, io

    port = 8080

    for res in socket.getaddrinfo("localhost", port, socket.AF_UNSPEC, socket.SOCK_STREAM):
        af, socktype, proto, canonname, sa = res
        sock = socket.socket(af, socktype, proto)
        try:
            sock.settimeout(3)
            sock.connect(sa)
        except socket.error:
            sock.close()
            sock = None
            continue
        break
    if not sock:
        raise Exception("could not open socket")
    
    #sink = socket
    sink = sock.makefile("wb", 65536)
    #sink = os.fdopen(sock.fileno(), "wb", 65536)
    #sink = os.fdopen(os.dup(sock.fileno()), "wb", 65536)

    field = pa.Field.from_py("f1", pa.float32())
    schema = pa.Schema.from_fields([field])

    writer = pa.StreamWriter(sink, schema)

[asfimport]

Bryan Cutler / @BryanCutler:
@wesm not sure if there is something I'm doing or not, but if this is an issue, it would be great to get into 0.3. I'm happy to work on this too if you are swamped with other things.

[asfimport]

Wes McKinney / @wesm:
I agree this should work and go into 0.3. It looks like you'll need to add a compatibility class for Python file-ish objects (that you can write to, but don't implement the full Python file API, like the tell method) – the arrow::io::OutputStream API expects to be able to report the stream position, so you can have it start at 0 and keep track of how many bytes have been written.

[asfimport]

Wes McKinney / @wesm:
see https://github.com/apache/arrow/blob/master/cpp/src/arrow/python/io.cc#L192

[asfimport]

Bryan Cutler / @BryanCutler:
Thanks @wesm, I'll start working on it today then

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 569
#569