-
Notifications
You must be signed in to change notification settings - Fork 76
Closed
Description
Some memory corruption is happening in my use case for Blosc. Blosc is being used to store numpy ndarrays in shared memory using plasma. It is unclear whether the issue is in compression/decompression or in writing/reading to shared memory. Here are my write/read methods:
def write_blosc(client: plasma.PlasmaClient, x: np.ndarray):
object_id = plasma.ObjectID(np.random.bytes(20))
buf = blosc.pack_array(x)
client.create_and_seal(object_id, buf)
return object_id
def read_blosc(client: plasma.PlasmaClient, object_id: plasma.ObjectID):
[buf] = client.get_buffers([object_id], timeout_ms=100)
res = blosc.unpack_array(buf.to_pybytes())
return resNumpy arrays are able to be successfully read and written most of the time, but occasionally (maybe 1 call out of every 10,000 to the read function) the error in the title shows up.
Metadata
Metadata
Assignees
Labels
No labels