-
Notifications
You must be signed in to change notification settings - Fork 850
Closed
Description
We create all chunks in memory as 1K long, and write the full 1K to store every time, even though many chunks will be half-full or less so we are writing a lot of trailing zero bytes.
AWS then base64-encodes all those zeros.
There doesn't seem to be an API on Prometheus (v1) chunks to find out how many bytes are used, but we could simply look for the last non-zero byte in the Marshal() output. We already include the data length in Chunk.Encode().
It's not clear to me whether we need to add back all the zeros when we read back the data, for double-delta encoding.
Slightly related: #300
Reactions are currently unavailable