feat(blob_v2): add Python API for Blob v2#5491
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
westonpace
left a comment
westonpace
left a comment
There was a problem hiding this comment.
This seems fine as input. However, I think the weirdest thing about blobs is going to be the fact that the output is not the same as the input. Right now the input is:
{
"data": large_binary,
"uri": utf8,
}
However, the output is the descriptions and the descriptions contain a number of fields (e.g. blob_id) which are internal details that don't make sense to the user. I'm wondering if we can, at some point, unify these two things. For example, when the user reads blob data, we can convert the blob_id, data file, etc. into a URI. So what if we add to the blob array position and length (with length=-1 meaning the whole file) and so the input and output are the same? The only difference is that the input might be a mix of data and uri but the output would always have data be null and uri be set?
| // Blob v2 columns are special: they can have different struct layouts | ||
| // (logical input vs. descriptor struct). We treat blob v2 structs as opaque | ||
| // during schema set operations (union/subtract). |
There was a problem hiding this comment.
I'm not sure I'd use the word "opaque" here. Maybe "we treat blob v2 structs as primitive fields (like a binary column) during schema set operations?
Seems like a nice idea to me, will create a follow up issue for this idea. |
This PR will expose blob v2 to python API, allow users to write blob v2 data. Well, I don't have much experience in designing Python APIs. @westonpace, could you please take a look at this shape? --- **Parts of this PR were drafted with assistance from Codex (with `gpt-5.2`) and fully reviewed and edited by me. I take full responsibility for all changes.**
This PR will expose blob v2 to python API, allow users to write blob v2 data. Well, I don't have much experience in designing Python APIs. @westonpace, could you please take a look at this shape? --- **Parts of this PR were drafted with assistance from Codex (with `gpt-5.2`) and fully reviewed and edited by me. I take full responsibility for all changes.**
2 participants
This PR will expose blob v2 to python API, allow users to write blob v2 data.
Well, I don't have much experience in designing Python APIs. @westonpace, could you please take a look at this shape?
Parts of this PR were drafted with assistance from Codex (with
gpt-5.2) and fully reviewed and edited by me. I take full responsibility for all changes.