Skip to content

Fix #187 by storing univariate_id as int64#198

Merged
skejserjensen merged 4 commits intomainfrom
dev/uid-as-int64
Aug 10, 2024
Merged

Fix #187 by storing univariate_id as int64#198
skejserjensen merged 4 commits intomainfrom
dev/uid-as-int64

Conversation

@skejserjensen
Copy link
Copy Markdown
Contributor

This PR fixes #187 by storing univariate_id as Int64Array instead of UInt64Array as the Delta Lake Protocol does not support unsigned integers and #197 probably will not be resolved for some time. When reviewing this PR, please double check the necessary casts are performed everywhere they are needed. Currently, UInt64Array is reinterpreted as Int64Array when writing compressed segments to disk and Int64Array is reinterpreted as UInt64Array when data points are reconstructed from segments and aggregates are computed directly from the segments.

@skejserjensen skejserjensen self-assigned this Aug 8, 2024
Comment thread crates/modelardb_common/src/storage.rs
Comment thread crates/modelardb_common/src/storage.rs
@skejserjensen skejserjensen requested a review from CGodiksen August 9, 2024 13:10
Comment thread crates/modelardb_common/src/storage.rs Outdated
Comment thread crates/modelardb_common/src/storage.rs
@skejserjensen skejserjensen requested a review from CGodiksen August 9, 2024 19:23
@skejserjensen skejserjensen merged commit 879c080 into main Aug 10, 2024
@skejserjensen skejserjensen deleted the dev/uid-as-int64 branch August 10, 2024 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some univariate_id cannot be written to Delta Lake

3 participants