Skip to content

Further Model Size Reductions#137

Merged
josephbirkner merged 5 commits intonoserdefrom
feature/split-field-storage
Mar 16, 2026
Merged

Further Model Size Reductions#137
josephbirkner merged 5 commits intonoserdefrom
feature/split-field-storage

Conversation

@josephbirkner
Copy link
Collaborator

Changes:

  • Reduce ObjectField size from 8B to 6B (skip padding) by introducing TwoPart<First, Second> storage support for ModelColumn/ArrayArena.
  • ArrayArena can store fixed-size one-element arrays without chunk-data overhead. Saves ~10% on array/object/geometry storage.

Benchmark results:

branch                         big-model-query-perf (s) 
------                         ------------------------  
main                           147.77
v0.6.3                         135.53
noserde                        139.22
feature/singleton-arrays       137.75
feature/split-field-storage    139.55

@cursor
Copy link

cursor bot commented Mar 6, 2026

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on March 22.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

Test Results

 1 files  ±0   1 suites  ±0   6m 51s ⏱️ -1s
91 tests +3  91 ✅ +3  0 💤 ±0  0 ❌ ±0 
96 runs  +3  96 ✅ +3  0 💤 ±0  0 ❌ ±0 

Results for commit 9a3911b. ± Comparison against base commit e9c17b1.

♻️ This comment has been updated with latest results.

@johannes-wolf
Copy link
Collaborator

johannes-wolf commented Mar 6, 2026

How does TwoPart<A, B> (aka. std::pair?) reduce the size of structs? This is a lot of changes and it is hard to tell how the size-optimization happens.
Both fields of a TwoPart<A, B> are stored in separate containers to skip field padding.


Chunk& updatedLast = ensure_capacity_and_get_last_chunk(a);
#ifdef ARRAY_ARENA_THREAD_SAFE
std::shared_lock guard(lock_);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to acquire a write lock here.

return compactHeads_.has_value();
}

[[nodiscard]] bool isCompact() const {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this duplicate.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 9, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@github-actions
Copy link

github-actions bot commented Mar 9, 2026

Package Line Rate Branch Rate Health
include.simfil 24% 10%
include.simfil.model 75% 46%
src 74% 46%
src.model 82% 46%
Summary 45% (7692 / 17020) 27% (4717 / 17367)

@josephbirkner josephbirkner merged commit c80b031 into noserde Mar 16, 2026
5 of 6 checks passed
@josephbirkner josephbirkner deleted the feature/split-field-storage branch March 16, 2026 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants