Skip to content

Conversation

@j-muldoon
Copy link
Contributor

When .wdb.writedownmode is set to `partbyfirstchar, there was a writedown happening for each unique sym, but the intended action was a writedown for each unique first character of syms.

This commit addresses this by grouping the syms by first letter, then altering the where clause in the functional select to select the corresponding syms before upserting to disk.

The number of upserts now equals the number of distinct starting characters.

@j-muldoon j-muldoon force-pushed the fixto_partbyinitialchar branch from 338c000 to 755fc59 Compare November 20, 2025 10:23
Co-authored-by: Ryan Cheale <ryan.cheale@dataintellect.com>
@j-muldoon j-muldoon force-pushed the fixto_partbyinitialchar branch from 755fc59 to af9fb4a Compare November 20, 2025 10:28
Comment on lines 173 to 174
if[writedownmode~`partbyfirstchar;
extrapartitions:value extrapartitions group .Q.fu[{first each string x}; raze extrapartitions]];
Copy link
Member

@jonathonmcmurray jonathonmcmurray Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this results in a nested list of symbols, if we can pass a simple list of symbols I think we don't need the changes to upserttopartition so maybe:

Suggested change
if[writedownmode~`partbyfirstchar;
extrapartitions:value extrapartitions group .Q.fu[{first each string x}; raze extrapartitions]];
if[writedownmode~`partbyfirstchar;
extrapartitions:raze extrapartitions;
extrapartitions:value extrapartitions group .Q.fu[{first each string x};extrapartitions]];

or we make an alternative to .merge.getextrapartitions that returns a list instead of nested list & then we don't have to raze, e.g. something like (untested):

.merge.getfirstcharpartitions:{[tablename;extrapartitiontype]
  value ?[tablename;();({first each string x};extrapartitiontype);(distinct;extrapartitiontype)]
 }

and then we can just do something like

extrapartitions:$[writedownmode~`partbyfirstchar;.merge.getfirstcharpartitions;.merge.getextrapartitions][tablename;extrapartitiontype];

and we shouldn't have to muck around with the extrapartitions variable afterwards

Copy link
Member

@jonathonmcmurray jonathonmcmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - if the other alternative works we can merge as a separate PR later

@ryanCheale ryanCheale merged commit a7f9610 into master Nov 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants