Hublabel by electricEpilith · Pull Request #239 · vgteam/libbdsg

electricEpilith · 2026-04-07T21:38:16Z

libbdsg changes to go with merging the hublabel branch of vg

…it returns, and which is a different size on mac

…rs that validate it

… debugging

…ost_graph

…opefully query off the right orientations

…ng the net graph

…ses that all can run

…apes and cases

…rtices from a net graph child. Add a bunch of comments exlaining why I am confused by the distance index orientation bookkeeping.

…nd do it

…ord access attempts

…arlRecord

…t that instead of unpicking it

…ue root handle

…tion

…divine the correct way to use them

…istances

…ances Add O(1) get_snarl_child_count() that reads the stored count directly. Replace Dijkstra fallback for oversized snarl internal distances with hub label (HHL) queries. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

# Conflicts: # bdsg/include/bdsg/snarl_distance_index.hpp

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The merge of origin/hublabel-debug introduced duplicate declarations in the header (lines 439 and 550) and duplicate definitions in the .cpp (lines 631 and 648). Removed the older versions, keeping the ones from hublabel-debug which have better comments and slightly cleaner impl. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

adamnovak

It looks like this brings along more of the other oracle approaches than we need to to implement just the contraction hierarchies and hierarchical hub labeling that actually gets used.

There's also some commented-out code that should be removed, and some documentation comments that ought to be written, some of which are for things I added.

Also, I didn't add a file comment for it, but if it's easy I might port away from Boost to our own data structure for what's now called the "Boost graph", to avoid the dependency. Right now we have Boost in vg only because of the old Vowpal-Wabbit MAPQ recalibration stuff in vgteam/vg#1621 which I don't know actually has any users. If it's not easy it probably wouldn't be worth doing, or at least might not be worth doing now. (Since it's only needed for the construction, we can change later if we want to without invalidating indexes.)

adamnovak · 2026-04-08T18:45:11Z

+    /// Indexing into iterator. Even though we type this as reference, remember
+    /// that we don't actually implement writing to our "references" and just
+    /// use the value type.
+    /// Result is undefined if itrators are to different collecitons.


Suggested change

/// Result is undefined if itrators are to different collecitons.

/// Result is undefined if iterators are to different collections.

adamnovak · 2026-04-08T18:45:27Z

+    // Comaprable iterator methods (TODO: Is there an STL concept name for this?)
+
+    /// Determine if this iterator is strictly before another.
+    /// Result is undefined if iterators are to different collecitons.


Suggested change

/// Result is undefined if iterators are to different collecitons.

/// Result is undefined if iterators are to different collections.

adamnovak · 2026-04-08T18:45:37Z

+    bool operator<(const IndexingIterator& other) const;
+
+    /// Determine if this iterator is before or at another.
+    /// Result is undefined if iterators are to different collecitons.


Suggested change

/// Result is undefined if iterators are to different collecitons.

/// Result is undefined if iterators are to different collections.

adamnovak · 2026-04-08T18:45:49Z

+    bool operator<=(const IndexingIterator& other) const;
+
+    /// Determine if this iterator is strictly after another.
+    /// Result is undefined if iterators are to different collecitons.


Suggested change

/// Result is undefined if iterators are to different collecitons.

/// Result is undefined if iterators are to different collections.

adamnovak · 2026-04-08T18:46:03Z

+    bool operator>(const IndexingIterator& other) const;
+
+    /// Determine if this iterator is at or after another.
+    /// Result is undefined if itrators are to different collecitons.


Suggested change

/// Result is undefined if itrators are to different collecitons.

/// Result is undefined if iterators are to different collections.

adamnovak · 2026-04-09T21:57:48Z

+    // need to debug
+    for (size_t a = 0; a < handles.size() * 2; a++) {
+      cerr << hhl_query(packed_labels.begin(), rank(1, false), a) << endl;
+    }


This has been debugged now, right? We should probably not print stuff from the tests unless they're built with a define.

adamnovak · 2026-04-09T21:58:25Z

@@ -5089,6 +5393,7 @@ int main(void) {
    test_packed_subgraph_overlay();
    test_multithreaded_overlay_construction();
    test_mapped_packed_graph();
-    test_hash_graph();
-    test_snarl_distance_index();
+    test_hash_graph(); */
+    test_hub_labeling();
+    //test_snarl_distance_index();


We have to turn the other tests back on.

adamnovak · 2026-04-09T21:58:56Z


+    // We limit threading on small inputs.
+    auto limited_threads = [&](size_t batch) {
+        return std::max<size_t>(1, std::min<size_t>(batch / 1024, get_thread_count()));


I think I added this, but this could use a constant instead of a magic number.

adamnovak · 2026-04-09T22:00:25Z

 # Find other system dependencies
 pkg_check_modules(Jansson REQUIRED IMPORTED_TARGET jansson)

+find_package(Boost REQUIRED)


If we're adding a dependency on Boost, we need to explain that in the README.

adamnovak · 2026-04-09T22:01:04Z

 OBJS += $(OBJ_DIR)/subgraph_overlay.o
 OBJS += $(OBJ_DIR)/vectorizable_overlays.o 
 OBJS += $(OBJ_DIR)/packed_subgraph_overlay.o 
+OBJS += $(OBJ_DIR)/ch.o


So we don't actually need hublabel.o and landmark.o and we can drop their CPP files?

planned by Claude Opus 4.7

electricEpilith and others added 30 commits December 9, 2025 12:54

add hublabel build and query code, get tests to build

c48617d

replace placeholder getter stuff, rewrite hhlquery to take iterator

9a5bbd3

Merge remote-tracking branch 'upstream/master' into hublabel

28add14

debugging etc.

3e009fa

Change hub_labels to a size_t vector, which the function to generate …

6755f44

…it returns, and which is a different size on mac

Merge remote-tracking branch 'upstream/hublabel' into hublabel

37492aa

Define a type for the complex temporary X index indexing, and accesso…

0065e97

…rs that validate it

Use indexing accessors in make_boost_graph()

420c9e9

Add const accessors and fix build errors

5edfaa0

Write some hub labeling interface docs as I remember it, and add some…

b9ba500

… debugging

Add synthetic debug logging and synthetic analysis results to make_bo…

23e6745

…ost_graph

Synthesize code to allocate vertices and avoid wasting them, and to h…

ad43882

…opefully query off the right orientations

Redesign to only think of indexing inward-facing handles for converti…

0e6aa47

…ng the net graph

Teach CMake about Boost dependency

3619491

Stop putting negative-unsigned entries in the hub labels

7e33257

Define what to do for asking about a node to itself, and test it

9a88f8f

Qualify std::move calls to appease compiler

ac07468

Const-ify arguments to label building

3b84fb0

De-comment the existing hub labeling test cases into distinct test ca…

227e4d1

…ses that all can run

Add some synthetic unit tests, with revisions, for some more graph sh…

9b33554

…apes and cases

Fix signature mismatch

385efa8

Fix type name

9ae76d4

Merge remote-tracking branch 'origin/master' into hublabel

1d50a68

Rework HHL vertex ID computation so we can actually find the right ve…

fb774f2

…rtices from a net graph child. Add a bunch of comments exlaining why I am confused by the distance index orientation bookkeeping.

Be clear about who is responsible for initializing NodeProp fields, a…

a5c20a8

…nd do it

Produce more descriptive errors for out of bounds temporary snarl rec…

9a7e4c3

…ord access attempts

Fix cases where simple snarls could be accesed with the non-simple Sn…

a868f92

…arlRecord

Figure out ChainRecord is used to decode snarls sometimes and documen…

f8de1dc

…t that instead of unpicking it

Stop trying to report types we aren't allowed to compute

41ccdf3

Handle ROOT record type as also needing to canonicalize to the one tr…

a3948d4

…ue root handle

adamnovak and others added 23 commits February 2, 2026 14:24

Turn off debugging prints and some of the checks

178857d

Flag off more debug prints

8a85c23

Merge branch 'hublabel' into HEAD

e9ff343

Merge remote-tracking branch 'upstream/hublabel' into hublabel

9dc0326

Adjust debugging for better diffing and add a distance promotion func…

d429581

…tion

Turn on debugging and add more query prints

43a6bc7

Rename orientation flags and start collecting constraints on them to …

baa9b49

…divine the correct way to use them

Turn on debugging again and fix adding to infinite unreachable loop d…

ee08df2

…istances

Turn off debugging

f522ff9

Implement storing regularity in index via the type

badfab6

Fix non-Python build

1522fa9

Enable debugging and stop OR-ing in enum values

46aa094

Turn off debugging

efa47f2

Try not to run a jillion threads during tests

bb713f4

Count simple and trivial snarls as regular because zip codes do

72eefa7

Merge branch 'mergePlaceholder' into hublabel

0be8fda

# Conflicts: # bdsg/include/bdsg/snarl_distance_index.hpp

add back get_snarl_child_count

28d2d2b

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/hublabel-debug' into hublabel

4c70990

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

upgrade version number to 5

fdb9a74

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

upgrade version number

ea70e55

fix conflict from accidentally doing the same version upgrade twice

09f9d6e

adamnovak reviewed Apr 9, 2026

View reviewed changes

adamnovak and others added 6 commits April 22, 2026 17:11

Add CHOverlay output

2c3c0db

Address some of my own review comments

a7602fd

prevent sdsl conflicts

a169eca

planned by Claude Opus 4.7

prevent libhandlegraph conflicts

531e77e

planned by Claude Opus 4.7

prevent hopscotch conflicts

5aff1a5

planned by Claude Opus 4.7

prevent mio conflicts

789ad6d

planned by Claude Opus 4.7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hublabel#239

Hublabel#239
electricEpilith wants to merge 60 commits intomasterfrom
hublabel

electricEpilith commented Apr 7, 2026

Uh oh!

adamnovak left a comment

Uh oh!

adamnovak Apr 8, 2026

Uh oh!

adamnovak Apr 8, 2026

Uh oh!

adamnovak Apr 8, 2026

Uh oh!

adamnovak Apr 8, 2026

Uh oh!

adamnovak Apr 8, 2026

Uh oh!

adamnovak Apr 9, 2026

Uh oh!

adamnovak Apr 9, 2026

Uh oh!

adamnovak Apr 9, 2026

Uh oh!

adamnovak Apr 9, 2026

Uh oh!

adamnovak Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	/// Result is undefined if itrators are to different collecitons.
	/// Result is undefined if iterators are to different collections.

Conversation

electricEpilith commented Apr 7, 2026

Uh oh!

adamnovak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants