Skip to content

Backlinks: preserve source order across same-target links#692

Merged
srid merged 9 commits intomasterfrom
fit-belt
Apr 30, 2026
Merged

Backlinks: preserve source order across same-target links#692
srid merged 9 commits intomasterfrom
fit-belt

Conversation

@srid
Copy link
Copy Markdown
Owner

@srid srid commented Apr 30, 2026

Backlink context cards from the same source note now render in source order, the order their links appear in the underlying .md file. Previously, when one note linked to a target several times, the cards were sorted lexicographically by surrounding block content (whatever fell out of Ord [B.Block]), and two links sharing one paragraph ([[Foo]] and [[Foo]]) collapsed into a single card under IxSet.fromList's set-dedup. Closes #186.

What broke and where

Bar.md ──▶ noteRels ──▶ Ix.fromList ──▶ modelRels (IxSet)
                                            │
                                            ▼
                                         backlinkRels (Ix.toList)
                                            │
                                            ▼
                                         modelLookupBacklinks (groupNE)
                                            │
                                            ▼
                                         backlinks.tpl

Two failures both rooted in Rel's derived Ord = (relFrom, relTo, relCtx):

Symptom Cause
Cards out of order Same (relFrom, relTo) → tie-break on Ord [B.Block] (lex on Pandoc inlines)
Duplicate-context cards collapsed Same (relFrom, relTo, relCtx) → equal under Eq → dropped by IxSet.fromList's set-dedup

The fix

Add _relSrcPos :: !Int between _relTo and _relCtx. The derived Ord then breaks ties between same-source-target rels by source position before falling back to lexicographic block order, and identical-context rels are no longer equal so the set-dedup leaves them alone. noteRels populates _relSrcPos via zipWith [0 ..] over the flattened per-note pandoc-link list.

Subtle: Text.Pandoc.LinkContext.queryLinksWithContext accumulates per-URL contexts via Map.fromListWith (<>), whose combiner runs as f new old — each later traversal entry is prepended, yielding the per-URL NonEmpty in reverse source order. The first cut of this PR missed that and the regression test caught it; noteRels now reverses the per-URL list back to forward order before zipping.

Tests

Two regressions in Emanote.Model.Link.RelSpec:

  • Source-order over lex-order — Z-paragraph (sorts last alphabetically) appears first in source; A-paragraph (sorts first) appears second. Without the tie-breaker, Ix.toList would yield A-then-Z; the assertion checks Z-then-A.
  • Identical-context dedup — one paragraph mentions Foo.md twice; length (Ix.toList (noteRels note)) must be 2.

Commit history

The branch traces a structural-review arc rather than a single squash:

Commit Purpose
01e85710 Primary fix — add field, rewrite noteRels
4538fd07 f40e8265 49e06936 1dab2e51 Hickey/Lowy refinements: narrow haddock, presentation-layer framing, field-order constraint, iteration-order note
d5dff3ab /code-police fact-check — caught the Map.fromListWith reversal bug
7a875c48 0e96b839 5682953c /code-police elegance — WL.plainify in tests, comment dedup, !Int strictness

Three follow-up issues filed for deferred structural critiques: #689 (newtype SrcPos wrapper), #690 (reusable WithSourcePos pattern when a second instance appears), #691 (consider relocating presentation order to modelLookupBacklinks).

Try it locally

nix run github:srid/emanote/fit-belt

Generated by /do on Claude Code (model claude-opus-4-7).

srid added 9 commits April 30, 2026 08:17
When one source note linked to a target several times, the backlinks UI
sorted those individual context cards lexicographically by their
surrounding block content (the derived `Ord [B.Block]` tie-breaker on
`Rel`) instead of by source-file order, and two links sharing an
identical context were collapsed into one card by `IxSet.fromList`'s
set-dedup.

Add a `_relSrcPos :: Int` field on `Rel` between `_relTo` and `_relCtx`,
populated in `noteRels` as the pandoc-traversal index. The derived `Ord`
now breaks ties between rels sharing `(_relFrom, _relTo)` by source
position, so source order survives `IxSet.toList` and the `groupNE`
aggregation in `modelLookupBacklinks` — and same-context multi-links
no longer collide on equality.

Two regression tests cover the pair of guarantees: lex-vs-source order
disagreement, and identical-context dedup.
…variant

Original wording — "Pandoc-traversal index of this link in the source
note" — overstated what the field actually guarantees. The flattening in
noteRels visits URL keys in Map order (alphabetical), not pandoc
traversal order; only within a single (relFrom, relTo) bucket does the
index reflect the order in which queryLinksWithContext yielded the
contexts. Narrow the haddock so a future reader cannot infer
document-wide source position from this field.
The field exists to drive backlink-card render order in
modelLookupBacklinks; it carries no cross-note graph-semantics meaning.
Document the boundary so future readers know other Rel consumers
(Resolve, Export.JSON, Graph.folgezettel*) intentionally do not depend
on it and so the field is not mistaken for part of the relation's
identity.
The Ord precedence (relFrom > relTo > relSrcPos > relCtx) is implied
purely by field declaration order, with no type-level enforcement. A
future field reorder for readability would silently break the
backlinks-source-order invariant — RelSpec catches this, but a comment
makes the intent legible at the site of the trap rather than only in
the test that closes it.
…oteRels

The fix's correctness for issue #186 hinges on the NonEmpty inside
queryLinksWithContext's per-URL value carrying instances in pandoc
traversal order. That assumption is upstream behavior — make it explicit
at the call site so a future maintainer who suspects the ordering bug
has resurfaced reads what we depend on without spelunking the
pandoc-link-context source.
…istWith reversal

The initial issue-#186 fix assumed queryLinksWithContext's per-URL
NonEmpty was in source-traversal order. It is not. The library
accumulates entries via Map.fromListWith (<>), whose combining function
is applied as f new old, so each later-traversed entry is prepended
onto the accumulator — yielding the per-URL NonEmpty in reverse
traversal order. zipWith [0 ..] over that list assigned srcPos to
later-source links first, so Ix.toList returned them out of order.

The new RelSpec test for source order caught it on the first run
(expected ["Z first: ", "A second: "], got ["A second: ", "Z first: "]).
Reverse the per-URL instances list before zipping so srcPos counts up
in source order.
…llbacks

The test's hand-rolled paragraph-text extractor had a "??" sentinel
fallback that would have turned a structural-shape regression into a
silent string-equality mismatch. Switch to the project's existing
'WL.plainify :: [B.Inline] -> Text' helper (used 6+ places in src,
e.g. Note.hs:489) and 'error' on unexpected ctx shape, and tighten
barRoute's failure message so a future fixture-invariant break names
itself.

The new assertion also reads the link label, so a regression that
preserves prefix text but corrupts the wikilink itself would still
fail.
…Rels

The hickey/lowy refinement commits added a 6-line NOTE block above
'data Rel', a 17-line haddock on '_relSrcPos', and a 16-line inline
comment above 'extractLinks' — three places explaining variations of
the same "Ord field-position breaks ties for issue #186" story, each
re-deriving the across-URL alphabetical caveat. Trim them so each
location says one thing once: the field-order invariant lives above
'data Rel' (3 lines), the field's contract lives in its haddock (8
lines), and the 'Map.fromListWith' reversal explanation lives at the
implementation site that needs it (5 lines). Net: ~28 lines removed,
no information lost.
The 'zipWith [0 ..]' lazy-list source means each rel currently carries
a thunk for its position field. Adding a bang annotation forces the
Int at construction time, dropping one thunk per rel from the global
'IxSet RelIxs Rel'. No call site reads '_relSrcPos' lazily — it only
participates in derived 'Ord' comparisons, which always force the
field anyway.
@srid
Copy link
Copy Markdown
Owner Author

srid commented Apr 30, 2026

Hickey/Lowy Analysis

# Lens Finding Disposition
1 Hickey _relSrcPos haddock overclaims "Pandoc-traversal index"; actual invariant is per-(from,to) group Fixed in this PR (4538fd0)
2 Hickey Field-order coupling in derived Ord is fragile against future reorders Fixed in this PR (49e0693)
3 Hickey _relSrcPos :: Int could leak as document-wide position; a newtype SrcPos would harden encapsulation Deferred #689
4 Hickey LC.queryLinksWithContext per-URL iteration assumption is undocumented at the call site Fixed in this PR (1dab2e5)
5 Lowy _relSrcPos is presentation-layer; comment should call out that other Rel consumers do not depend on it Fixed in this PR (f40e826)
6 Lowy Dedup opt-out under IxSet.fromList should be documented as intentional No-op — already in initial haddock
7 Lowy "Tie-break by source position" pattern is one-off; if a second pandoc-extracted type needs it, factor a WithSourcePos wrapper Deferred #690
8 Lowy Architectural alternative: move _relSrcPos out of Rel into modelLookupBacklinks's return type Deferred #691

Hickey rationale

The diff was reviewed at three angles: complecting, fragility, and token-vs-data.

Complecting (no finding). _relSrcPos looks like it mixes "rel identity" with "presentation order", but the apparent symmetry is false. The immediate problem is deduplication, not ordering — IxSet.fromList collapses two same-(from, to, ctx) rels into one, erasing the second link before any presentation layer sees it. The field is load-bearing at the storage layer (it makes equal-by-content rels distinct); the source-order benefit is a downstream consequence of having made them distinct. Sorting in modelLookupBacklinks instead of adding the field would solve the ordering half but leave the dedup half unfixed.

Fragility (Finding 2 fixed). The derived Ord for Rel compares fields in declaration order. The invariant relies on _relSrcPos preceding _relCtx, which is positional, not structural — a future field reorder would silently break source ordering with no compile error. The existing regression test catches this, and the field's haddock plus a NOTE block above the data declaration now make the dependency legible at the trap.

Token vs data (Finding 3 deferred). _relSrcPos's value is "the index in the flattened pandoc-link-context output for this note", not a domain fact. It carries no business meaning. The token is well-encapsulated today (not serialized, not indexed in IxSet, no external readers), so the encapsulation risk is residual. A newtype SrcPos = SrcPos Int would close the gap by making the type uninformative — filed as #689.

Hidden dependency (Finding 1 + 4 fixed). The original haddock claimed "Pandoc-traversal index of this link in the source note" — implying document-wide source position. The actual invariant is narrower: within rels sharing (_relFrom, _relTo), _relSrcPos reflects the order in which queryLinksWithContext yielded their contexts. Across distinct _relTo values the index is alphabetical-by-URL because of Map.toList, not document-wide source order. The haddock now states this precisely, and the noteRels site comments document the upstream iteration assumption.

Lowy rationale

Volatility-based decomposition asks: which axes of change does this fix encapsulate, and at which boundary?

Volatility map. Two independent axes are at play: cross-note graph semantics (a relation is a triple (from, to, context)) and presentation order (the order context cards render). The fix couples them inside Rel by adding a field that exists only to drive UI order. The asymmetry with Calendar.backlinkSortKey is instructive — that ordering decision lives in modelLookupBacklinks, the presentation tier; this one now lives in the graph type.

Why the storage-layer site is acceptable here (Finding 8 deferred). The architectural alternative — extracting _relSrcPos into modelLookupBacklinks's return type or building a parallel Map (relFrom, relTo, relCtx) -> srcPos — runs into the dedup constraint Hickey called out. IxSet.fromList's set-dedup is at the storage layer, so the discriminant has to be at the storage layer too, otherwise the second link is gone before any sort can run. Both alternatives are materially more complex than the field. The field is the pragmatic site; the boundary is shaped around the implementation reality (set-dedup discriminant) rather than the volatility axis (presentation order). Filed as #691.

Encapsulation hardening (Finding 5 + 6). Two documentation gaps: nothing said the field was a presentation-layer artifact (so a future reader might mistake it for graph state and propagate it through Resolve / Export.JSON), and nothing said the dedup opt-out was intentional (so a future reader might "tighten" Eq and silently re-introduce the collapse). The first is now in the field's haddock; the second was already in the initial haddock copy.

Reuse (Finding 7 deferred). The zipWith [0 ..] pattern is one instance of "tie-break by source position". If a second pandoc-extracted type ever needs the same trick, factoring WithSourcePos a would beat re-discovering the pattern. Premature today — filed as #690.

@srid
Copy link
Copy Markdown
Owner Author

srid commented Apr 30, 2026

Evidence

Before (master) After (this PR)
before after

Bar.md links to Foo three times in source order Zeta → Mu → Alpha. On master the backlink context cards render in lexicographic block order (Alpha → Mu → Zeta) — visible in the Before screenshot. With this PR the cards render in source order (Zeta → Mu → Alpha), matching how the paragraphs actually appear in Bar.md — visible in the After screenshot.

@srid
Copy link
Copy Markdown
Owner Author

srid commented Apr 30, 2026

/do results

Step Status Duration Verification
sync 0s git fetch ok; forge=github; noGit=false
research 9m 24s Bug isolated to Rel's derived Ord tie-breaking on Ord [B.Block].
branch 14s On feature branch fit-belt (worktree), up-to-date with origin/master.
implement 1m 38s Added _relSrcPos :: Int; updated noteRels with zipWith [0 ..]; two new RelSpec tests.
check 52s cabal build all clean (60 modules + executable).
docs 23s CHANGELOG entry under 1.6.0.0 (Unreleased) → Bug fixes.
fmt 26s just fmt clean (cabal-fmt, fourmolu, hlint, nixpkgs-fmt).
commit 18s Primary commit 01e85710 pushed; upstream set.
hickey+lowy 9m 9s 4 fix-in-PR commits + 3 deferred issues (#689, #690, #691).
police 13m 2s Rules clean; fact-check caught Map.fromListWith reversal bug; elegance landed 3 commits via /simplify.
test 23s cabal test all 49/49 pass.
create-pr 2m 3s Draft PR #692 + Hickey/Lowy ledger comment.
ci 2m 16s vira ci signed off both arches; e2e-{live,static,morph} green at HEAD.
evidence 3m 26s Before/after screenshots posted under ## Evidence.
Total 43m 49s

Slowest step: police (13m 2s, ~30% of total).

Optimization suggestions

  • police was the slowest because the fact-check pass caught a real bug (the Map.fromListWith (<>) reversal) requiring an extra commit + test re-run, and elegance ran three sub-fix cycles each with their own fmt + test. That is the right outcome — the bug would have shipped otherwise — so the time was load-bearing, not waste.
  • research (9m 24s) and hickey+lowy (9m 9s) are sub-agent-driven and dominate after police. For follow-up commits to this PR, --from polish or --from ci-only skips both.
  • The Map.fromListWith reversal is non-obvious enough that a one-line property-style test ("noteRels indices count up with source order") would have caught it during implement rather than police. Filing as a small RelSpec extension would shorten future fact-check on adjacent rel-ordering work.

Workflow completed at 2026-04-30.

@srid srid marked this pull request as ready for review April 30, 2026 13:29
@srid srid merged commit 682955c into master Apr 30, 2026
6 checks passed
@srid srid deleted the fit-belt branch April 30, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Individual backlinks not in right order

1 participant