Skip to content

feat(generator): skip duplicate books when Sefaria version exists#41

Merged
kdroidFilter merged 1 commit intodevfrom
feat/sefaria-priority-dedup
Jan 24, 2026
Merged

feat(generator): skip duplicate books when Sefaria version exists#41
kdroidFilter merged 1 commit intodevfrom
feat/sefaria-priority-dedup

Conversation

@kdroidFilter
Copy link
Copy Markdown
Owner

@kdroidFilter kdroidFilter commented Jan 24, 2026

Summary

  • Add deduplication logic in Otzaria generator to prioritize Sefaria source
  • When a book with the same heRef already exists from Sefaria, skip insertion from other sources

Changes

  • Check for existing book by heRef before inserting in Otzaria generator
  • If existing book is from Sefaria source, skip the duplicate

Test plan

  • Build SeforimLibrary: ./gradlew :SeforimLibrary:build
  • Regenerate database and verify no duplicates from Sefaria exist

Add deduplication logic in Otzaria generator to skip books that already
exist from Sefaria source. Sefaria has priority over other sources.
@kdroidFilter kdroidFilter force-pushed the feat/sefaria-priority-dedup branch from e2b77a4 to 3b4ed71 Compare January 24, 2026 20:04
@kdroidFilter kdroidFilter marked this pull request as ready for review January 24, 2026 20:14
@kdroidFilter kdroidFilter merged commit 2425b1b into dev Jan 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant