Conversation
Replace mutable flat Yin/Yang files with content-addressed delta blobs tracked by a signed append-only object ID log. Introduces add-zettel-ids-yin, add-zettel-ids-yang, and migrate-zettel-ids commands. Uses horizontal versioning (v0=flat files, v1=logged) with box-format signed entries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
11-task plan covering type registration, directory layout, log entry types, provider loading, migrate-zettel-ids, add-zettel-ids-yin/yang commands, genesis changes, and test updates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AppendEntry opens the log file in create-or-append mode and encodes a single typed blob entry using the triple_hyphen_io coder. ReadAllEntries decodes all entries sequentially with a shared bufio.Reader, returning an empty slice when the file does not exist. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NewFromLog replays object ID log entries to reconstruct yin/yang providers from delta blobs. A BlobResolver function parameter allows the caller at a higher layer to provide blob access. When the log does not exist or is empty, falls back to reading flat Yin/Yang files via the existing New function. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reads existing flat Yin/Yang provider files, writes them as content-addressed blobs, and appends V1 entries to the object ID log. Idempotent — skips migration if log already contains entries. Also renames DirObjectIdLog to FileObjectIdLog (it's a file path, not a directory), removes it from DirsGenesis, and fixes multi-entry log parsing by pre-segmenting entries before decoding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These commands read text from stdin, extract unique word components, filter against existing words in both Yin and Yang pools (cross-side uniqueness), write new words as content-addressed blobs, append log entries, and update flat file caches. The repo file lock is held during the critical section (log append + flat file update). Pool size output reports the cartesian product (Yin * Yang) which represents the total number of allocatable zettel IDs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Genesis now processes raw text input through ExtractUniqueComponents, enforces cross-side uniqueness, applies Clean for provider consistency, writes word blobs to the blob store, and appends V1 log entries alongside flat file caches. Also fixes a debug print in Tai.MarshalText that leaked TAI timestamps to stderr, and updates test fixtures and helpers for the new NATO-phonetic word lists (alpha/golf instead of one/uno). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update 31 test files to reflect the new NATO phonetic word-based zettel ID allocation (alpha/golf, alpha/hotel, bravo/golf, etc.) replacing the old numeric word lists (one/uno, one/dos, two/uno, etc.). Fix sort order in organize and workspace tests where golf < hotel differs from the old dos < uno ordering. Update abbreviated ID references (o/u -> a/g) and directory name assertions (one/ -> alpha/). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch add-zettel-ids and migrate-zettel-ids to the codebase's req.Must(errors.MakeFuncContextFromFuncErr(...)) pattern for locksmith lock/unlock instead of manual error checking. Add missing repo lock to migrate-zettel-ids. Also add early return after ContextCancelWithError in readAndExtractCandidates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
friedenberg
left a comment
There was a problem hiding this comment.
lots of code duplication and style issues. but most importantly, the strategy for test fixture updating was a mistake: the fixture yin and yang files should not have been changed, instead dedicated tests that initalize / add zettel ids with new data should've been added.
| "code.linenisgreat.com/dodder/go/src/charlie/files" | ||
| "code.linenisgreat.com/dodder/go/src/echo/ids" | ||
| "code.linenisgreat.com/dodder/go/src/foxtrot/triple_hyphen_io" | ||
| ) |
There was a problem hiding this comment.
this whole file could be restructured to be object-oriented: methods on a struct, rather than dangling functions
There was a problem hiding this comment.
use inventory_list_coders as an example
go/src/foxtrot/object_id_log/log.go
Outdated
|
|
||
| defer errors.DeferredCloser(&err, file) | ||
|
|
||
| segments, err := segmentEntries(bufio.NewReader(file)) |
There was a problem hiding this comment.
this should be using the pools.GetBufferedReader() pattern
go/src/foxtrot/object_id_log/log.go
Outdated
|
|
||
| if _, err = Coder.DecodeFrom( | ||
| &typedBlob, | ||
| strings.NewReader(segment), |
There was a problem hiding this comment.
this should be using the pools.GetStringReader() pattern
go/src/foxtrot/object_id_log/log.go
Outdated
| var current strings.Builder | ||
| boundaryCount := 0 | ||
|
|
||
| for { |
There was a problem hiding this comment.
we should extract the common *bufio.Reader forloop line-reading into the ohio package and expose it using the go iterator API
| } | ||
| } | ||
|
|
||
| func (env *Env) genesisObjectIds(bigBang BigBang) { |
There was a problem hiding this comment.
all of the functions used in this method should be rewritten as methods on *Env
| utility.AddCmd("migrate-zettel-ids", &MigrateZettelIds{}) | ||
| } | ||
|
|
||
| type MigrateZettelIds struct { |
There was a problem hiding this comment.
lots of code duplication with the existing methods added for zettel id handling
| # as those will point to the bats executable's location or the preprocessed file respectively | ||
| DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")" >/dev/null 2>&1 && pwd)" | ||
|
|
||
| cat_yin() ( |
There was a problem hiding this comment.
this was a serious mistake that required changing all the tests. Revert this change and all the modifications to zz-tests_bats/*.bats (except for the add_zettel_ids.bats file). Rather than changing the fixtures for everything, you can create a dedicated test fixture for add_zettel_ids.bats and init.bats.
zz-tests_bats/add_zettel_ids.bats
Outdated
| echo "a sentence about ceroplastes" | ||
| echo "another line about midtown" | ||
| echo "something about harbor" | ||
| } >"$input" |
There was a problem hiding this comment.
prefer heredocs / nowdocs over subshell echo chains
zz-tests_bats/add_zettel_ids.bats
Outdated
|
|
||
| run_dodder add-zettel-ids-yin <"$input" | ||
| assert_success | ||
| assert_output --partial "added 3 words to Yin" |
There was a problem hiding this comment.
prefer non --partial asserts when possible (the output for this is deterministic, so it's possible)
| [one/uno @blake2b256-gu738nunyrnsqukgqkuaau9zslu0fhwg4dgs9ltuyvnlp42wal8sdpn2hc !md "wow" tag] | ||
| [alpha/golf @blake2b256-gu738nunyrnsqukgqkuaau9zslu0fhwg4dgs9ltuyvnlp42wal8sdpn2hc !md "wow" tag] | ||
| EOM | ||
| } |
There was a problem hiding this comment.
rather than having changed the fixtures for all the tests, a new test in this file that performs initialization with custom yin / yang files would've been much simpler
fixup! test: update all BATS tests for NATO phonetic zettel IDs Revert all test changes - keep old word lists (one/two/three, uno/dos/tres). Only retain new subcommand entries in complete.bats. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing fixup! feat: update genesis to use object ID log with raw text processing Remove ExtractUniqueComponents from genesis - it filters short words, breaking existing word lists. Genesis reads word-per-line files directly. ExtractUniqueComponents is only for add-zettel-ids raw text input. Revert common.bash to original word lists (one/two/three, uno/dos/tres). Add dedicated NATO word init to add_zettel_ids.bats and init.bats. Regenerate fixtures with old word lists + new object_id_log. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: implement object ID log reader and writer Restructure log.go to OO: methods on Log struct instead of package functions. Use pool.GetBufferedReader/GetStringReader for reader recycling. Update all callers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing fixup! feat: update genesis to use object ID log with raw text processing Convert genesis helper functions to methods on *Env, use pool.GetBufferedReader instead of bufio.NewScanner, read file lines into maps instead of slices, iterate-write words individually instead of strings.Join. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add add-zettel-ids-yin and add-zettel-ids-yang commands Merge add_zettel_ids_yin.go and add_zettel_ids_yang.go into single add_zettel_ids.go. writeWordsAsBlob and appendWordsToFlatFile now cancel context instead of returning errors, use iterate-write instead of strings.Join. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add migrate-zettel-ids command Reduce duplication: loop over yin/yang sides instead of repeated blocks, merge countLines into writeFlatFileAsBlob (seek-based single file open), cancel context instead of returning errors, use pool.GetBufferedReader. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: support log-based provider loading with flat file fallback Rename object_id_provider package to zettel_id_provider. Updates directory, package declarations, import paths, and all qualified references across 10 files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: implement object ID log reader and writer Extract common *bufio.Reader line-reading loop into ohio.MakeLineSeqFromReader iterator. Update segmentEntries to use iterator, removing manual EOF handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing fixup! feat: update genesis to use object ID log with raw text processing Use ohio.MakeLineSeqFromReader iterator in readAndCleanFileLines instead of manual ReadString/EOF loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add migrate-zettel-ids command Use ohio.MakeLineSeqFromReader iterator in writeFlatFileAsBlob instead of manual ReadString/EOF loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add add-zettel-ids-yin and add-zettel-ids-yang commands Use ohio.MakeLineSeqFromReader iterator in readAndExtractCandidates instead of manual ReadString/EOF loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing
fixup! feat: update genesis to use object ID log with raw text processing
Use order-preserving slices instead of maps in genesis to fix
non-deterministic word ordering. Change test NATO word entries from
multi-word ("the alpha") to single-word ("alpha") to match provider
storage format and fix cross-side rejection. Regenerate fixtures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add zettel id logs
No description provided.