Skip to content

Claude zettel id component log#2

Open
friedenberg wants to merge 26 commits intomasterfrom
claude-zettel_id_component_log
Open

Claude zettel id component log#2
friedenberg wants to merge 26 commits intomasterfrom
claude-zettel_id_component_log

Conversation

@friedenberg
Copy link
Collaborator

No description provided.

friedenberg and others added 13 commits February 25, 2026 08:50
Replace mutable flat Yin/Yang files with content-addressed delta blobs
tracked by a signed append-only object ID log. Introduces add-zettel-ids-yin,
add-zettel-ids-yang, and migrate-zettel-ids commands. Uses horizontal
versioning (v0=flat files, v1=logged) with box-format signed entries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
11-task plan covering type registration, directory layout, log entry
types, provider loading, migrate-zettel-ids, add-zettel-ids-yin/yang
commands, genesis changes, and test updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AppendEntry opens the log file in create-or-append mode and encodes a
single typed blob entry using the triple_hyphen_io coder. ReadAllEntries
decodes all entries sequentially with a shared bufio.Reader, returning
an empty slice when the file does not exist.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NewFromLog replays object ID log entries to reconstruct yin/yang
providers from delta blobs. A BlobResolver function parameter allows
the caller at a higher layer to provide blob access. When the log
does not exist or is empty, falls back to reading flat Yin/Yang files
via the existing New function.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reads existing flat Yin/Yang provider files, writes them as content-addressed
blobs, and appends V1 entries to the object ID log. Idempotent — skips
migration if log already contains entries.

Also renames DirObjectIdLog to FileObjectIdLog (it's a file path, not a
directory), removes it from DirsGenesis, and fixes multi-entry log parsing
by pre-segmenting entries before decoding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These commands read text from stdin, extract unique word components,
filter against existing words in both Yin and Yang pools (cross-side
uniqueness), write new words as content-addressed blobs, append log
entries, and update flat file caches. The repo file lock is held
during the critical section (log append + flat file update).

Pool size output reports the cartesian product (Yin * Yang) which
represents the total number of allocatable zettel IDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Genesis now processes raw text input through ExtractUniqueComponents,
enforces cross-side uniqueness, applies Clean for provider consistency,
writes word blobs to the blob store, and appends V1 log entries alongside
flat file caches. Also fixes a debug print in Tai.MarshalText that leaked
TAI timestamps to stderr, and updates test fixtures and helpers for the
new NATO-phonetic word lists (alpha/golf instead of one/uno).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update 31 test files to reflect the new NATO phonetic word-based zettel
ID allocation (alpha/golf, alpha/hotel, bravo/golf, etc.) replacing the
old numeric word lists (one/uno, one/dos, two/uno, etc.). Fix sort order
in organize and workspace tests where golf < hotel differs from the old
dos < uno ordering. Update abbreviated ID references (o/u -> a/g) and
directory name assertions (one/ -> alpha/).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch add-zettel-ids and migrate-zettel-ids to the codebase's
req.Must(errors.MakeFuncContextFromFuncErr(...)) pattern for
locksmith lock/unlock instead of manual error checking. Add
missing repo lock to migrate-zettel-ids. Also add early return
after ContextCancelWithError in readAndExtractCandidates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Collaborator Author

@friedenberg friedenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lots of code duplication and style issues. but most importantly, the strategy for test fixture updating was a mistake: the fixture yin and yang files should not have been changed, instead dedicated tests that initalize / add zettel ids with new data should've been added.

"code.linenisgreat.com/dodder/go/src/charlie/files"
"code.linenisgreat.com/dodder/go/src/echo/ids"
"code.linenisgreat.com/dodder/go/src/foxtrot/triple_hyphen_io"
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this whole file could be restructured to be object-oriented: methods on a struct, rather than dangling functions

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use inventory_list_coders as an example


defer errors.DeferredCloser(&err, file)

segments, err := segmentEntries(bufio.NewReader(file))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be using the pools.GetBufferedReader() pattern


if _, err = Coder.DecodeFrom(
&typedBlob,
strings.NewReader(segment),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be using the pools.GetStringReader() pattern

var current strings.Builder
boundaryCount := 0

for {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should extract the common *bufio.Reader forloop line-reading into the ohio package and expose it using the go iterator API

}
}

func (env *Env) genesisObjectIds(bigBang BigBang) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all of the functions used in this method should be rewritten as methods on *Env

utility.AddCmd("migrate-zettel-ids", &MigrateZettelIds{})
}

type MigrateZettelIds struct {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lots of code duplication with the existing methods added for zettel id handling

# as those will point to the bats executable's location or the preprocessed file respectively
DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")" >/dev/null 2>&1 && pwd)"

cat_yin() (
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a serious mistake that required changing all the tests. Revert this change and all the modifications to zz-tests_bats/*.bats (except for the add_zettel_ids.bats file). Rather than changing the fixtures for everything, you can create a dedicated test fixture for add_zettel_ids.bats and init.bats.

echo "a sentence about ceroplastes"
echo "another line about midtown"
echo "something about harbor"
} >"$input"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer heredocs / nowdocs over subshell echo chains


run_dodder add-zettel-ids-yin <"$input"
assert_success
assert_output --partial "added 3 words to Yin"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer non --partial asserts when possible (the output for this is deterministic, so it's possible)

[one/uno @blake2b256-gu738nunyrnsqukgqkuaau9zslu0fhwg4dgs9ltuyvnlp42wal8sdpn2hc !md "wow" tag]
[alpha/golf @blake2b256-gu738nunyrnsqukgqkuaau9zslu0fhwg4dgs9ltuyvnlp42wal8sdpn2hc !md "wow" tag]
EOM
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather than having changed the fixtures for all the tests, a new test in this file that performs initialization with custom yin / yang files would've been much simpler

friedenberg and others added 13 commits February 26, 2026 06:46
fixup! test: update all BATS tests for NATO phonetic zettel IDs

Revert all test changes - keep old word lists (one/two/three, uno/dos/tres).
Only retain new subcommand entries in complete.bats.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing

fixup! feat: update genesis to use object ID log with raw text processing

Remove ExtractUniqueComponents from genesis - it filters short words,
breaking existing word lists. Genesis reads word-per-line files directly.
ExtractUniqueComponents is only for add-zettel-ids raw text input.

Revert common.bash to original word lists (one/two/three, uno/dos/tres).
Add dedicated NATO word init to add_zettel_ids.bats and init.bats.
Regenerate fixtures with old word lists + new object_id_log.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: implement object ID log reader and writer

Restructure log.go to OO: methods on Log struct instead of package
functions. Use pool.GetBufferedReader/GetStringReader for reader
recycling. Update all callers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing

fixup! feat: update genesis to use object ID log with raw text processing

Convert genesis helper functions to methods on *Env, use pool.GetBufferedReader
instead of bufio.NewScanner, read file lines into maps instead of slices,
iterate-write words individually instead of strings.Join.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add add-zettel-ids-yin and add-zettel-ids-yang commands

Merge add_zettel_ids_yin.go and add_zettel_ids_yang.go into single
add_zettel_ids.go. writeWordsAsBlob and appendWordsToFlatFile now cancel
context instead of returning errors, use iterate-write instead of
strings.Join.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add migrate-zettel-ids command

Reduce duplication: loop over yin/yang sides instead of repeated blocks,
merge countLines into writeFlatFileAsBlob (seek-based single file open),
cancel context instead of returning errors, use pool.GetBufferedReader.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: support log-based provider loading with flat file fallback

Rename object_id_provider package to zettel_id_provider. Updates
directory, package declarations, import paths, and all qualified
references across 10 files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: implement object ID log reader and writer

Extract common *bufio.Reader line-reading loop into ohio.MakeLineSeqFromReader
iterator. Update segmentEntries to use iterator, removing manual EOF handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing

fixup! feat: update genesis to use object ID log with raw text processing

Use ohio.MakeLineSeqFromReader iterator in readAndCleanFileLines instead
of manual ReadString/EOF loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add migrate-zettel-ids command

Use ohio.MakeLineSeqFromReader iterator in writeFlatFileAsBlob instead
of manual ReadString/EOF loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fixup! feat: add add-zettel-ids-yin and add-zettel-ids-yang commands

Use ohio.MakeLineSeqFromReader iterator in readAndExtractCandidates
instead of manual ReadString/EOF loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sing

fixup! feat: update genesis to use object ID log with raw text processing

Use order-preserving slices instead of maps in genesis to fix
non-deterministic word ordering. Change test NATO word entries from
multi-word ("the alpha") to single-word ("alpha") to match provider
storage format and fix cross-side rejection. Regenerate fixtures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant