Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
67f6a90
refactor: update dj package to use new name
eywalker Jun 19, 2025
ac228b0
feat: add ability to skip computation in pod
eywalker Jun 19, 2025
450ec90
refactor: major change of structure and implementation of pipeline
eywalker Jun 24, 2025
09eb947
refactor: implement ContentHashableBase
eywalker Jun 24, 2025
bd3c7a8
refactor: significantly clean up label logic
eywalker Jun 25, 2025
90b9dad
optim: avoid len call by using list comprehension
eywalker Jun 25, 2025
1e61259
refactor: place Operator back in base
eywalker Jun 25, 2025
df58134
refactor: place operator in base and add additional operator methods …
eywalker Jun 25, 2025
6e4d4bd
wip: change to content identifable base
eywalker Jun 25, 2025
e8efa44
Merge branch 'main' of https://github.com/walkerlab/orcabridge into p…
eywalker Jun 26, 2025
5fb2435
style: apply ruff formatting
eywalker Jun 26, 2025
09f59cb
refactor: clean up test of name orcabridge
eywalker Jun 26, 2025
c5fcb3d
test: remove filepath specification
eywalker Jun 27, 2025
22215ca
fix: remove orcabridge reference
eywalker Jun 27, 2025
56d559a
refactor: rename module to match class
eywalker Jun 27, 2025
59ad526
refactor: move core to legacy_core
eywalker Jun 27, 2025
3e0cdf4
fix: update reference to core
eywalker Jun 27, 2025
50e0772
refactor: rename semantic arrow hasher module to generic arrow hashers
eywalker Jun 27, 2025
33103b8
refactor: rename variables to typespec
eywalker Jun 27, 2025
e35b024
feat: collect refined hashing functions
eywalker Jun 27, 2025
02412d0
feat: collect semantic type hashsers into a module
eywalker Jun 27, 2025
1e90679
refactor: make file hasher return bytes
eywalker Jun 27, 2025
78fdead
feat: add new defaut object hasher
eywalker Jun 27, 2025
3dcaa0b
test: update ref
eywalker Jun 27, 2025
89ddd76
fix: handle type vars in process_structure
eywalker Jun 27, 2025
905f915
wip: use new schema system
eywalker Jun 30, 2025
a3ba172
feat: add field source tracking
eywalker Jul 1, 2025
d3b66de
feat: support map and join on packets with source info
eywalker Jul 1, 2025
0bafbaa
fix: keep all columns internally
eywalker Jul 1, 2025
6321467
wip: update legacy file related tests and rename to stores
eywalker Jul 1, 2025
41f1b63
test: fix legacy tests
eywalker Jul 1, 2025
fe423f7
fix: make all tests functional
eywalker Jul 1, 2025
ba1f45d
refactor: cleanup imports and use versioned object hasher
eywalker Jul 1, 2025
e689d0d
fix: failure to reset cache due to mro mixup
eywalker Jul 1, 2025
6222064
style: apply ruff format
eywalker Jul 1, 2025
cbe82ab
fix: legacy_core imports
eywalker Jul 1, 2025
caca67b
wip: arrow logical serialization
eywalker Jul 1, 2025
7bc98e1
refactor: utils renaming and relocation
eywalker Jul 1, 2025
51f3da2
fix: cleanup imports and fix issue in recursive structure processing
eywalker Jul 2, 2025
3d54067
refactor: add more robust arrow serialization strategy and use @ for …
eywalker Jul 2, 2025
1ac2be6
feat: logical serialization for arrow table
eywalker Jul 2, 2025
dab3378
feat: update versioned arrow hasher to use new serialization
eywalker Jul 2, 2025
4f07927
wip: delta table store implementation
eywalker Jul 2, 2025
1b7519e
feat: better handling of stores and add flushing to stores and pipeline
eywalker Jul 2, 2025
07fd76e
feat: integrate actual saving to parquet into simple in memory store
eywalker Jul 2, 2025
8411b40
refactor: cleanup improt and comment out old packet converter for fut…
eywalker Jul 2, 2025
d90e5c6
fix: attach label on kernel invocation to the invocation object
eywalker Jul 3, 2025
fe35aba
fix: invoke superclass init
eywalker Jul 3, 2025
ef301b3
feat: expose explicit check for assigned label on content identifiabl…
eywalker Jul 3, 2025
ead6704
feat: add label on wrapped invocation
eywalker Jul 3, 2025
cbb8754
doc: add tutorial notebook
eywalker Jul 3, 2025
73b2638
refactor: clean up store package
eywalker Jul 3, 2025
555a751
feat: improve pipeline usability with typechecks and convenience attr…
eywalker Jul 3, 2025
083134b
fix: use new store name
eywalker Jul 3, 2025
7e33bae
test: update to use new package name
eywalker Jul 3, 2025
5641810
fix: wrong import
eywalker Jul 3, 2025
00b4066
Merge pull request #27 from eywalker/pipeline
brian-arnold Jul 3, 2025
c66920c
doc: handle typing corner cases
eywalker Jul 3, 2025
58d7e40
Merge branch 'pipeline' of https://github.com/walkerlab/orcabridge in…
eywalker Jul 3, 2025
7ace5a4
doc: reorganize tutorials
eywalker Jul 5, 2025
6f41f52
feat: cleaned up delta store
eywalker Jul 5, 2025
523291f
feat: add protocols
eywalker Jul 10, 2025
83118ab
refactor: use protocols in hashing package
eywalker Jul 10, 2025
5fc78f8
refactor: temporarily stop top level import while refactoring
eywalker Jul 10, 2025
93beb0f
refactor: remove protocol-relevant definitions
eywalker Jul 10, 2025
4d7761f
refactor: add concrete component implementation in data package
eywalker Jul 10, 2025
cac1855
refactor: cleanup protocols
eywalker Jul 11, 2025
5a178b5
refactor: further refinement of tracker protocols
eywalker Jul 11, 2025
53527b1
feat: refine kernel and pod interaction with tracker
eywalker Jul 11, 2025
6e2bdd7
feat: implement pure immutable datagram
eywalker Jul 12, 2025
7293749
fix: preparation of output stream in pod
eywalker Jul 12, 2025
7f49de0
feat: add feature to include content hash in arrow table
eywalker Jul 12, 2025
ff99495
doc: add comprehensive documentation to datagrams
eywalker Jul 12, 2025
5c8f85d
refactor: remove unused datagram base
eywalker Jul 14, 2025
3d3e946
refactor: combine pre-foward step into one for simplicity
eywalker Jul 14, 2025
0d8f7cb
refactor: adopt the new method signature for pre-forward step
eywalker Jul 14, 2025
a7531bf
feat: add non-zero input operator
eywalker Jul 14, 2025
730f72b
wip: major refactoring of package structure
eywalker Jul 18, 2025
29b8004
feat: implement data context to capture shared hashing and semantic c…
eywalker Jul 19, 2025
4c710db
refactor: clean up protocol around types
eywalker Jul 19, 2025
8b84c02
wip: further refinement of datagram implementations
eywalker Jul 22, 2025
8429611
fix: handling of schema when merging tables
eywalker Jul 22, 2025
228f469
refactor: clean up unused imports and move old code into renamed module
eywalker Jul 22, 2025
c871bfb
feat: add lazyloading system
eywalker Jul 22, 2025
a416c20
refactor: refine kernel and pod setup
eywalker Jul 22, 2025
af75ab7
refactor: refine tracker system
eywalker Jul 22, 2025
2854068
feat: add wrapped stream
eywalker Jul 22, 2025
15bfc4c
refactor: use hasher id consistently
eywalker Jul 22, 2025
534e810
refactor: remove fixed stream from kernel and clean up cached pod
eywalker Jul 23, 2025
c443a32
refactor: consistent copy logic and ability to specify meta info in c…
eywalker Jul 23, 2025
08fa0ef
feat: clean implementation of pipeline nodes
eywalker Jul 23, 2025
38b155b
refactor: rename pre-kernel step to be more explicit
eywalker Jul 24, 2025
3351cf9
refactor: extract node base class
eywalker Jul 24, 2025
7ff5a51
refactor: import cleanup and additional todos
eywalker Jul 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ RUN \
USER vscode
ENV PATH=/home/vscode/.local/bin:$PATH
WORKDIR /home/vscode
COPY --chown=vscode:nogroup src/orcabridge/requirements.txt /tmp/requirements.txt
COPY --chown=vscode:nogroup src/orcapod/requirements.txt /tmp/requirements.txt
RUN \
# python setup
curl -LsSf https://astral.sh/uv/install.sh | sh && \
Expand Down
6 changes: 3 additions & 3 deletions misc/demo_redis_mocking.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@ def demonstrate_redis_mocking():

# Patch the Redis availability and exceptions
with (
patch("orcabridge.hashing.string_cachers.REDIS_AVAILABLE", True),
patch("orcabridge.hashing.string_cachers.redis.RedisError", MockRedisError),
patch("orcapod.hashing.string_cachers.REDIS_AVAILABLE", True),
patch("orcapod.hashing.string_cachers.redis.RedisError", MockRedisError),
patch(
"orcabridge.hashing.string_cachers.redis.ConnectionError",
"orcapod.hashing.string_cachers.redis.ConnectionError",
MockConnectionError,
),
):
Expand Down
Loading
Loading