Skip to content

Conversation

@eywalker
Copy link
Contributor

  • Added placeholder, auto-generated tests for streams and operations. Most are NOT functional and must be fixed/rewritten.
  • Minor refactoring of package structure and renaming to match the nature of contents (e.g. mapper -> mappers to reflect the fact it is a collection of multiple mappers).
  • Added claims_unique_tags to serve as a mechanism to determine claimed uniqueness of the tag values in a stream. Makes use of the Stream -> Invocation -> Operation -> Stream -> ... recursive patterns used in keys and identity_structure to infer the uniqueness of tags by climbing up the processing pipeline chain.
  • Made FunctionPod and pipeline pickleable.

@eywalker eywalker requested review from brian-arnold and Copilot June 17, 2025 22:18
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds placeholder tests for streams and operations while also performing minor package refactors and API enhancements.

  • Introduces auto-generated tests (placeholders) for streams and operations.
  • Refactors package structure and renames modules (e.g. mapper → mappers).
  • Adds a new TransferCacher and extends hashing functionality with a name_override parameter.

Reviewed Changes

Copilot reviewed 90 out of 90 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/orcabridge/hashing/string_cachers.py Added TransferCacher and introduced Redis logging for caching.
src/orcabridge/hashing/hashing_legacy.py Updated inline comments for parameter renaming.
src/orcabridge/hashing/files.py Reordered and adjusted imports.
src/orcabridge/hashing/file_hashers.py Reordered import statements.
src/orcabridge/hashing/defaults.py Added a helper to retrieve a composite hasher with a cacher.
src/orcabridge/hashing/core.py Introduced name_override parameter and refined function hashing logic.
src/orcabridge/dj/*.py Updated import paths and adjusted namespace references.
src/orcabridge/base.py Made Operation and Tracker abstract; added claims_unique_tags logic.
pyproject.toml Added new dependencies for pandas, pyyaml, pyarrow, polars, etc.
notebooks/*.ipynb Minor adjustments to import statements and formatting.
Comments suppressed due to low confidence (3)

src/orcabridge/hashing/core.py:777

  • The function signature for 'get_function_signature' was updated with the 'name_override' parameter, but the docstring does not reflect this change. Please update the docstring to describe the new parameter and its purpose.
def get_function_signature(

src/orcabridge/base.py:310

  • The docstring for 'claims_unique_tags' in SyncStream indicates that the method may return None if uniqueness cannot be determined, yet the signature specifies a boolean return type. Please clarify the intended behavior and update the docstring or return type accordingly.
    def claims_unique_tags(self, *, trigger_run=True) -> bool:

src/orcabridge/hashing/string_cachers.py:685

  • The logger is used in this file without an import or initialization, which could lead to a runtime error. Consider adding 'import logging' and initializing a logger (e.g., 'logger = logging.getLogger(name)') at the top of the file.
                logger.info(f"Retrieved cached value from Redis for key {cache_key}")

Copy link
Collaborator

@brian-arnold brian-arnold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a brief look through.

@eywalker eywalker merged commit 84bf717 into walkerlab:main Jun 18, 2025
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants