feat: sam,sam2 promtable and cellsam integration #978
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix KeyError 'x' when adding subsequent points (use framePointsData not pointsDataPos) - Add missing model_segment_kwargs argument to startMagicPromptsWorkerAndWait call - Fix zoom_slice to be tuple of slices for proper array indexing - Handle None acdc_df in _get_data_visited to prevent TypeError - Reorder SAM2 model_types to put 'Large' first (matching segment_anything) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add acdcPromptSegment.py for SAM2 with point prompts support - Add acdcPromptSegment.py for segment_anything with point prompts support - Both use treat_other_objects_as_background to prevent mask overlap - Add heapq-based area sorting to prevent small masks being overwritten - Add tests for promptable segmentation models Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
SAM and SAM2's predict() expects point_labels to be binary: - 0 = negative (background) - 1 = positive (foreground) Previously, the code passed actual object IDs (e.g., [0, 0, 5, 5]) which worked by accident since SAM treats any non-zero as positive. Now correctly passes [0, 0, 1, 1] as intended by the API. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously "union" would overwrite existing labels with new masks. Now it only adds new mask pixels where there's no existing label, preserving existing segmentation masks. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implemented proper set-theoretic logic for combining masks: For each overlapping pair (old ID p, new ID q): - union: p OR q region → all become p (old absorbs new) - intersection: only p AND q → p; p XOR q → 0 (deleted) - new: paint p then q → overlap becomes q (new overwrites) Non-overlapping regions: - Old-only IDs: preserved in all modes - New-only IDs: added in 'new' and 'union', not in 'intersection' Extracted _build_combined_mask() and _apply_overlap_rule() helpers. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- lab_new is now simply _build_combined_mask(model_out) - Removed 'new' mode from _apply_overlap_rule (no longer needed) - Kept edited_IDs parameter for API compatibility but marked deprecated - Uses sorted() instead of heapq for area-based ordering Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace duplicate heapq-based sorting logic in promptable models with shared build_combined_mask utility. Each model now paints masks directly, then build_combined_mask handles area-based sorting (largest first so smaller objects end up on top). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add CellSAM (vanvalenlab/cellSAM) as a new segmentation model option. CellSAM is a foundation model for cell segmentation that works across various cellular targets and imaging modalities without manual prompts. Features: - Automatic cell detection via CellFinder + SAM architecture - WSI tiling mode for large images (>3000 cells) - Low contrast enhancement preprocessing option - Z-stack support with slice-by-slice processing - Two model variants: General (paper) and Extra (extended datasets) Key parameters: - bbox_threshold: precision/recall control (default 0.4) - use_wsi: enable tiling for large images - block_size/overlap: tiling configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add tests/utils/segmentation.py with shared helpers for data loading, validation, and plotting - Add ensure_sam(), ensure_sam2(), ensure_cellsam() import guards - Add get_test_posdata(), load_normalized_frames(), load_single_normalized_frame() - Add validate_labels(), save_segmentation_overlay(), save_labels_image() - Refactor all segm and prompt_segm tests to use shared utilities - Add cellsam extra to pyproject.toml - Rename segment_anything extra to sam Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add get_test_dataset() to return full dataset object - Add get_ground_truth_centroids() to extract centroids from GT masks - Update promptable SAM/SAM2 tests to prompt with GT centroids - Use save_segmentation_overlay() for better visualization Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add log_mask_selection() to print mask sizes and scores for debugging - Plot prompt points as red X markers on segmentation overlays - Use Large model size for both SAM and SAM2 tests for fair comparison Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add check_install_cellsam() function in myutils.py following SAM/SAM2 pattern - Display token requirement in GUI installation dialog via note parameter - Add runtime error handling with helpful message for auth failures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
treat_other_objects_as_background=Falseis actually better for interactive usage, passed unit testcellacdc/promptable_models/utils.pybe careful, could affect nninterative, didn't fully test in gui