Skip to content

Configure ci/cd#136

Open
nsyzrantsev wants to merge 64 commits intomainfrom
add-ci-cd
Open

Configure ci/cd#136
nsyzrantsev wants to merge 64 commits intomainfrom
add-ci-cd

Conversation

@nsyzrantsev
Copy link
Collaborator

No description provided.

nsyzrantsev and others added 30 commits August 9, 2025 19:59
Resolved conflict in conf/modules.config by combining:
- Published container image from add-ci-cd (ghcr.io/bostongene/pyigmap-fastp:1.0.0)
- FastpSingle process support from main (#135)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…eline

Implement a comprehensive CI/CD architecture with three separate workflows:

- ci.yml: Fast quality checks (lint, type, unit-tests) on every PR/push
  - Runs in ~5 minutes for immediate feedback
  - No Docker builds required
  - Removed component and integration tests

- cd.yml: Build and push Docker images on version changes
  - Added path filter to trigger only on bin/*/pyproject.toml changes
  - Removed pull_request trigger
  - Removed add-ci-cd branch from triggers
  - Only builds when cutting releases (version bumps)

- e2e.yml: End-to-end testing after image deployment (new)
  - Triggers automatically after cd.yml completes successfully
  - Pulls production images from GHCR
  - Correctly maps GHCR image names to local tags
  - Runs component and integration tests against production images

Benefits:
- Fast feedback loop: quality checks in ~5 min on every PR
- Resource efficiency: only build images on version bumps
- Production validation: E2E tests use actual GHCR images
- Clear separation: CI = quality, CD = build, E2E = validate

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…nd unit tests

Fix CI type checking and unit testing by ensuring all component packages
and their dependencies are available in the root venv:

- pyproject.toml: Add reporter visualization dependencies (panel, plotly,
  bokeh, matplotlib, seaborn, logomaker) to dev dependency group
  - Required for pyrefly type checker to resolve imports in reporter/viz.py

- justfile: Modify update-dev to install all component packages in editable mode
  - Installs pyumi, calib_dedup, reporter, fastp, vidjil, igblast,
    cdr3nt-error-corrector, fq-downloader in editable mode
  - Allows unit tests to import from component packages using absolute imports

- uv.lock: Update with new dev dependencies

Fixes:
- Type checking: reporter now passes (was failing on missing panel/plotly/bokeh)
- Unit tests: pyumi and cdr3nt-error-corrector tests now pass (were failing on
  ModuleNotFoundError for component packages)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add ffq==0.3.1 to dev dependencies to fix type checking for fq-downloader.

The type checker was failing because it couldn't find the ffq.ffq module
imported in bin/fq-downloader/src/fq_downloader/run.py.

All components now pass type checking with 0 errors.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant