Skip to content

Add python-3.12.10-scientific-slim runenv variant#59

Merged
PaulNewling merged 2 commits intomainfrom
paulnewling/add-scientific-slim-variant
Apr 23, 2026
Merged

Add python-3.12.10-scientific-slim runenv variant#59
PaulNewling merged 2 commits intomainfrom
paulnewling/add-scientific-slim-variant

Conversation

@PaulNewling
Copy link
Copy Markdown
Contributor

@PaulNewling PaulNewling commented Apr 22, 2026

Summary

  • New python-3.12.10-scientific-slim runenv variant bundling polars-lts-cpu==1.33.1, numpy==2.2.6, scipy==1.15.3, and pyarrow==21.0.0.
  • Registered in pnpm-workspace.yaml and the catalogue package's entrypoints/deps.
  • Changeset: patch on the new variant, minor on the catalogue package (new entrypoint added).

Motivation

platforma-open/titeseq-analysis CI currently fails from disk exhaustion on ubuntu-latest while extracting the ~800 MB TensorFlow wheel shipped by the base 3.12.10 runenv (~2.5 GB total across 36 ML packages). That block does pure Hill-equation fitting against MiXCR output — it does not use TF, torch, cudf, transformers, etc. This slim variant ships ~100 MB of wheels that match titeseq's existing software/src/requirements.txt pins exactly, so runtime pip can resolve them under the hardcoded --no-index --find-links={runenv}/packages/ flags in install-deps.tpl.tengo.

Variant layout

python-3.12.10-scientific-slim/
├── package.json  (mirrors python-3.12.10-h5ad/package.json structure; version 1.0.0)
└── config.json   (4 pinned deps, same schema as h5ad/sccoda)

Notes

  • No changes to existing variants or the base runenv.
  • Future consumers with identical scipy/numpy/polars/pyarrow stacks could migrate off the heavy base (e.g., paratope-clustering, cell-type-annotation, differential-expression — each shares the same runenv and will hit the same wall once they add real integration tests).

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a lightweight python-3.12.10-scientific-slim run environment variant (~100 MB) bundling polars-lts-cpu, numpy, scipy, and pyarrow. Feedback suggests adding pandas for enhanced utility and aligning the config.json schema with the base environment for consistency.

Comment thread python-3.12.10-scientific-slim/config.json
Comment thread python-3.12.10-scientific-slim/config.json
@PaulNewling PaulNewling marked this pull request as ready for review April 22, 2026 15:49
Bundles only polars-lts-cpu==1.33.1, numpy==2.2.6, scipy==1.15.3, and
pyarrow==21.0.0 for blocks with tabular/scientific Python stacks that do
not need the full ML toolchain (TensorFlow, torch, transformers, etc.)
shipped by the base 3.12.10 runenv.

Intended first consumer: platforma-open/titeseq-analysis, whose CI
currently exhausts the ubuntu-latest runner disk while extracting the
base runenv's 800 MB TensorFlow wheel. Bundled size drops from ~2.5 GB
to ~100 MB, leaving ample headroom for that block's multi-block
integration test.
@PaulNewling PaulNewling force-pushed the paulnewling/add-scientific-slim-variant branch from 7071642 to bff45ce Compare April 22, 2026 19:43
@blackcat blackcat self-requested a review April 23, 2026 09:11
@PaulNewling PaulNewling merged commit 9f130fe into main Apr 23, 2026
38 checks passed
@PaulNewling PaulNewling deleted the paulnewling/add-scientific-slim-variant branch April 23, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants