Skip to content

ci: refactor using uv by default#5210

Closed
Xuanwo wants to merge 9 commits intomainfrom
refactor-ci
Closed

ci: refactor using uv by default#5210
Xuanwo wants to merge 9 commits intomainfrom
refactor-ci

Conversation

@Xuanwo
Copy link
Copy Markdown
Collaborator

@Xuanwo Xuanwo commented Nov 11, 2025

This PR will refactor our python dep management into uv entirely.


This PR was primarily authored with Codex using GPT-5-Codex and then hand-reviewed by me. I AM responsible for every change made in this PR. I aimed to keep it aligned with our goals, though I may have missed minor issues. Please flag anything that feels off, I'll fix it quickly.

@github-actions github-actions Bot added python ci Github Action or Test issues labels Nov 11, 2025
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci-benchmarks.yml
Comment thread .github/workflows/benchmarks.yml
Comment thread .github/workflows/file_verification.yml
@Xuanwo Xuanwo marked this pull request as draft November 11, 2025 10:48
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Comment on lines +211 to +249
name: Python Torch Tests (3.11 Linux)
runs-on: "ubuntu-24.04"
timeout-minutes: 45
defaults:
run:
shell: bash
working-directory: python
env:
UV_PYTHON: "3.11"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
lfs: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- uses: Swatinem/rust-cache@v2
with:
workspaces: python
prefix-key: ${{ env.CACHE_PREFIX }}
cache-targets: false
cache-workspace-crates: true
- name: Install system dependencies
run: |
sudo apt update
sudo apt install -y protobuf-compiler libssl-dev
- name: Sync torch environment
run: |
uv sync --frozen --project python/tests/torch_tests
- name: Run torch tests
run: make test-torch

pandas:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
Comment on lines +250 to +288
name: Python Pandas/SQL Tests (3.11 Linux)
runs-on: "ubuntu-24.04"
timeout-minutes: 45
defaults:
run:
shell: bash
working-directory: python
env:
UV_PYTHON: "3.11"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
lfs: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- uses: Swatinem/rust-cache@v2
with:
workspaces: python
prefix-key: ${{ env.CACHE_PREFIX }}
cache-targets: false
cache-workspace-crates: true
- name: Install system dependencies
run: |
sudo apt update
sudo apt install -y protobuf-compiler libssl-dev
- name: Sync pandas environment
run: |
uv sync --frozen --project python/tests/pandas_tests
- name: Run pandas tests
run: make test-pandas

tensorflow:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 5 months ago

To fix this issue, you should set an explicit permissions block in the workflow, restricting permissions to the minimum required. The best approach is to add permissions: contents: read at the workflow root—line 2, after the name: declaration—so that all jobs inherit this minimal permission unless they require more or override it. This satisfies principle of least privilege and the CodeQL rule. No additional imports or methods are needed in a GitHub Actions workflow file; the edit is purely to the YAML.

If any later jobs are shown to require additional permissions, they would need overrides, but for the sections shown (torch, pandas, tensorflow, etc.), only code checkout and test running occurs, so contents: read should be sufficient.


Suggested changeset 1
.github/workflows/python.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
--- a/.github/workflows/python.yml
+++ b/.github/workflows/python.yml
@@ -1,4 +1,6 @@
 name: Python
+permissions:
+  contents: read
 
 on:
   push:
EOF
@@ -1,4 +1,6 @@
name: Python
permissions:
contents: read

on:
push:
Copilot is powered by AI and may make mistakes. Always verify output.
Xuanwo added a commit that referenced this pull request Nov 19, 2025
This PR is based on #5210

---

This PR intends to add a blob arrow extension type (aka, logical type)
in lance.

**This PR was primarily authored with Codex using GPT-5-Codex and then
hand-reviewed by me. I AM responsible for every change made in this PR.
I aimed to keep it aligned with our goals, though I may have missed
minor issues. Please flag anything that feels off, I'll fix it
quickly.**

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
jackye1995 pushed a commit to jackye1995/lance that referenced this pull request Jan 21, 2026
This PR is based on lance-format#5210

---

This PR intends to add a blob arrow extension type (aka, logical type)
in lance.

**This PR was primarily authored with Codex using GPT-5-Codex and then
hand-reviewed by me. I AM responsible for every change made in this PR.
I aimed to keep it aligned with our goals, though I may have missed
minor issues. Please flag anything that feels off, I'll fix it
quickly.**

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo closed this Mar 30, 2026
Xuanwo added a commit that referenced this pull request Mar 30, 2026
This updates the Python contributor docs to prefer `uv` for local
environment initialization and day-to-day command execution, while
keeping the documented `Makefile` targets unchanged.

I intentionally did not migrate CI in this PR. We have already tried
broader `uv` rollouts in #4221 and are still carrying a larger CI
refactor in #5210; the current Python workflows still have materially
different dependency shapes across lint, wheel-install test, benchmark,
and optional Torch / TensorFlow / Ray paths, so I don't see a single
environment definition we can switch to safely without a dedicated CI
pass.

This keeps the local guidance moving in the right direction without
pretending the CI environment has already been unified. No tests were
run because this is a docs-only change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Github Action or Test issues python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants