From 1eb84e41299996e00d770ad39e2dd03303a1a163 Mon Sep 17 00:00:00 2001 From: Lili Deng Date: Wed, 13 May 2026 12:42:20 +0800 Subject: [PATCH 1/2] Add PyPI publish workflow, RELEASE.md, prune lisa/ai/data from sdist - .github/workflows/publish.yml: tag-triggered build then publish via PyPI Trusted Publishing (OIDC); TestPyPI first, then PyPI gated by GitHub Environment reviewer approval. No tokens stored. - RELEASE.md: one-time bootstrap (pending publishers, GitHub environments, tag protection) plus per-release SOP and known limitations. - MANIFEST.in: prune lisa/ai/data; deeply nested log paths trigger Windows 260-char limit during sdist build. --- .github/workflows/publish.yml | 79 +++++++++++++++++++++ MANIFEST.in | 5 ++ RELEASE.md | 126 ++++++++++++++++++++++++++++++++++ 3 files changed, 210 insertions(+) create mode 100644 .github/workflows/publish.yml create mode 100644 RELEASE.md diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000000..10b0490b28 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,79 @@ +name: Publish to PyPI + +# Publishes mslisa to PyPI when a version tag (vX.Y.Z) is pushed. +# Uses PyPI Trusted Publishing (OIDC) — no API tokens stored. +# +# Bootstrap (one-time, see RELEASE.md): +# 1. Add a pending publisher on PyPI: +# project=mslisa, owner=microsoft, repo=lisa, +# workflow=publish.yml, environment=pypi +# 2. Add a pending publisher on TestPyPI with environment=testpypi +# 3. Create GitHub Environments "testpypi" and "pypi"; +# put required reviewers on "pypi". + +on: + push: + tags: + - "v*.*.*" + workflow_dispatch: # manual run for build-only smoke tests + +permissions: + contents: read + +jobs: + build: + name: Build sdist + wheel + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # setuptools_scm needs full tag history + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Install build tooling + run: python -m pip install --upgrade build twine + - name: Build distributions + run: python -m build + - name: Validate metadata + run: python -m twine check dist/* + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish-testpypi: + name: Publish to TestPyPI + needs: build + if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') + runs-on: ubuntu-latest + permissions: + id-token: write # required for OIDC + environment: + name: testpypi + url: https://test.pypi.org/project/mslisa/ + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist + - uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + + publish-pypi: + name: Publish to PyPI + needs: publish-testpypi + runs-on: ubuntu-latest + permissions: + id-token: write + environment: + name: pypi # add required reviewers here for human approval gate + url: https://pypi.org/project/mslisa/ + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist + - uses: pypa/gh-action-pypi-publish@release/v1 diff --git a/MANIFEST.in b/MANIFEST.in index 5dda3f17db..284afe5631 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -6,4 +6,9 @@ prune .github exclude .git* +# AI training data is large and not needed at runtime; exclude from sdist/wheel. +# Without pruning, the deeply-nested log paths exceed the Windows 260-char limit +# during `python -m build` and break sdist creation. +prune lisa/ai/data + # Everything else tracked by git is include automatically by setuptools-scm diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000000..26efc84e34 --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,126 @@ +# Releasing `mslisa` to PyPI + +LISA is published to PyPI as the [`mslisa`](https://pypi.org/project/mslisa/) +package. Releases are driven by the +[`publish.yml`](.github/workflows/publish.yml) workflow and use **PyPI Trusted +Publishing (OIDC)** — no API tokens are stored anywhere. + +End user install: + +```bash +pip install mslisa # core only +pip install "mslisa[azure]" # most common: with Azure SDK extras +pip install "mslisa[azure,libvirt]" +lisa --help +``` + +--- + +## One-time bootstrap (do once per project / per environment) + +These steps are needed before the first successful release. After bootstrap they +do not need to be repeated. + +### 1. PyPI pending publishers + +Any maintainer signs in to PyPI **and** TestPyPI with their personal account +(2FA required), then adds a **pending publisher**: + +- PyPI → → *Add a new pending publisher* + - PyPI Project Name: `mslisa` + - Owner: `microsoft` + - Repository name: `lisa` + - Workflow name: `publish.yml` + - Environment name: `pypi` +- TestPyPI → → same form, but + - Environment name: `testpypi` + +The first successful workflow run claims the project name automatically. After +that the personal account can be removed from the project; the trust is bound +to `microsoft/lisa` + `publish.yml`, not to the person. + +### 2. GitHub Environments + +In `microsoft/lisa` → **Settings → Environments**, create two environments: + +| Environment | Required reviewers | Purpose | +|---|---|---| +| `testpypi` | (none, optional) | Auto-publishes pre-release artifacts | +| `pypi` | 1–2 LSG maintainers | Human approval gate before PyPI | + +Reviewers approve via the Actions run UI when the workflow pauses on the `pypi` +environment. + +### 3. Tag protection (recommended) + +**Settings → Tags → Add rule** → pattern `v*.*.*`, restrict push to release +managers. Prevents accidental tag-based releases. + +--- + +## Cutting a release + +1. Make sure `main` is green. +2. Decide the version using semver (`vMAJOR.MINOR.PATCH`). +3. Update `CHANGELOG.md` (or release notes draft) and merge. +4. Tag and push: + ```bash + git checkout main + git pull --ff-only + git tag -a v3.13.0 -m "v3.13.0" + git push origin v3.13.0 + ``` +5. Watch **Actions → Publish to PyPI**. +6. After the `publish-testpypi` job succeeds, smoke test in a clean venv: + ```powershell + py -3.12 -m venv C:\tmp\mslisa-rc + & C:\tmp\mslisa-rc\Scripts\python.exe -m pip install ` + --index-url https://test.pypi.org/simple/ ` + --extra-index-url https://pypi.org/simple/ ` + "mslisa[azure]==3.13.0" + & C:\tmp\mslisa-rc\Scripts\lisa.exe --help + ``` +7. Approve the `pypi` environment in the workflow run. +8. Verify the live release: + ```bash + pip install --upgrade "mslisa==3.13.0" + lisa --version + ``` + +> A version that has been uploaded to PyPI **cannot** be replaced. To fix a bad +> release, yank it on PyPI and publish a new patch version. + +--- + +## Local dry run (no upload) + +Use this any time you change packaging metadata, `MANIFEST.in`, or +`pyproject.toml`: + +```powershell +cd lisa +.\.venv\Scripts\python.exe -m pip install --upgrade build twine +.\.venv\Scripts\python.exe -m build --wheel # wheel only on Windows; + # CI builds sdist on Linux +.\.venv\Scripts\python.exe -m twine check dist\* + +# Try installing into a fresh venv +py -3.12 -m venv C:\tmp\mslisa-local +$wheel = (Get-Item dist\mslisa-*.whl).FullName +& C:\tmp\mslisa-local\Scripts\python.exe -m pip install "$wheel[azure]" +& C:\tmp\mslisa-local\Scripts\lisa.exe --help +``` + +--- + +## Known limitations + +- **sdist build fails on Windows** because `setuptools_scm` includes every git- + tracked file (including deeply nested logs under `lisa/ai/data/...`) and the + resulting paths exceed Windows' 260-character limit. CI builds on Linux are + unaffected. The wheel is what users actually install. +- **`MANIFEST.in` `prune` rules don't apply** to files already tracked by git + when `setuptools_scm` is the file finder. To shrink the sdist, move + `lisa/ai/data/` out of git (git-lfs or a sibling repo). +- **Optional extras** (`baremetal`, `aws`, `ai`, etc.) are *not* installed by + `pip install mslisa`. Users opt in with `pip install "mslisa[azure,ai,…]"`. From c6ecad4c7d6f203188601d059c33910a1b8d1b32 Mon Sep 17 00:00:00 2001 From: Lili Deng Date: Wed, 13 May 2026 12:47:25 +0800 Subject: [PATCH 2/2] publish: switch tag pattern to CalVer YYYYMMDD.N --- .github/workflows/publish.yml | 7 ++++--- RELEASE.md | 20 +++++++++++++------- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 10b0490b28..7ced64a0f6 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -1,6 +1,6 @@ name: Publish to PyPI -# Publishes mslisa to PyPI when a version tag (vX.Y.Z) is pushed. +# Publishes mslisa to PyPI when a CalVer tag (YYYYMMDD.N) is pushed. # Uses PyPI Trusted Publishing (OIDC) — no API tokens stored. # # Bootstrap (one-time, see RELEASE.md): @@ -14,7 +14,8 @@ name: Publish to PyPI on: push: tags: - - "v*.*.*" + # CalVer: e.g. 20260420.1, 20260420.2 + - "2[0-9][0-9][0-9][0-9][0-9][0-9][0-9].*" workflow_dispatch: # manual run for build-only smoke tests permissions: @@ -46,7 +47,7 @@ jobs: publish-testpypi: name: Publish to TestPyPI needs: build - if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') + if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') runs-on: ubuntu-latest permissions: id-token: write # required for OIDC diff --git a/RELEASE.md b/RELEASE.md index 26efc84e34..056d8904b4 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -53,37 +53,43 @@ environment. ### 3. Tag protection (recommended) -**Settings → Tags → Add rule** → pattern `v*.*.*`, restrict push to release -managers. Prevents accidental tag-based releases. +**Settings → Tags → Add rule** → pattern `2[0-9][0-9][0-9][0-9][0-9][0-9][0-9].*`, +restrict push to release managers. Prevents accidental tag-based releases. --- ## Cutting a release +LISA uses **CalVer** tags in the form `YYYYMMDD.N` (e.g. `20260420.1`, +`20260420.2`). `setuptools_scm` derives the package version directly from the +tag, so the PyPI version equals the tag (no `v` prefix). + 1. Make sure `main` is green. -2. Decide the version using semver (`vMAJOR.MINOR.PATCH`). +2. Pick today's date and the next sequence number for that day. 3. Update `CHANGELOG.md` (or release notes draft) and merge. 4. Tag and push: ```bash git checkout main git pull --ff-only - git tag -a v3.13.0 -m "v3.13.0" - git push origin v3.13.0 + TAG=$(date +%Y%m%d).1 # bump .1 -> .2 if a tag for today exists + git tag -a "$TAG" -m "$TAG" + git push origin "$TAG" ``` 5. Watch **Actions → Publish to PyPI**. 6. After the `publish-testpypi` job succeeds, smoke test in a clean venv: ```powershell + $TAG = "20260513.1" # use the tag you just pushed py -3.12 -m venv C:\tmp\mslisa-rc & C:\tmp\mslisa-rc\Scripts\python.exe -m pip install ` --index-url https://test.pypi.org/simple/ ` --extra-index-url https://pypi.org/simple/ ` - "mslisa[azure]==3.13.0" + "mslisa[azure]==$TAG" & C:\tmp\mslisa-rc\Scripts\lisa.exe --help ``` 7. Approve the `pypi` environment in the workflow run. 8. Verify the live release: ```bash - pip install --upgrade "mslisa==3.13.0" + pip install --upgrade "mslisa==20260513.1" lisa --version ```