diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000000..7ced64a0f6 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,80 @@ +name: Publish to PyPI + +# Publishes mslisa to PyPI when a CalVer tag (YYYYMMDD.N) is pushed. +# Uses PyPI Trusted Publishing (OIDC) — no API tokens stored. +# +# Bootstrap (one-time, see RELEASE.md): +# 1. Add a pending publisher on PyPI: +# project=mslisa, owner=microsoft, repo=lisa, +# workflow=publish.yml, environment=pypi +# 2. Add a pending publisher on TestPyPI with environment=testpypi +# 3. Create GitHub Environments "testpypi" and "pypi"; +# put required reviewers on "pypi". + +on: + push: + tags: + # CalVer: e.g. 20260420.1, 20260420.2 + - "2[0-9][0-9][0-9][0-9][0-9][0-9][0-9].*" + workflow_dispatch: # manual run for build-only smoke tests + +permissions: + contents: read + +jobs: + build: + name: Build sdist + wheel + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # setuptools_scm needs full tag history + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Install build tooling + run: python -m pip install --upgrade build twine + - name: Build distributions + run: python -m build + - name: Validate metadata + run: python -m twine check dist/* + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish-testpypi: + name: Publish to TestPyPI + needs: build + if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') + runs-on: ubuntu-latest + permissions: + id-token: write # required for OIDC + environment: + name: testpypi + url: https://test.pypi.org/project/mslisa/ + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist + - uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + + publish-pypi: + name: Publish to PyPI + needs: publish-testpypi + runs-on: ubuntu-latest + permissions: + id-token: write + environment: + name: pypi # add required reviewers here for human approval gate + url: https://pypi.org/project/mslisa/ + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist + - uses: pypa/gh-action-pypi-publish@release/v1 diff --git a/MANIFEST.in b/MANIFEST.in index 5dda3f17db..284afe5631 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -6,4 +6,9 @@ prune .github exclude .git* +# AI training data is large and not needed at runtime; exclude from sdist/wheel. +# Without pruning, the deeply-nested log paths exceed the Windows 260-char limit +# during `python -m build` and break sdist creation. +prune lisa/ai/data + # Everything else tracked by git is include automatically by setuptools-scm diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000000..056d8904b4 --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,132 @@ +# Releasing `mslisa` to PyPI + +LISA is published to PyPI as the [`mslisa`](https://pypi.org/project/mslisa/) +package. Releases are driven by the +[`publish.yml`](.github/workflows/publish.yml) workflow and use **PyPI Trusted +Publishing (OIDC)** — no API tokens are stored anywhere. + +End user install: + +```bash +pip install mslisa # core only +pip install "mslisa[azure]" # most common: with Azure SDK extras +pip install "mslisa[azure,libvirt]" +lisa --help +``` + +--- + +## One-time bootstrap (do once per project / per environment) + +These steps are needed before the first successful release. After bootstrap they +do not need to be repeated. + +### 1. PyPI pending publishers + +Any maintainer signs in to PyPI **and** TestPyPI with their personal account +(2FA required), then adds a **pending publisher**: + +- PyPI → → *Add a new pending publisher* + - PyPI Project Name: `mslisa` + - Owner: `microsoft` + - Repository name: `lisa` + - Workflow name: `publish.yml` + - Environment name: `pypi` +- TestPyPI → → same form, but + - Environment name: `testpypi` + +The first successful workflow run claims the project name automatically. After +that the personal account can be removed from the project; the trust is bound +to `microsoft/lisa` + `publish.yml`, not to the person. + +### 2. GitHub Environments + +In `microsoft/lisa` → **Settings → Environments**, create two environments: + +| Environment | Required reviewers | Purpose | +|---|---|---| +| `testpypi` | (none, optional) | Auto-publishes pre-release artifacts | +| `pypi` | 1–2 LSG maintainers | Human approval gate before PyPI | + +Reviewers approve via the Actions run UI when the workflow pauses on the `pypi` +environment. + +### 3. Tag protection (recommended) + +**Settings → Tags → Add rule** → pattern `2[0-9][0-9][0-9][0-9][0-9][0-9][0-9].*`, +restrict push to release managers. Prevents accidental tag-based releases. + +--- + +## Cutting a release + +LISA uses **CalVer** tags in the form `YYYYMMDD.N` (e.g. `20260420.1`, +`20260420.2`). `setuptools_scm` derives the package version directly from the +tag, so the PyPI version equals the tag (no `v` prefix). + +1. Make sure `main` is green. +2. Pick today's date and the next sequence number for that day. +3. Update `CHANGELOG.md` (or release notes draft) and merge. +4. Tag and push: + ```bash + git checkout main + git pull --ff-only + TAG=$(date +%Y%m%d).1 # bump .1 -> .2 if a tag for today exists + git tag -a "$TAG" -m "$TAG" + git push origin "$TAG" + ``` +5. Watch **Actions → Publish to PyPI**. +6. After the `publish-testpypi` job succeeds, smoke test in a clean venv: + ```powershell + $TAG = "20260513.1" # use the tag you just pushed + py -3.12 -m venv C:\tmp\mslisa-rc + & C:\tmp\mslisa-rc\Scripts\python.exe -m pip install ` + --index-url https://test.pypi.org/simple/ ` + --extra-index-url https://pypi.org/simple/ ` + "mslisa[azure]==$TAG" + & C:\tmp\mslisa-rc\Scripts\lisa.exe --help + ``` +7. Approve the `pypi` environment in the workflow run. +8. Verify the live release: + ```bash + pip install --upgrade "mslisa==20260513.1" + lisa --version + ``` + +> A version that has been uploaded to PyPI **cannot** be replaced. To fix a bad +> release, yank it on PyPI and publish a new patch version. + +--- + +## Local dry run (no upload) + +Use this any time you change packaging metadata, `MANIFEST.in`, or +`pyproject.toml`: + +```powershell +cd lisa +.\.venv\Scripts\python.exe -m pip install --upgrade build twine +.\.venv\Scripts\python.exe -m build --wheel # wheel only on Windows; + # CI builds sdist on Linux +.\.venv\Scripts\python.exe -m twine check dist\* + +# Try installing into a fresh venv +py -3.12 -m venv C:\tmp\mslisa-local +$wheel = (Get-Item dist\mslisa-*.whl).FullName +& C:\tmp\mslisa-local\Scripts\python.exe -m pip install "$wheel[azure]" +& C:\tmp\mslisa-local\Scripts\lisa.exe --help +``` + +--- + +## Known limitations + +- **sdist build fails on Windows** because `setuptools_scm` includes every git- + tracked file (including deeply nested logs under `lisa/ai/data/...`) and the + resulting paths exceed Windows' 260-character limit. CI builds on Linux are + unaffected. The wheel is what users actually install. +- **`MANIFEST.in` `prune` rules don't apply** to files already tracked by git + when `setuptools_scm` is the file finder. To shrink the sdist, move + `lisa/ai/data/` out of git (git-lfs or a sibling repo). +- **Optional extras** (`baremetal`, `aws`, `ai`, etc.) are *not* installed by + `pip install mslisa`. Users opt in with `pip install "mslisa[azure,ai,…]"`.