diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 13399c87..2bc4e2fa 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -15,6 +15,6 @@ The architectural decisions and basis for the project in that document are only the inspiring foundation. It can and should always be challenged when needed and is not meant as the only truth, but a very useful context and grounding research. - The project is meant for the Open Source community and should be open to contributions and follow the standards of the community. - The project is meant to be used by developers and should be easy to use, with a focus on developer experience. -- The philosophy when architecting and implementing the project is to prime speed and simplicity over complexity. Do NOT implement backwards compatibility: ship fast. Do NOT over-engineer, but rather build a solid foundation that can be iterated on. -- We have no users, we do not need backwards compatibility, we do not need migration guides when we ship breaking changes, we do not need to worry. -- The goal is to deliver a solid and scalable architecture but simple starting implementation. Not building something complex from the start and then having to simplify it later. Remember we are delivering a new tool to the developer community and we will need to rapidly adapt to what's really useful, evolving standards, etc. \ No newline at end of file +- The philosophy when architecting and implementing the project is to prime speed and simplicity over complexity. Do NOT over-engineer, but rather build a solid foundation that can be iterated on. +- APM is an active OSS project under the `microsoft` org with a growing community (250+ stars, external contributors). Breaking changes should be communicated clearly (CHANGELOG.md), but we still favor shipping fast over lengthy deprecation cycles. +- The goal is to deliver a solid and scalable architecture but simple starting implementation. Not building something complex from the start and then having to simplify it later. Remember we are delivering a new tool to the developer community and we will need to rapidly adapt to what's really useful, evolving standards, etc. \ No newline at end of file diff --git a/.github/instructions/cicd.instructions.md b/.github/instructions/cicd.instructions.md index e3921ffd..31a42582 100644 --- a/.github/instructions/cicd.instructions.md +++ b/.github/instructions/cicd.instructions.md @@ -9,15 +9,21 @@ description: "CI/CD Pipeline configuration for PyInstaller binary packaging and Three workflows split by trigger and secret requirements: 1. **`ci.yml`** — `pull_request` trigger (all PRs, including forks) - - Unit tests + build. No secrets needed. Gives fast feedback. - - Uploads binary artifacts for downstream integration testing. + - **Linux-only** (ubuntu-24.04). Unit tests + single binary build. No secrets needed. Fast PR feedback (~3 min). + - Uploads Linux x86_64 binary artifact for downstream integration testing. 2. **`ci-integration.yml`** — `workflow_run` trigger (after CI completes, environment-gated) - - Smoke tests, integration tests, release validation. Requires `integration-tests` environment approval. + - **Linux-only**. Smoke tests, integration tests, release validation. Requires `integration-tests` environment approval. - Security: uses `workflow_run` (not `pull_request_target`) — PR code is NEVER checked out. - - Downloads binary artifacts from ci.yml, runs test scripts from default branch (main). + - Downloads Linux binary artifact from ci.yml, runs test scripts from default branch (main). - Reports results back to PR via commit status API. 3. **`build-release.yml`** — `push` to main, tags, schedule, `workflow_dispatch` - - Full pipeline for post-merge / release. Secrets always available. + - **Full 4-platform matrix** (linux x86_64/arm64, darwin x86_64/arm64). Secrets always available. + - macOS builds and cross-platform validation happen here, where queue time doesn't block PRs. + +## Platform Testing Strategy +- **PR time**: Linux-only for speed. Catches logic bugs, dependency issues, and binary packaging problems. +- **Post-merge**: Full 4-platform matrix catches platform-specific issues immediately on main. +- **Rationale**: PR-time Linux coverage gives fast feedback on logic, dependency, and packaging changes, while the post-merge full-matrix workflows quickly catch any remaining platform-specific issues. ## PyInstaller Binary Packaging - **CRITICAL**: Uses `--onedir` mode (NOT `--onefile`) for faster CLI startup performance @@ -36,8 +42,8 @@ Three workflows split by trigger and secret requirements: 3. **Path Resolution**: Use symlinks and PATH manipulation for isolated binary testing ## Release Flow Dependencies -- **PR workflow**: ci.yml (test → build) then ci-integration.yml via workflow_run (approve → smoke-test → integration-tests → release-validation → report-status) -- **Push/Release workflow**: test → build → integration-tests → release-validation → create-release → publish-pypi → update-homebrew +- **PR workflow**: ci.yml (test → build, Linux-only) then ci-integration.yml via workflow_run (approve → smoke-test → integration-tests → release-validation → report-status, all Linux-only) +- **Push/Release workflow**: test → build → integration-tests → release-validation → create-release → publish-pypi → update-homebrew (full 4-platform matrix) - **Tag Triggers**: Only `v*.*.*` tags trigger full release pipeline - **Artifact Retention**: 30 days for debugging failed releases - **Cross-workflow artifacts**: ci-integration.yml downloads artifacts from ci.yml using `run-id` and `github-token` @@ -54,7 +60,7 @@ Three workflows split by trigger and secret requirements: - `GITHUB_TOKEN` - Fallback token for compatibility (GitHub Actions built-in) ## Performance Considerations +- **PR CI is Linux-only**: Eliminates macOS runner queue delays (20-40+ min). Full platform coverage runs post-merge. - UPX compression when available (reduces binary size ~50%) - Python optimization level 2 in PyInstaller -- Aggressive module exclusions (tkinter, matplotlib, etc.) -- Matrix builds across platforms but sequential execution prevents resource conflicts \ No newline at end of file +- Aggressive module exclusions (tkinter, matplotlib, etc.) \ No newline at end of file diff --git a/.github/workflows/ci-integration.yml b/.github/workflows/ci-integration.yml index 1950704f..7e8286be 100644 --- a/.github/workflows/ci-integration.yml +++ b/.github/workflows/ci-integration.yml @@ -44,30 +44,15 @@ jobs: steps: - run: echo "Internal PR auto-approved for ${{ github.event.workflow_run.head_branch }}" + # Linux-only for fast PR feedback. Full platform smoke tests run post-merge. smoke-test: needs: [approve-fork, approve-internal] # Run if either approval job succeeded (the other will be skipped) if: always() && (needs.approve-fork.result == 'success' || needs.approve-internal.result == 'success') - runs-on: ${{ matrix.os }} + runs-on: ubuntu-24.04 permissions: contents: read actions: read - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-24.04 - arch: x86_64 - platform: linux - - os: ubuntu-24.04-arm - arch: arm64 - platform: linux - - os: macos-15-intel - arch: x86_64 - platform: darwin - - os: macos-latest - arch: arm64 - platform: darwin steps: # Checkout default branch (main) — never PR code @@ -83,14 +68,6 @@ jobs: with: python-version: ${{ env.PYTHON_VERSION }} - - name: Setup build environment (macOS-15 Intel) - if: matrix.os == 'macos-15-intel' - run: | - # Install Xcode Command Line Tools - sudo xcode-select --install 2>/dev/null || true - # Wait for installation to complete - until xcode-select -p >/dev/null 2>&1; do sleep 5; done - - name: Install uv run: | curl -LsSf https://astral.sh/uv/install.sh | sh @@ -115,31 +92,12 @@ jobs: GITHUB_APM_PAT: ${{ secrets.GH_CLI_PAT }} run: uv run pytest tests/integration/test_runtime_smoke.py -v + # Linux-only — downloads the single Linux binary artifact from ci.yml. integration-tests: name: Integration Tests needs: [smoke-test] - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-24.04 - arch: x86_64 - platform: linux - binary_name: apm-linux-x86_64 - - os: ubuntu-24.04-arm - arch: arm64 - platform: linux - binary_name: apm-linux-arm64 - - os: macos-15-intel - arch: x86_64 - platform: darwin - binary_name: apm-darwin-x86_64 - - os: macos-latest - arch: arm64 - platform: darwin - binary_name: apm-darwin-arm64 - - runs-on: ${{ matrix.os }} + if: needs.smoke-test.result == 'success' + runs-on: ubuntu-24.04 permissions: contents: read actions: read @@ -154,7 +112,7 @@ jobs: - name: Download APM binary from CI workflow uses: actions/download-artifact@v4 with: - name: ${{ matrix.binary_name }} + name: apm-linux-x86_64 run-id: ${{ github.event.workflow_run.id }} github-token: ${{ secrets.GITHUB_TOKEN }} @@ -168,14 +126,6 @@ jobs: with: python-version: ${{ env.PYTHON_VERSION }} - - name: Setup build environment (macOS-15-intel) - if: matrix.os == 'macos-15-intel' - run: | - # Install Xcode Command Line Tools - sudo xcode-select --install 2>/dev/null || true - # Wait for installation to complete - until xcode-select -p >/dev/null 2>&1; do sleep 5; done - - name: Install uv run: | curl -LsSf https://astral.sh/uv/install.sh | sh @@ -195,31 +145,12 @@ jobs: uv run ./scripts/test-integration.sh timeout-minutes: 20 + # Linux-only — validates the Linux binary in isolation. Full platform validation runs post-merge. release-validation: name: Release Validation needs: [integration-tests] - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-24.04 - arch: x86_64 - platform: linux - binary_name: apm-linux-x86_64 - - os: ubuntu-24.04-arm - arch: arm64 - platform: linux - binary_name: apm-linux-arm64 - - os: macos-15-intel - arch: x86_64 - platform: darwin - binary_name: apm-darwin-x86_64 - - os: macos-latest - arch: arm64 - platform: darwin - binary_name: apm-darwin-arm64 - - runs-on: ${{ matrix.os }} + if: needs.integration-tests.result == 'success' + runs-on: ubuntu-24.04 permissions: contents: read actions: read @@ -243,19 +174,11 @@ jobs: with: python-version: ${{ env.PYTHON_VERSION }} - - name: Setup build environment (macOS-15-intel) - if: matrix.os == 'macos-15-intel' - run: | - # Install Xcode Command Line Tools - sudo xcode-select --install 2>/dev/null || true - # Wait for installation to complete - until xcode-select -p >/dev/null 2>&1; do sleep 5; done - # Download binary artifact to isolated test directory - name: Download APM binary from CI workflow uses: actions/download-artifact@v4 with: - name: ${{ matrix.binary_name }} + name: apm-linux-x86_64 run-id: ${{ github.event.workflow_run.id }} github-token: ${{ secrets.GITHUB_TOKEN }} path: /tmp/apm-isolated-test/ @@ -268,12 +191,12 @@ jobs: find . -name "apm" -type f ls -la ./dist/ - chmod +x ./dist/${{ matrix.binary_name }}/apm + chmod +x ./dist/apm-linux-x86_64/apm - name: Create APM symlink for testing run: | cd /tmp/apm-isolated-test - ln -s "$(pwd)/dist/${{ matrix.binary_name }}/apm" "$(pwd)/apm" + ln -s "$(pwd)/dist/apm-linux-x86_64/apm" "$(pwd)/apm" echo "/tmp/apm-isolated-test" >> $GITHUB_PATH - name: Prepare test scripts from default branch diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b9958536..a7079e88 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -15,26 +15,11 @@ permissions: contents: read jobs: + # Linux-only for fast PR feedback. Full platform matrix runs post-merge in build-release.yml. test: - runs-on: ${{ matrix.os }} + runs-on: ubuntu-24.04 permissions: contents: read - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-24.04 - arch: x86_64 - platform: linux - - os: ubuntu-24.04-arm - arch: arm64 - platform: linux - - os: macos-15-intel - arch: x86_64 - platform: darwin - - os: macos-latest - arch: arm64 - platform: darwin steps: - uses: actions/checkout@v4 @@ -49,14 +34,6 @@ jobs: with: python-version: ${{ env.PYTHON_VERSION }} - - name: Setup build environment (macOS-15 Intel) - if: matrix.os == 'macos-15-intel' - run: | - # Install Xcode Command Line Tools - sudo xcode-select --install 2>/dev/null || true - # Wait for installation to complete - until xcode-select -p >/dev/null 2>&1; do sleep 5; done - - name: Install uv run: | curl -LsSf https://astral.sh/uv/install.sh | sh @@ -78,31 +55,11 @@ jobs: - name: Test with pytest run: uv run pytest tests/unit tests/test_console.py + # Linux-only binary build for PR validation. Full platform builds run post-merge. build: name: Build APM Binary needs: [test] - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-24.04 - platform: linux - arch: x86_64 - binary_name: apm-linux-x86_64 - - os: ubuntu-24.04-arm - platform: linux - arch: arm64 - binary_name: apm-linux-arm64 - - os: macos-15-intel - platform: darwin - arch: x86_64 - binary_name: apm-darwin-x86_64 - - os: macos-latest - platform: darwin - arch: arm64 - binary_name: apm-darwin-arm64 - - runs-on: ${{ matrix.os }} + runs-on: ubuntu-24.04 permissions: contents: read @@ -115,21 +72,11 @@ jobs: with: python-version: ${{ env.PYTHON_VERSION }} - - name: Install UPX (Linux) - if: matrix.platform == 'linux' + - name: Install UPX run: | sudo apt-get update sudo apt-get install -y upx-ucl - - name: Install UPX and XCode (macOS) - if: matrix.platform == 'darwin' - run: | - # Install Xcode Command Line Tools - sudo xcode-select --install 2>/dev/null || true - # Wait for installation to complete - until xcode-select -p >/dev/null 2>&1; do sleep 5; done - brew install upx - - name: Install uv run: | curl -LsSf https://astral.sh/uv/install.sh | sh @@ -147,10 +94,10 @@ jobs: - name: Upload binary as workflow artifact uses: actions/upload-artifact@v4 with: - name: ${{ matrix.binary_name }} + name: apm-linux-x86_64 path: | - ./dist/${{ matrix.binary_name }} - ./dist/${{ matrix.binary_name }}.sha256 + ./dist/apm-linux-x86_64 + ./dist/apm-linux-x86_64.sha256 ./scripts/test-release-validation.sh ./scripts/github-token-helper.sh include-hidden-files: true