Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,6 @@
The architectural decisions and basis for the project in that document are only the inspiring foundation. It can and should always be challenged when needed and is not meant as the only truth, but a very useful context and grounding research.
- The project is meant for the Open Source community and should be open to contributions and follow the standards of the community.
- The project is meant to be used by developers and should be easy to use, with a focus on developer experience.
- The philosophy when architecting and implementing the project is to prime speed and simplicity over complexity. Do NOT implement backwards compatibility: ship fast. Do NOT over-engineer, but rather build a solid foundation that can be iterated on.
- We have no users, we do not need backwards compatibility, we do not need migration guides when we ship breaking changes, we do not need to worry.
- The goal is to deliver a solid and scalable architecture but simple starting implementation. Not building something complex from the start and then having to simplify it later. Remember we are delivering a new tool to the developer community and we will need to rapidly adapt to what's really useful, evolving standards, etc.
- The philosophy when architecting and implementing the project is to prime speed and simplicity over complexity. Do NOT over-engineer, but rather build a solid foundation that can be iterated on.
- APM is an active OSS project under the `microsoft` org with a growing community (250+ stars, external contributors). Breaking changes should be communicated clearly (CHANGELOG.md), but we still favor shipping fast over lengthy deprecation cycles.
- The goal is to deliver a solid and scalable architecture but simple starting implementation. Not building something complex from the start and then having to simplify it later. Remember we are delivering a new tool to the developer community and we will need to rapidly adapt to what's really useful, evolving standards, etc.
24 changes: 15 additions & 9 deletions .github/instructions/cicd.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,21 @@ description: "CI/CD Pipeline configuration for PyInstaller binary packaging and
Three workflows split by trigger and secret requirements:

1. **`ci.yml`** — `pull_request` trigger (all PRs, including forks)
- Unit tests + build. No secrets needed. Gives fast feedback.
- Uploads binary artifacts for downstream integration testing.
- **Linux-only** (ubuntu-24.04). Unit tests + single binary build. No secrets needed. Fast PR feedback (~3 min).
- Uploads Linux x86_64 binary artifact for downstream integration testing.
2. **`ci-integration.yml`** — `workflow_run` trigger (after CI completes, environment-gated)
- Smoke tests, integration tests, release validation. Requires `integration-tests` environment approval.
- **Linux-only**. Smoke tests, integration tests, release validation. Requires `integration-tests` environment approval.
- Security: uses `workflow_run` (not `pull_request_target`) — PR code is NEVER checked out.
- Downloads binary artifacts from ci.yml, runs test scripts from default branch (main).
- Downloads Linux binary artifact from ci.yml, runs test scripts from default branch (main).
- Reports results back to PR via commit status API.
3. **`build-release.yml`** — `push` to main, tags, schedule, `workflow_dispatch`
- Full pipeline for post-merge / release. Secrets always available.
- **Full 4-platform matrix** (linux x86_64/arm64, darwin x86_64/arm64). Secrets always available.
- macOS builds and cross-platform validation happen here, where queue time doesn't block PRs.

## Platform Testing Strategy
- **PR time**: Linux-only for speed. Catches logic bugs, dependency issues, and binary packaging problems.
- **Post-merge**: Full 4-platform matrix catches platform-specific issues immediately on main.
- **Rationale**: PR-time Linux coverage gives fast feedback on logic, dependency, and packaging changes, while the post-merge full-matrix workflows quickly catch any remaining platform-specific issues.

## PyInstaller Binary Packaging
- **CRITICAL**: Uses `--onedir` mode (NOT `--onefile`) for faster CLI startup performance
Expand All @@ -36,8 +42,8 @@ Three workflows split by trigger and secret requirements:
3. **Path Resolution**: Use symlinks and PATH manipulation for isolated binary testing

## Release Flow Dependencies
- **PR workflow**: ci.yml (test → build) then ci-integration.yml via workflow_run (approve → smoke-test → integration-tests → release-validation → report-status)
- **Push/Release workflow**: test → build → integration-tests → release-validation → create-release → publish-pypi → update-homebrew
- **PR workflow**: ci.yml (test → build, Linux-only) then ci-integration.yml via workflow_run (approve → smoke-test → integration-tests → release-validation → report-status, all Linux-only)
- **Push/Release workflow**: test → build → integration-tests → release-validation → create-release → publish-pypi → update-homebrew (full 4-platform matrix)
- **Tag Triggers**: Only `v*.*.*` tags trigger full release pipeline
- **Artifact Retention**: 30 days for debugging failed releases
- **Cross-workflow artifacts**: ci-integration.yml downloads artifacts from ci.yml using `run-id` and `github-token`
Expand All @@ -54,7 +60,7 @@ Three workflows split by trigger and secret requirements:
- `GITHUB_TOKEN` - Fallback token for compatibility (GitHub Actions built-in)

## Performance Considerations
- **PR CI is Linux-only**: Eliminates macOS runner queue delays (20-40+ min). Full platform coverage runs post-merge.
- UPX compression when available (reduces binary size ~50%)
- Python optimization level 2 in PyInstaller
- Aggressive module exclusions (tkinter, matplotlib, etc.)
- Matrix builds across platforms but sequential execution prevents resource conflicts
- Aggressive module exclusions (tkinter, matplotlib, etc.)
101 changes: 12 additions & 89 deletions .github/workflows/ci-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,30 +44,15 @@ jobs:
steps:
- run: echo "Internal PR auto-approved for ${{ github.event.workflow_run.head_branch }}"

# Linux-only for fast PR feedback. Full platform smoke tests run post-merge.
smoke-test:
needs: [approve-fork, approve-internal]
# Run if either approval job succeeded (the other will be skipped)
if: always() && (needs.approve-fork.result == 'success' || needs.approve-internal.result == 'success')
runs-on: ${{ matrix.os }}
runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
arch: x86_64
platform: linux
- os: ubuntu-24.04-arm
arch: arm64
platform: linux
- os: macos-15-intel
arch: x86_64
platform: darwin
- os: macos-latest
arch: arm64
platform: darwin

steps:
# Checkout default branch (main) — never PR code
Expand All @@ -83,14 +68,6 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Setup build environment (macOS-15 Intel)
if: matrix.os == 'macos-15-intel'
run: |
# Install Xcode Command Line Tools
sudo xcode-select --install 2>/dev/null || true
# Wait for installation to complete
until xcode-select -p >/dev/null 2>&1; do sleep 5; done

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
Expand All @@ -115,31 +92,12 @@ jobs:
GITHUB_APM_PAT: ${{ secrets.GH_CLI_PAT }}
run: uv run pytest tests/integration/test_runtime_smoke.py -v

# Linux-only — downloads the single Linux binary artifact from ci.yml.
integration-tests:
name: Integration Tests
needs: [smoke-test]
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
arch: x86_64
platform: linux
binary_name: apm-linux-x86_64
- os: ubuntu-24.04-arm
arch: arm64
platform: linux
binary_name: apm-linux-arm64
- os: macos-15-intel
arch: x86_64
platform: darwin
binary_name: apm-darwin-x86_64
- os: macos-latest
arch: arm64
platform: darwin
binary_name: apm-darwin-arm64

runs-on: ${{ matrix.os }}
if: needs.smoke-test.result == 'success'
runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
Expand All @@ -154,7 +112,7 @@ jobs:
- name: Download APM binary from CI workflow
uses: actions/download-artifact@v4
with:
name: ${{ matrix.binary_name }}
name: apm-linux-x86_64
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

Expand All @@ -168,14 +126,6 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Setup build environment (macOS-15-intel)
if: matrix.os == 'macos-15-intel'
run: |
# Install Xcode Command Line Tools
sudo xcode-select --install 2>/dev/null || true
# Wait for installation to complete
until xcode-select -p >/dev/null 2>&1; do sleep 5; done

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
Expand All @@ -195,31 +145,12 @@ jobs:
uv run ./scripts/test-integration.sh
timeout-minutes: 20

# Linux-only — validates the Linux binary in isolation. Full platform validation runs post-merge.
release-validation:
name: Release Validation
needs: [integration-tests]
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
arch: x86_64
platform: linux
binary_name: apm-linux-x86_64
- os: ubuntu-24.04-arm
arch: arm64
platform: linux
binary_name: apm-linux-arm64
- os: macos-15-intel
arch: x86_64
platform: darwin
binary_name: apm-darwin-x86_64
- os: macos-latest
arch: arm64
platform: darwin
binary_name: apm-darwin-arm64

runs-on: ${{ matrix.os }}
if: needs.integration-tests.result == 'success'
runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
Expand All @@ -243,19 +174,11 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Setup build environment (macOS-15-intel)
if: matrix.os == 'macos-15-intel'
run: |
# Install Xcode Command Line Tools
sudo xcode-select --install 2>/dev/null || true
# Wait for installation to complete
until xcode-select -p >/dev/null 2>&1; do sleep 5; done

# Download binary artifact to isolated test directory
- name: Download APM binary from CI workflow
uses: actions/download-artifact@v4
with:
name: ${{ matrix.binary_name }}
name: apm-linux-x86_64
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
path: /tmp/apm-isolated-test/
Expand All @@ -268,12 +191,12 @@ jobs:
find . -name "apm" -type f
ls -la ./dist/

chmod +x ./dist/${{ matrix.binary_name }}/apm
chmod +x ./dist/apm-linux-x86_64/apm

- name: Create APM symlink for testing
run: |
cd /tmp/apm-isolated-test
ln -s "$(pwd)/dist/${{ matrix.binary_name }}/apm" "$(pwd)/apm"
ln -s "$(pwd)/dist/apm-linux-x86_64/apm" "$(pwd)/apm"
echo "/tmp/apm-isolated-test" >> $GITHUB_PATH

- name: Prepare test scripts from default branch
Expand Down
69 changes: 8 additions & 61 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,26 +15,11 @@ permissions:
contents: read

jobs:
# Linux-only for fast PR feedback. Full platform matrix runs post-merge in build-release.yml.
test:
runs-on: ${{ matrix.os }}
runs-on: ubuntu-24.04
permissions:
contents: read
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
arch: x86_64
platform: linux
- os: ubuntu-24.04-arm
arch: arm64
platform: linux
- os: macos-15-intel
arch: x86_64
platform: darwin
- os: macos-latest
arch: arm64
platform: darwin

steps:
- uses: actions/checkout@v4
Expand All @@ -49,14 +34,6 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Setup build environment (macOS-15 Intel)
if: matrix.os == 'macos-15-intel'
run: |
# Install Xcode Command Line Tools
sudo xcode-select --install 2>/dev/null || true
# Wait for installation to complete
until xcode-select -p >/dev/null 2>&1; do sleep 5; done

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
Expand All @@ -78,31 +55,11 @@ jobs:
- name: Test with pytest
run: uv run pytest tests/unit tests/test_console.py

# Linux-only binary build for PR validation. Full platform builds run post-merge.
build:
name: Build APM Binary
needs: [test]
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
platform: linux
arch: x86_64
binary_name: apm-linux-x86_64
- os: ubuntu-24.04-arm
platform: linux
arch: arm64
binary_name: apm-linux-arm64
- os: macos-15-intel
platform: darwin
arch: x86_64
binary_name: apm-darwin-x86_64
- os: macos-latest
platform: darwin
arch: arm64
binary_name: apm-darwin-arm64

runs-on: ${{ matrix.os }}
runs-on: ubuntu-24.04
permissions:
contents: read

Expand All @@ -115,21 +72,11 @@ jobs:
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install UPX (Linux)
if: matrix.platform == 'linux'
- name: Install UPX
run: |
sudo apt-get update
sudo apt-get install -y upx-ucl

- name: Install UPX and XCode (macOS)
if: matrix.platform == 'darwin'
run: |
# Install Xcode Command Line Tools
sudo xcode-select --install 2>/dev/null || true
# Wait for installation to complete
until xcode-select -p >/dev/null 2>&1; do sleep 5; done
brew install upx

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
Expand All @@ -147,10 +94,10 @@ jobs:
- name: Upload binary as workflow artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.binary_name }}
name: apm-linux-x86_64
path: |
./dist/${{ matrix.binary_name }}
./dist/${{ matrix.binary_name }}.sha256
./dist/apm-linux-x86_64
./dist/apm-linux-x86_64.sha256
./scripts/test-release-validation.sh
./scripts/github-token-helper.sh
include-hidden-files: true
Expand Down