-
Notifications
You must be signed in to change notification settings - Fork 0
Improve workflow to deploy build versions to Pypi #88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
120910f
Add GitHub Actions workflows for building and releasing to TestPyPI f…
Perdiga 8c812f4
Rename workflow to clarify its purpose: Release Feature to TestPyPI
Perdiga bc99181
Rename workflow to clarify its purpose: Release Feature Branch to Tes…
Perdiga 5ce0dca
Bump version to 0.3.0 in pyproject.toml
Perdiga 58ddbcb
Fix quoting in GitHub Actions output assignments in get_latest_build.sh
Perdiga File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,187 @@ | ||
| # Build and TestPyPI Release workflow for CodeQL Wrapper | ||
| # This workflow creates build versions (e.g., 0.1.12-build.1) and publishes them to TestPyPI | ||
|
|
||
| name: Release Feature Branch to TestPyPI | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - feature/* | ||
| paths: | ||
| - 'src/**' | ||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| PYTHON_VERSION: "3.13" | ||
| POETRY_VERSION: "1.8.3" | ||
|
|
||
| jobs: | ||
| test: | ||
| name: Run Tests | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ env.PYTHON_VERSION }} | ||
|
|
||
| - name: Install Poetry | ||
| uses: snok/install-poetry@v1 | ||
| with: | ||
| version: ${{ env.POETRY_VERSION }} | ||
| virtualenvs-create: true | ||
| virtualenvs-in-project: true | ||
|
|
||
| - name: Install project | ||
| run: poetry install --no-interaction | ||
|
|
||
| - name: Run linting | ||
| run: | | ||
| echo "Running code quality checks..." | ||
| poetry run poe quality | ||
| echo "All code quality checks passed" | ||
|
|
||
| build-and-release: | ||
| name: Build and Release | ||
| needs: [test] | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| packages: write | ||
| pull-requests: write | ||
| id-token: write | ||
| environment: | ||
| name: development | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ env.PYTHON_VERSION }} | ||
|
|
||
| - name: Install Poetry | ||
| uses: snok/install-poetry@v1 | ||
| with: | ||
| version: ${{ env.POETRY_VERSION }} | ||
| virtualenvs-create: true | ||
| virtualenvs-in-project: true | ||
|
|
||
| - name: Install project | ||
| run: poetry install --no-interaction | ||
|
|
||
| - name: Configure Git | ||
| run: | | ||
| git config --local user.email "action@github.com" | ||
| git config --local user.name "GitHub Action" | ||
|
|
||
| - name: Get latest build from TestPyPI and create new build version | ||
| id: get_testpypi_build | ||
| run: | | ||
| echo "Running build version detection script..." | ||
| ./.github/workflows/scripts/get_latest_build.sh | ||
|
|
||
| - name: Update package version | ||
| run: | | ||
| NEW_VERSION="${{ steps.get_testpypi_build.outputs.new_version }}" | ||
| echo "Updating version to $NEW_VERSION..." | ||
| poetry version $NEW_VERSION | ||
|
|
||
| echo "Reinstalling package with new version..." | ||
| poetry install --no-interaction | ||
|
|
||
| - name: Verify package version | ||
| run: | | ||
| NEW_VERSION="${{ steps.get_testpypi_build.outputs.new_version }}" | ||
| PACKAGE_VERSION=$(poetry run python -c "from codeql_wrapper import __version__; print(__version__)") | ||
| echo "Package version: $PACKAGE_VERSION" | ||
|
|
||
| # For build versions, we expect the version to match exactly | ||
| # since build versions like "0.1.12-build.1" are already in proper format | ||
| echo "Expected version: $NEW_VERSION" | ||
|
|
||
| if [ "$PACKAGE_VERSION" != "$NEW_VERSION" ]; then | ||
| echo "Package version mismatch. Expected $NEW_VERSION, got $PACKAGE_VERSION" | ||
| exit 1 | ||
| fi | ||
| echo "Package version verified" | ||
|
|
||
| - name: Build package | ||
| run: | | ||
| echo "Building package..." | ||
| poetry build | ||
|
|
||
| # Verify build artifacts | ||
| ls -la dist/ | ||
| echo "Package built successfully" | ||
|
|
||
| - name: Create bump build commit | ||
| run: | | ||
| NEW_VERSION="${{ steps.get_testpypi_build.outputs.new_version }}" | ||
|
|
||
| echo "Creating bump build commit on current branch..." | ||
|
|
||
| # Commit version bump to current branch | ||
| git add pyproject.toml | ||
| git commit -m "Bump version to $NEW_VERSION | ||
|
|
||
| - Bump version to $NEW_VERSION | ||
| - Update package metadata | ||
|
|
||
| Version Bumped via GitHub Actions" | ||
|
|
||
| echo "Bump build commit created on current branch" | ||
|
|
||
| - name: Push current branch | ||
| run: | | ||
| NEW_VERSION="${{ steps.get_testpypi_build.outputs.new_version }}" | ||
| CURRENT_BRANCH=$(git branch --show-current) | ||
|
|
||
| echo "Pushing current branch: $CURRENT_BRANCH" | ||
|
|
||
| # Push with error handling | ||
| if git push origin $CURRENT_BRANCH; then | ||
| echo "Current branch pushed successfully" | ||
| else | ||
| echo "Warning: Failed to push branch. This might be due to branch protection rules." | ||
| echo "The package will still be published to TestPyPI." | ||
| fi | ||
|
|
||
| - name: Publish to PyPI (Trusted Publishing) | ||
| if: false # Disabled for build versions - we only publish to TestPyPI | ||
| uses: pypa/gh-action-pypi-publish@release/v1 | ||
| with: | ||
| print-hash: true | ||
| verbose: true | ||
|
|
||
| - name: Publish to TestPyPI (Build Versions) | ||
| uses: pypa/gh-action-pypi-publish@release/v1 | ||
| with: | ||
| repository-url: https://test.pypi.org/legacy/ | ||
| print-hash: true | ||
| verbose: true | ||
| skip-existing: true | ||
|
|
||
| - name: Success Summary | ||
| run: | | ||
| NEW_VERSION="${{ steps.get_testpypi_build.outputs.new_version }}" | ||
|
|
||
| echo "BUILD VERSION PUBLISHED SUCCESSFULLY!" | ||
| echo "==================================" | ||
| echo "Version: v$NEW_VERSION" | ||
| echo "TestPyPI Package: Published (Build version)" | ||
| echo "TestPyPI: https://test.pypi.org/project/codeql-wrapper/$NEW_VERSION/" | ||
| echo "" | ||
| echo "Install command:" | ||
| echo "pip install --index-url https://test.pypi.org/simple/ codeql-wrapper==$NEW_VERSION" | ||
| echo "" | ||
| echo "This is a build version published to TestPyPI for testing purposes." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Script to get the latest build version from TestPyPI and create a new build version | ||
| # Location: .github/workflows/scripts/get_latest_build.sh | ||
| # Usage: ./.github/workflows/scripts/get_latest_build.sh | ||
| # | ||
| # This script: | ||
| # 1. Gets the base version from pyproject.toml | ||
| # 2. Queries TestPyPI for existing build versions | ||
| # 3. Determines the next build number | ||
| # 4. Outputs the new version and build information | ||
| # | ||
| # Outputs (via GitHub Actions output format): | ||
| # - new_version: The new build version to use (e.g., "0.1.12-build.3") | ||
| # - latest_build_number: The build number for the new version | ||
| # - latest_build_filename: The filename of the most recent build (if any) | ||
| # - build_count: Total number of existing builds for this base version | ||
|
|
||
| set -euo pipefail # Exit on error, undefined vars, pipe failures | ||
|
|
||
| # Configuration | ||
| PACKAGE="codeql-wrapper" | ||
|
|
||
| # Get base version from pyproject.toml | ||
| BASE_VERSION=$(poetry version --short) | ||
| echo "Detected base version from pyproject.toml: $BASE_VERSION" | ||
| echo "Looking for existing builds of $PACKAGE with pattern $BASE_VERSION-build.* on TestPyPI..." | ||
|
|
||
| # Query TestPyPI JSON API for the package with error handling | ||
| JSON_URL="https://test.pypi.org/pypi/${PACKAGE}/json" | ||
| echo "Querying TestPyPI API: $JSON_URL" | ||
|
|
||
| if ! RESPONSE=$(curl -s -f "$JSON_URL" 2>/dev/null); then | ||
| echo "Failed to query TestPyPI API or package not found" | ||
| NEW_VERSION="${BASE_VERSION}-build.1" | ||
|
|
||
| # Set outputs for GitHub Actions (if running in GitHub Actions) | ||
| if [ -n "${GITHUB_OUTPUT:-}" ]; then | ||
| echo "new_version=$NEW_VERSION" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_number=1" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_filename=" >> "$GITHUB_OUTPUT" | ||
| echo "build_count=0" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "Not running in GitHub Actions - outputs:" | ||
| echo " new_version=$NEW_VERSION" | ||
| echo " latest_build_number=1" | ||
| echo " latest_build_filename=" | ||
| echo " build_count=0" | ||
| fi | ||
| exit 0 | ||
| fi | ||
|
|
||
| # Extract all release versions from the package | ||
| ALL_VERSIONS=$(echo "$RESPONSE" | python3 -c "import sys, json; data=json.load(sys.stdin); [print(v) for v in data.get('releases', {}).keys()]" 2>/dev/null || echo "") | ||
|
|
||
| if [ -z "$ALL_VERSIONS" ]; then | ||
| echo "No versions found for $PACKAGE on TestPyPI." | ||
| NEW_VERSION="${BASE_VERSION}-build.1" | ||
|
|
||
| # Set outputs for GitHub Actions (if running in GitHub Actions) | ||
| if [ -n "${GITHUB_OUTPUT:-}" ]; then | ||
| echo "new_version=$NEW_VERSION" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_number=1" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_filename=" >> "$GITHUB_OUTPUT" | ||
| echo "build_count=0" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "Not running in GitHub Actions - outputs:" | ||
| echo " new_version=$NEW_VERSION" | ||
| echo " latest_build_number=1" | ||
| echo " latest_build_filename=" | ||
| echo " build_count=0" | ||
| fi | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "All versions found for $PACKAGE on TestPyPI:" | ||
| echo "$ALL_VERSIONS" | ||
|
|
||
| # Filter versions that match our base version with build pattern: {BASE_VERSION}-build.{N} | ||
| BUILD_VERSIONS=$(echo "$ALL_VERSIONS" | grep "^${BASE_VERSION}-build\." | sort -V || echo "") | ||
|
|
||
| if [ -z "$BUILD_VERSIONS" ]; then | ||
| echo "No build versions found for base version $BASE_VERSION." | ||
| NEW_VERSION="${BASE_VERSION}-build.1" | ||
| BUILD_NUMBER=1 | ||
| BUILD_COUNT=0 | ||
| LATEST_BUILD_FILENAME="" | ||
| else | ||
| echo "Build versions found for $BASE_VERSION:" | ||
| echo "$BUILD_VERSIONS" | ||
|
|
||
| # Extract build numbers from versions | ||
| BUILD_NUMBERS=$(echo "$BUILD_VERSIONS" | sed -n "s/^${BASE_VERSION}-build\.\([0-9]\+\)$/\1/p" | sort -n) | ||
|
|
||
| if [ -n "$BUILD_NUMBERS" ]; then | ||
| LATEST_BUILD_NUMBER=$(echo "$BUILD_NUMBERS" | tail -n 1) | ||
| BUILD_NUMBER=$((LATEST_BUILD_NUMBER + 1)) | ||
| echo "Found build numbers: $(echo $BUILD_NUMBERS | tr '\n' ' ')" | ||
| echo "Latest build number: $LATEST_BUILD_NUMBER" | ||
| else | ||
| BUILD_NUMBER=1 | ||
| echo "Could not extract build numbers, starting with 1" | ||
| fi | ||
|
|
||
| NEW_VERSION="${BASE_VERSION}-build.${BUILD_NUMBER}" | ||
| BUILD_COUNT=$(echo "$BUILD_VERSIONS" | wc -l | xargs) | ||
|
|
||
| # Get the latest build version and its filename | ||
| LATEST_BUILD_VERSION=$(echo "$BUILD_VERSIONS" | tail -n 1) | ||
| LATEST_BUILD_FILES=$(echo "$RESPONSE" | python3 -c "import sys, json; data=json.load(sys.stdin); [print(f['filename']) for f in data.get('releases', {}).get(\"$LATEST_BUILD_VERSION\", [])]" 2>/dev/null || echo "") | ||
| LATEST_BUILD_FILENAME=$(echo "$LATEST_BUILD_FILES" | tail -n 1) | ||
| fi | ||
|
|
||
| echo "New version will be: $NEW_VERSION" | ||
| echo "Build number: $BUILD_NUMBER" | ||
| echo "Latest build filename: $LATEST_BUILD_FILENAME" | ||
|
|
||
| # Set outputs for GitHub Actions (if running in GitHub Actions) | ||
| if [ -n "${GITHUB_OUTPUT:-}" ]; then | ||
| echo "new_version=$NEW_VERSION" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_number=$BUILD_NUMBER" >> "$GITHUB_OUTPUT" | ||
| echo "latest_build_filename=$LATEST_BUILD_FILENAME" >> "$GITHUB_OUTPUT" | ||
| echo "build_count=$BUILD_COUNT" >> "$GITHUB_OUTPUT" | ||
| echo "GitHub Actions outputs set." | ||
| else | ||
| echo "Not running in GitHub Actions - outputs:" | ||
| echo " new_version=$NEW_VERSION" | ||
| echo " latest_build_number=$BUILD_NUMBER" | ||
| echo " latest_build_filename=$LATEST_BUILD_FILENAME" | ||
| echo " build_count=$BUILD_COUNT" | ||
| fi | ||
|
|
||
| echo "Build version detection completed successfully." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The variable
$BUILD_NUMBERSshould be quoted to prevent word splitting. Change toecho "$BUILD_NUMBERS"to avoid potential issues with spaces in the output.