Skip to content

[fix][CI] Don't run "Pulsar CI checks completed" too early and fix disk space issue in saving docker image #17584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lhotari merged 4 commits into from
Sep 12, 2022
Merged

[fix][CI] Don't run "Pulsar CI checks completed" too early and fix disk space issue in saving docker image #17584

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fdb5488
[fix][CI] Don't run "Pulsar CI checks completed" too early
lhotari Sep 12, 2022
13bf4af
Clean up disk space in "Build Pulsar docker image"
lhotari Sep 12, 2022
add4e21
Show disk space before and after cleaning
lhotari Sep 12, 2022
28aa9ff
Fix deleting files
lhotari Sep 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/pulsar-ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -555,6 +555,30 @@ jobs:
-Pmain,docker -Dmaven.test.skip=true -Ddocker.squash=true \
-Dspotbugs.skip=true -Dlicense.skip=true -Dcheckstyle.skip=true -Drat.skip=true

- name: Clean up disk space
run: |
# release disk space since saving docker image consumes local disk space
#
echo "::group::Available diskspace before cleaning"
time df -BM / /mnt
echo "::endgroup::"
echo "::group::Clean build directory"
# docker build changes some files to root ownership, fix this before deleting files
sudo chown -R $USER:$GROUP .
# clean build directories
time git clean -fdx
echo "::endgroup::"
echo "::group::Available diskspace after cleaning build directory"
time df -BM / /mnt
echo "::endgroup::"
echo "::group::Delete maven repository"
# delete maven repository
time rm -rf ~/.m2/repository
echo "::endgroup::"
echo "::group::Available diskspace after cleaning maven repository"
time df -BM / /mnt
echo "::endgroup::"

- name: save docker image apachepulsar/pulsar-test-latest-version:latest to Github artifact cache
run: |
$GITHUB_WORKSPACE/build/pulsar_ci_tool.sh docker_save_image_to_github_actions_artifacts apachepulsar/pulsar-test-latest-version:latest pulsar-test-latest-version-image
Expand Down Expand Up @@ -844,7 +868,6 @@ jobs:
'system-tests',
'macos-build'
]
if: always()
Copy link
Member

@tisonkun tisonkun Sep 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ever wrote another script for running always but check the result of needs to determinate which cleanups can be trigger.

See also https://github.com/apache/incubator-kvrocks/blob/2402dfec05c582235dfb195bb1d4210584e59848/.github/workflows/kvrocks.yaml#L220-L233.

Copy link
Member Author

@lhotari lhotari Sep 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, nice example. I didn't know about that ${{ needs.build-and-test.result }} trick.

steps:
- name: checkout
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
Expand Down