Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Jun 15, 2023

Which issue does this PR close?

Closes #6676

Rationale for this change

As shown on #6676 (comment), disabling incremental complication saves 2GB of builder space

With incremental compilation enabled (the default):

https://github.com/apache/arrow-datafusion/actions/runs/5281745040/jobs/9555736433?pr=6678

Run du -s -h $GITHUB_WORKSPACE/target/debug/* || true
429M	/__w/arrow-datafusion/arrow-datafusion/target/debug/build
21G	/__w/arrow-datafusion/arrow-datafusion/target/debug/deps
4.0K	/__w/arrow-datafusion/arrow-datafusion/target/debug/examples
2.3G	/__w/arrow-datafusion/arrow-datafusion/target/debug/incremental

Without incremental compilation: https://github.com/apache/arrow-datafusion/actions/runs/5281960476/jobs/9556242542

Run du -s -h $GITHUB_WORKSPACE/target/debug/* || true
429M	/__w/arrow-datafusion/arrow-datafusion/target/debug/build
22G	/__w/arrow-datafusion/arrow-datafusion/target/debug/deps
4.0K	/__w/arrow-datafusion/arrow-datafusion/target/debug/examples
4.0K	/__w/arrow-datafusion/arrow-datafusion/target/debug/incremental

Total space used on the builder's filesystem

main (before this PR):

35G	/

without incremental compilation (this PR):

33G	/

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@alamb
Copy link
Contributor Author

alamb commented Jun 15, 2023

Given I think this PR gives us some more breathing room and doesn't change any code (just the build on CI), I plan to merge it if/when the tests pass. I will merge the other cleanup PRs for #6676 after review

@alamb alamb added the development-process Related to development process of DataFusion label Jun 15, 2023
@alamb alamb merged commit 7354c56 into apache:main Jun 15, 2023
@alamb alamb deleted the alamb/no_incremental_updates branch June 15, 2023 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI is failing due to "/usr/bin/ld: final link failed: No space left on device"

1 participant