[VL] Change to use Velox's wget_and_untar in setup-centos7.sh#9207
[VL] Change to use Velox's wget_and_untar in setup-centos7.sh#9207philo-he merged 1 commit intoapache:mainfrom
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
ep/build-velox/src/setup-centos7.sh
Outdated
| @@ -48,14 +48,29 @@ function yum_install { | |||
|
|
|||
| function wget_and_untar { | |||
There was a problem hiding this comment.
@yaooqinn, thanks for work!
I note upstream velox's wget_and_untar has a check for dir to let user decide whether to re-download the tar. Does it meet our needs?
Actually, by removing this wget_and_untar function, Velox's wget_and_untar (placed in setup-helper-functions.sh) will be used in this script. I am wondering whether this improvement can be put in upstream velox. Then, both setup-centos7.sh and setup-centos8.sh (maintained in Gluten) can use that, as well as setup-centos9.sh and setup-ubuntu.sh (maintained in Velox).
There was a problem hiding this comment.
Hi, @philo-he. Thanks for the suggestion.
I had considered using velox's wget_and_untar, but I didn't apply it here.
- Some of the dependency names might conflict as these names are in the form of
vx.y.z. - velox's wget_and_untar does not check whether the tar ball is valid or not.
I am wondering whether this improvement can be put in upstream velox.
This is a good suggestion, I will try to send a PR to the upstream. But how about we focus the dependency management here first
There was a problem hiding this comment.
On second thought, I agree w/ you about removing the wget_and_untar and use the upstream one
philo-he
left a comment
There was a problem hiding this comment.
Thanks for your continuous work!
This issue was related to the recent pr #9207 which changes the script to use upstream Velox wget_and_untar function.
What changes were proposed in this pull request?
This PR makes the dependency installation look up the local download source tar tall first.
We can reduce some network issues for developers while they build their own CI or just daily dev.
How was this patch tested?
tested locally