From 05d8da0fd1336dd2f06d16ec726c61063e4296c7 Mon Sep 17 00:00:00 2001 From: BInwei Yang Date: Wed, 15 Jan 2025 10:04:30 -0800 Subject: [PATCH 1/3] Update Velox.md --- docs/get-started/Velox.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/get-started/Velox.md b/docs/get-started/Velox.md index 8c8cb7bbd818..61ffb6d08b36 100644 --- a/docs/get-started/Velox.md +++ b/docs/get-started/Velox.md @@ -9,7 +9,7 @@ parent: Getting-Started | Type | Version | |-------|------------------------------| -| Spark | 3.2.2, 3.3.1, 3.4.3, 3.5.1 | +| Spark | 3.2.2, 3.3.1, 3.4.3, 3.5.2 | | OS | Ubuntu20.04/22.04, Centos7/8 | | jdk | openjdk8/jdk17 | | scala | 2.12 | @@ -18,7 +18,7 @@ parent: Getting-Started Currently, with static build Gluten+Velox backend supports all the Linux OSes, but is only tested on **Ubuntu20.04/Ubuntu22.04/Centos7/Centos8**. With dynamic build, Gluten+Velox backend support **Ubuntu20.04/Ubuntu22.04/Centos7/Centos8** and their variants. -Currently, the officially supported Spark versions are 3.2.2, 3.3.1, 3.4.3 and 3.5.1. +Currently, the officially supported Spark versions are 3.2.2, 3.3.1, 3.4.3 and 3.5.2. We need to set up the `JAVA_HOME` env. Currently, Gluten supports **java 8** and **java 17**. From 9f7b064d292a9e710b1f3d2bbe325b2e7c052c3b Mon Sep 17 00:00:00 2001 From: BInwei Yang Date: Wed, 15 Jan 2025 10:05:49 -0800 Subject: [PATCH 2/3] Update build-guide.md --- docs/get-started/build-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/get-started/build-guide.md b/docs/get-started/build-guide.md index d9c3beaab9dc..f135999cc608 100644 --- a/docs/get-started/build-guide.md +++ b/docs/get-started/build-guide.md @@ -74,4 +74,4 @@ It's name pattern is `gluten--bundle-spark_< | 3.2.2 | 3.2 | 2.12 | | 3.3.1 | 3.3 | 2.12 | | 3.4.3 | 3.4 | 2.12 | -| 3.5.1 | 3.5 | 2.12 | +| 3.5.2 | 3.5 | 2.12 | From 73a0a39d7304dd710b5706898be1587d4416896d Mon Sep 17 00:00:00 2001 From: BInwei Yang Date: Wed, 15 Jan 2025 11:05:56 -0800 Subject: [PATCH 3/3] Update Velox.md --- docs/get-started/Velox.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/get-started/Velox.md b/docs/get-started/Velox.md index 61ffb6d08b36..6ed65f283e47 100644 --- a/docs/get-started/Velox.md +++ b/docs/get-started/Velox.md @@ -113,11 +113,11 @@ Alternatively you may refer to [build in docker](docs/developers/velox-backend-b With build option `enable_vcpkg=ON`, all dependency libraries will be statically linked to `libvelox.so` and `libgluten.so` which are packed into the gluten-jar. In this way, only the gluten-jar is needed to add to `spark..extraClassPath` and spark will deploy the jar to each worker node. It's better to build the static version using a clean docker image without any extra libraries installed ( [build in docker](docs/developers/velox-backend-build-in-docker.md) ). On host with -some libraries like jemalloc installed, the script may crash with odd message. You may need to uninstall those libraries to get a clean host. We ** strongly recommend ** user to build Gluten in this way to avoid dependency lacking issue. +some libraries like jemalloc installed, the script may crash with odd message. You may need to uninstall those libraries to get a clean host. We **strongly recommend** user to build Gluten in this way to avoid dependency lacking issue. With build option `enable_vcpkg=OFF`, not all dependency libraries will be dynamically linked. After building, you need to separately execute `./dev/build-thirdparty.sh` to pack required shared libraries into another jar named `gluten-thirdparty-lib-$LINUX_OS-$VERSION-$ARCH.jar`. Then you need to add the jar to Spark config `extraClassPath` and -set `spark.gluten.loadLibFromJar=true`. Otherwise, you need to install required shared libraries with ** exactly the same versions ** on each worker node . You may find the +set `spark.gluten.loadLibFromJar=true`. Otherwise, you need to install required shared libraries with **exactly the same versions** on each worker node . You may find the libraries list from the third-party jar. # Remote storage support