[feat](cloud) S3 storage vault support property `use_path_style` #43060

swjtu-zhanglei · 2024-11-01T06:29:59Z

example

  CREATE STORAGE VAULT IF NOT EXISTS demo_vault
    PROPERTIES (
      "type"="S3",
      ...
      "use_path_style" = "true"
  );

doc apache/doris-website#1344

doris-robot · 2024-11-01T06:30:03Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

swjtu-zhanglei · 2024-11-01T06:30:20Z

run buildall

github-actions · 2024-11-01T06:35:01Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-11-01T07:38:40Z

TeamCity be ut coverage result:
Function Coverage: 37.82% (9820/25968)
Line Coverage: 28.98% (81577/281502)
Region Coverage: 28.26% (42144/149105)
Branch Coverage: 24.83% (21374/86070)
Coverage Report: http://coverage.selectdb-in.cc/coverage/261f97943bf7764d120d19caa554670d41aaff37_261f97943bf7764d120d19caa554670d41aaff37/report/index.html

dataroaring

LGTM

github-actions · 2024-11-01T08:15:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-01T08:15:27Z

PR approved by anyone and no changes requested.

github-actions · 2024-11-01T08:46:54Z

clang-tidy review says "All clean, LGTM! 👍"

gavinchou · 2024-11-01T14:31:40Z

fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/S3Properties.java

            builder.setProvider(Provider.valueOf(properties.get(S3Properties.PROVIDER).toUpperCase()));
        }
+
+        if (properties.containsKey(PropertyConverter.USE_PATH_STYLE)


can it be modified by alter vault stmt?

can it be modified by alter vault stmt?

done

swjtu-zhanglei · 2024-11-04T08:13:37Z

run buildall

github-actions · 2024-11-04T08:18:06Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-11-04T09:21:57Z

TeamCity be ut coverage result:
Function Coverage: 37.80% (9819/25974)
Line Coverage: 28.98% (81660/281810)
Region Coverage: 28.21% (42094/149194)
Branch Coverage: 24.80% (21361/86140)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6d7486e2d4b0ffca6842b076495c7a9cc0f0b5e6_6d7486e2d4b0ffca6842b076495c7a9cc0f0b5e6/report/index.html

swjtu-zhanglei · 2024-11-04T11:54:07Z

run buildall

github-actions · 2024-11-04T11:58:35Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-11-04T12:59:25Z

TeamCity be ut coverage result:
Function Coverage: 37.81% (9822/25975)
Line Coverage: 28.99% (81711/281818)
Region Coverage: 28.23% (42121/149203)
Branch Coverage: 24.81% (21375/86144)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ac77d1b29afb175eb61f09e22750a1158dd9fab2_ac77d1b29afb175eb61f09e22750a1158dd9fab2/report/index.html

gavinchou · 2024-11-04T15:43:51Z

fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/S3Properties.java

+            String value = properties.get(PropertyConverter.USE_PATH_STYLE);
+            Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "use_path_style cannot be empty");
+            Preconditions.checkArgument(value.equalsIgnoreCase("true")
+                    || value.equalsIgnoreCase("false"), "Invalid use_path_style value: %s", value);


"Invalid use_path_style value: %s, only 'true' or 'false' is acceptable"

"Invalid use_path_style value: %s, only 'true' or 'false' is acceptable"

resolved

gavinchou · 2024-11-04T15:45:04Z

fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/S3Properties.java

+            if (properties.get(PropertyConverter.USE_PATH_STYLE).equalsIgnoreCase("true")) {
+                builder.setUsePathStyle(true);
+            } else {
+                builder.setUsePathStyle(false);
+            }


Suggested change

if (properties.get(PropertyConverter.USE_PATH_STYLE).equalsIgnoreCase("true")) {

builder.setUsePathStyle(true);

} else {

builder.setUsePathStyle(false);

}

builder.setUsePathStyle(value.equalsIgnoreCase("true"));

maybe alter set use_path_style from true to false

swjtu-zhanglei · 2024-11-05T03:35:44Z

run buildall

github-actions · 2024-11-05T03:39:51Z

clang-tidy review says "All clean, LGTM! 👍"

swjtu-zhanglei · 2024-11-05T03:45:43Z

run buildall

github-actions · 2024-11-05T03:46:57Z

PR approved by at least one committer and no changes requested.

doris-robot · 2024-11-05T04:48:51Z

TeamCity be ut coverage result:
Function Coverage: 37.79% (9819/25981)
Line Coverage: 28.96% (81638/281927)
Region Coverage: 28.21% (42099/149239)
Branch Coverage: 24.79% (21357/86164)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1ff0dd6ee60f4c0c32f1237f8259e77d5de5aa2d_1ff0dd6ee60f4c0c32f1237f8259e77d5de5aa2d/report/index.html

swjtu-zhanglei · 2024-11-05T09:18:56Z

run buildall

github-actions · 2024-11-05T09:23:17Z

clang-tidy review says "All clean, LGTM! 👍"

swjtu-zhanglei · 2024-11-05T10:37:04Z

run buildall

github-actions · 2024-11-05T10:39:49Z

clang-tidy review says "All clean, LGTM! 👍"

``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

swjtu-zhanglei · 2024-11-06T01:54:33Z

run buildall

github-actions · 2024-11-06T01:57:43Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-11-06T02:23:15Z

TeamCity be ut coverage result:
Function Coverage: 37.86% (9837/25982)
Line Coverage: 29.03% (81810/281850)
Region Coverage: 28.27% (42167/149164)
Branch Coverage: 24.86% (21402/86100)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b5f9d28d0a63ad0fa6f863add6088fa625a2d2d2_b5f9d28d0a63ad0fa6f863add6088fa625a2d2d2/report/index.html

swjtu-zhanglei · 2024-11-06T03:59:55Z

run p0

swjtu-zhanglei · 2024-11-06T04:18:25Z

run cloud_p0

swjtu-zhanglei · 2024-11-06T05:38:40Z

run p0

dataroaring

LGTM

) example ``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

…che#43060) example ``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

…_style` (#43330) Cherry-picked from #43060 Co-authored-by: Lei Zhang <27994433+SWJTU-ZhangLei@users.noreply.github.com>

) (#43343) example ``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

dataroaring previously approved these changes Nov 1, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 1, 2024

github-actions bot added the reviewed label Nov 1, 2024

gavinchou reviewed Nov 1, 2024

View reviewed changes

swjtu-zhanglei dismissed dataroaring’s stale review via 6d7486e November 4, 2024 08:13

swjtu-zhanglei force-pushed the master-fix1 branch from 2046a26 to 6d7486e Compare November 4, 2024 08:13

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 4, 2024

swjtu-zhanglei force-pushed the master-fix1 branch from 6d7486e to ac77d1b Compare November 4, 2024 11:53

gavinchou reviewed Nov 4, 2024

View reviewed changes

swjtu-zhanglei force-pushed the master-fix1 branch from ac77d1b to f86ef7b Compare November 5, 2024 03:35

swjtu-zhanglei force-pushed the master-fix1 branch from f86ef7b to 1ff0dd6 Compare November 5, 2024 03:45

gavinchou approved these changes Nov 5, 2024

View reviewed changes

Yukang-Lian approved these changes Nov 5, 2024

View reviewed changes

[feat](cloud) S3 storage vault support property use_path_style

b5f9d28

``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

swjtu-zhanglei force-pushed the master-fix1 branch from c175d5b to b5f9d28 Compare November 6, 2024 01:53

dataroaring added dev/3.0.x p0_b labels Nov 6, 2024

dataroaring approved these changes Nov 6, 2024

View reviewed changes

dataroaring merged commit 4a7b504 into apache:master Nov 6, 2024

github-actions bot pushed a commit that referenced this pull request Nov 6, 2024

[feat](cloud) S3 storage vault support property use_path_style (#43060

85ac2e6

) example ``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

github-actions bot mentioned this pull request Nov 6, 2024

branch-3.0: [feat](cloud) S3 storage vault support property use_path_style #43330

Merged

swjtu-zhanglei mentioned this pull request Nov 6, 2024

[feat](cloud) S3 storage vault support property use_path_style (#43060) #43343

Merged

16 tasks

dataroaring pushed a commit that referenced this pull request Nov 7, 2024

branch-3.0: [feat](cloud) S3 storage vault support property `use_path…

3a47e3d

…_style` (#43330) Cherry-picked from #43060 Co-authored-by: Lei Zhang <27994433+SWJTU-ZhangLei@users.noreply.github.com>

dataroaring added dev/3.0.3-merged and removed dev/3.0.x labels Nov 7, 2024

gavinchou pushed a commit that referenced this pull request Nov 8, 2024

[feat](cloud) S3 storage vault support property use_path_style (#43060

16a83fb

) (#43343) example ``` CREATE STORAGE VAULT IF NOT EXISTS demo_vault PROPERTIES ( "type"="S3", ... "use_path_style" = "true" ); ```

gavinchou added cir and removed approved Indicates a PR has been approved by one committer. labels Nov 11, 2024

swjtu-zhanglei mentioned this pull request Nov 13, 2024

[opt] opt storage vault doc apache/doris-website#1344

Merged

4 tasks

gavinchou mentioned this pull request Nov 26, 2024

Release Note 3.0.3 #44522

Closed

[feat](cloud) S3 storage vault support property use_path_style #43060

[feat](cloud) S3 storage vault support property use_path_style #43060

Uh oh!

Conversation

swjtu-zhanglei commented Nov 1, 2024 • edited by gavinchou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Nov 1, 2024

Uh oh!

swjtu-zhanglei commented Nov 1, 2024

Uh oh!

github-actions bot commented Nov 1, 2024

Uh oh!

doris-robot commented Nov 1, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 1, 2024

Uh oh!

github-actions bot commented Nov 1, 2024

Uh oh!

github-actions bot commented Nov 1, 2024

Uh oh!

gavinchou Nov 1, 2024

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei Nov 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei commented Nov 4, 2024

Uh oh!

github-actions bot commented Nov 4, 2024

Uh oh!

doris-robot commented Nov 4, 2024

Uh oh!

swjtu-zhanglei commented Nov 4, 2024

Uh oh!

github-actions bot commented Nov 4, 2024

Uh oh!

doris-robot commented Nov 4, 2024

Uh oh!

gavinchou Nov 4, 2024

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei Nov 5, 2024

Choose a reason for hiding this comment

Uh oh!

gavinchou Nov 4, 2024

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei Nov 5, 2024

Choose a reason for hiding this comment

Uh oh!

swjtu-zhanglei commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024

Uh oh!

swjtu-zhanglei commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024

Uh oh!

doris-robot commented Nov 5, 2024

Uh oh!

swjtu-zhanglei commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024

Uh oh!

swjtu-zhanglei commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024

Uh oh!

swjtu-zhanglei commented Nov 6, 2024

Uh oh!

github-actions bot commented Nov 6, 2024

Uh oh!

doris-robot commented Nov 6, 2024

Uh oh!

swjtu-zhanglei commented Nov 6, 2024

[feat](cloud) S3 storage vault support property `use_path_style` #43060

[feat](cloud) S3 storage vault support property `use_path_style` #43060

swjtu-zhanglei commented Nov 1, 2024 •

edited by gavinchou

Loading

swjtu-zhanglei Nov 4, 2024 •

edited

Loading