[Enhancement](maxCompute)Implement compatibility with existing maxcompute catalogs from previous versions. #41386

hubgeter · 2024-09-27T03:51:52Z

before pr #40225

Proposed changes

Issue Number: close #xxx

…vious versions.

doris-robot · 2024-09-27T03:51:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

hubgeter · 2024-09-27T10:45:05Z

run buildall

doris-robot · 2024-09-27T11:21:36Z

TeamCity be ut coverage result:
Function Coverage: 37.30% (9627/25811)
Line Coverage: 28.70% (79677/277599)
Region Coverage: 28.13% (41197/146444)
Branch Coverage: 24.75% (20980/84758)
Coverage Report: http://coverage.selectdb-in.cc/coverage/601635fc9b047407dcc7dca109f46f0dc3178fd8_601635fc9b047407dcc7dca109f46f0dc3178fd8/report/index.html

morningman · 2024-09-27T16:11:52Z

...ions/max-compute-scanner/src/main/java/org/apache/doris/maxcompute/MaxComputeJniScanner.java

+            timeZone = ZoneId.of(timeZoneName);
+        } catch (Exception e) {
+            LOG.info(e.getMessage());
+            LOG.info("set timeZoneName = " + timeZoneName + "fail, use systemDefault.");


use LOG.warn, and use one line.

morningman · 2024-09-27T16:12:36Z

fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalCatalog.java

 import java.util.stream.Collectors;

 public class MaxComputeExternalCatalog extends ExternalCatalog {
+    private static final String endpointTemplate = "http://service.{}.maxcompute.aliyun-inc.com/api";


Add aliyun doc link about this endpoint.

ok . i will add it

morningman · 2024-09-27T16:13:14Z

fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalCatalog.java

+        if (props.containsKey(MCProperties.ENDPOINT)) {
+            endpoint = props.get(MCProperties.ENDPOINT);
+            return;
+        } else if (props.containsKey(MCProperties.TUNNEL_SDK_ENDPOINT)) {


Add some example in comment to explain this conversion.

ok . i will add it

morningman · 2024-09-27T16:15:28Z

fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/source/MaxComputeScanNode.java

-                ScalarType dstType = ScalarType.createDatetimeV2Type(3);
-                return  " \"" + dateLiteral.getStringValue(dstType) + "\" ";
-            }
+            // case DATETIME: {


why comment out?

I should determine which time zone the datetime needs to be passed in.

morningman · 2024-09-29T15:07:39Z

fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalCatalog.java

 import com.google.common.collect.ImmutableList;
 import com.google.gson.JsonObject;
 import com.google.gson.JsonParser;
+import jdk.internal.org.jline.utils.Log;


wrong log import

hubgeter · 2024-09-29T15:15:17Z

run buildall

doris-robot · 2024-09-29T15:59:17Z

TeamCity be ut coverage result:
Function Coverage: 37.29% (9627/25815)
Line Coverage: 28.69% (79689/277776)
Region Coverage: 28.13% (41220/146535)
Branch Coverage: 24.74% (20987/84816)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1bb3914bd3b20ac4a3208e10d2382e4540b72cf4_1bb3914bd3b20ac4a3208e10d2382e4540b72cf4/report/index.html

morningman

LGTM

github-actions · 2024-10-08T06:08:31Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-10-08T06:08:34Z

PR approved by anyone and no changes requested.

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

…pute catalogs from previous versions. (apache#41386) before pr apache#40225 (cherry picked from commit e8a1a16)

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

… API.(#40225 , #40888 ,#41386 ) (#41614) bp #40225 , #40888 ,#41386

… API.(#40225 , #40888 ,#41386 ) (#41610) bp #40225 , #40888 ,#41386 ## Proposed changes Among them, #40225 is the new api of mc, #40888 is used to fix the bug when reading null between the new and old apis, #41386 is used for compatibility between the new and old versions

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

… Storage API.(apache#40225 , apache#40888 ,apache#41386 ) (apache#41610)" This reverts commit 8c0f73c.

[Enhancement](maxCompute)Achieve compatibility with catalogs from pre…

82eeec7

…vious versions.

hubgeter added 2 commits September 27, 2024 18:40

fix time zone

15a3048

fix error

601635f

morningman reviewed Sep 27, 2024

View reviewed changes

done.

754f122

hubgeter requested a review from CalvinKirs as a code owner September 29, 2024 12:40

morningman reviewed Sep 29, 2024

View reviewed changes

fix log

1bb3914

morningman approved these changes Oct 8, 2024

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 8, 2024

wuwenchi approved these changes Oct 8, 2024

View reviewed changes

CalvinKirs approved these changes Oct 8, 2024

View reviewed changes

morningman merged commit e8a1a16 into apache:master Oct 8, 2024

morningman added dev/2.1.x dev/3.0.x labels Oct 8, 2024

hubgeter added a commit to hubgeter/doris that referenced this pull request Oct 9, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

d212a98

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

hubgeter mentioned this pull request Oct 9, 2024

[Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API.(#40225 , #40888 ,#41386 ) #41610

Merged

hubgeter added a commit to hubgeter/doris that referenced this pull request Oct 9, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

410922f

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

hubgeter mentioned this pull request Oct 9, 2024

[Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API.(#40225 , #40888 ,#41386 ) #41614

Merged

hubgeter added a commit to hubgeter/doris that referenced this pull request Oct 10, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

8a9a3c9

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

hubgeter added a commit to hubgeter/doris that referenced this pull request Oct 10, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

9fa6aae

…pute catalogs from previous versions. (apache#41386) before pr apache#40225 (cherry picked from commit e8a1a16)

eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 10, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

3982657

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

morningman pushed a commit that referenced this pull request Oct 11, 2024

[Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage…

a1f8383

… API.(#40225 , #40888 ,#41386 ) (#41614) bp #40225 , #40888 ,#41386

morningman added dev/3.0.3-merged and removed dev/3.0.x labels Oct 11, 2024

morningman added dev/2.1.7-merged and removed dev/2.1.x labels Oct 11, 2024

cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

12cf2b0

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

amorynan pushed a commit to amorynan/doris that referenced this pull request Oct 12, 2024

[Enhancement](maxCompute)Implement compatibility with existing maxcom…

dc05aee

…pute catalogs from previous versions. (apache#41386) before pr apache#40225

hubgeter added a commit to hubgeter/doris that referenced this pull request Nov 20, 2024

Revert "[Enhancement](MaxCompute)Refactoring maxCompute catalog using…

edab727

… Storage API.(apache#40225 , apache#40888 ,apache#41386 ) (apache#41610)" This reverts commit 8c0f73c.

hubgeter added a commit to hubgeter/doris that referenced this pull request Nov 20, 2024

Revert "[Enhancement](MaxCompute)Refactoring maxCompute catalog using…

b551c95

… Storage API.(apache#40225 , apache#40888 ,apache#41386 ) (apache#41610)" This reverts commit 8c0f73c.

[Enhancement](maxCompute)Implement compatibility with existing maxcompute catalogs from previous versions. #41386

[Enhancement](maxCompute)Implement compatibility with existing maxcompute catalogs from previous versions. #41386

Uh oh!

Conversation

hubgeter commented Sep 27, 2024

Proposed changes

Uh oh!

doris-robot commented Sep 27, 2024

Uh oh!

hubgeter commented Sep 27, 2024

Uh oh!

doris-robot commented Sep 27, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hubgeter commented Sep 29, 2024

Uh oh!

doris-robot commented Sep 29, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants