[Doris On ES][WIP] Support external ES table with `SSL` secured and configurable node sniffing by stalary · Pull Request #5325 · apache/doris

stalary · 2021-01-30T09:17:51Z

Proposed changes

Support doris connect https es server and support config esNodesDiscovery

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[] Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] Documentation Update (if none of the other choices apply)
[] Code refactor (Modify the code structure, format the code, etc...)

Issue: #5324

wuyunfeng

This is a very good job, I left some comment

be/src/exec/es/es_scan_reader.cpp

be/src/exec/es/es_scan_reader.h

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsNodeInfo.java

gensrc/thrift/Descriptors.thrift

fe/fe-core/src/main/java/org/apache/doris/catalog/EsTable.java

be/src/exec/es/es_scan_reader.h

be/src/http/http_client.h

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/PartitionPhase.java

fe/fe-core/src/main/java/org/apache/doris/catalog/EsTable.java

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/PartitionPhase.java

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java

docs/en/extending-doris/doris-on-es.md

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsNodeInfo.java

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsRestClient.java

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsShardPartitions.java

docs/en/extending-doris/doris-on-es.md

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsRestClient.java

docs/en/extending-doris/doris-on-es.md

wuyunfeng

LGTM

…add protocol prefix code, remove networkClient static

wuyunfeng

LGTM

…onfigurable node sniffing (apache#5325) Support external ES table with `SSL` secured and configurable node sniffing

wuyunfeng requested changes Feb 23, 2021

View reviewed changes

wuyunfeng reviewed Feb 23, 2021

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/catalog/EsTable.java Outdated Show resolved Hide resolved

wuyunfeng changed the title ~~WIP：Support doris connect https es server and support config esNodesDiscovery~~ [Doris On ES][WIP]：Support ES with SSL secured and support disabling ES nodes detecting Feb 23, 2021

wuyunfeng self-assigned this Feb 23, 2021

wuyunfeng added the area/doris-on-es Issues or PRs related to Doris on ElasticSearch label Feb 23, 2021

wuyunfeng changed the title ~~[Doris On ES][WIP]：Support ES with SSL secured and support disabling ES nodes detecting~~ [Doris On ES][WIP]：Support ES with SSL secured and disabling ES nodes detecting Feb 23, 2021

wuyunfeng changed the title ~~[Doris On ES][WIP]：Support ES with SSL secured and disabling ES nodes detecting~~ [Doris On ES][WIP]：Support ES with SSL secured and configurable node sniffing Feb 23, 2021

wuyunfeng changed the title ~~[Doris On ES][WIP]：Support ES with SSL secured and configurable node sniffing~~ [Doris On ES][WIP] Support external ES table with SSL secured and configurable node sniffing Feb 23, 2021

wuyunfeng linked an issue Feb 23, 2021 that may be closed by this pull request

[Doris On ES][Feature] Does Doris support external ES table with SSL secured and configurable node sniffing? #5324

Closed

wuyunfeng requested changes Feb 24, 2021

View reviewed changes

be/src/exec/es/es_scan_reader.h Outdated Show resolved Hide resolved

be/src/http/http_client.h Outdated Show resolved Hide resolved

wuyunfeng reviewed Feb 24, 2021

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/PartitionPhase.java Outdated Show resolved Hide resolved

wuyunfeng requested changes Feb 24, 2021

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/catalog/EsTable.java Outdated Show resolved Hide resolved

wuyunfeng requested changes Mar 3, 2021

View reviewed changes

apache deleted a comment from stalary Mar 4, 2021

wuyunfeng requested changes Mar 11, 2021

View reviewed changes

wuyunfeng requested changes Mar 22, 2021

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsRestClient.java Outdated Show resolved Hide resolved

fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsShardPartitions.java Outdated Show resolved Hide resolved

wuyunfeng requested changes Apr 7, 2021

View reviewed changes

docs/en/extending-doris/doris-on-es.md Outdated Show resolved Hide resolved

docs/en/extending-doris/doris-on-es.md Outdated Show resolved Hide resolved

wuyunfeng previously approved these changes Apr 9, 2021

View reviewed changes

wuyunfeng added the approved Indicates a PR has been approved by one committer. label Apr 9, 2021

stalary added 10 commits April 9, 2021 10:48

MOD: support fe connect https es

0e157df

MOD: add be ssl switch

d019eb9

MOD: modify some code

3437102

MOD: modify some code

b8fc83d

MOD: modify code format, add comment

6719303

MOD: modify propName from use_ssl_client to es_net_ssl,remove manual …

6eda420

…add protocol prefix code, remove networkClient static

ADD: add doc

465e0c0

MOD: modify networkClient use way, EsNodeInfo add useSslClient field

0bbf7e1

MOD: modify name

f0925b4

MOD: modify some code

76ac015

stalary added 3 commits April 9, 2021 10:48

MOD: modify doc

b75950b

MOD: modify doc

94458e8

MOD: modify doc

57e5c32

stalary dismissed wuyunfeng’s stale review via 57e5c32 April 9, 2021 02:51

stalary force-pushed the es_ssl branch from 6a4ce5b to 57e5c32 Compare April 9, 2021 02:51

wuyunfeng approved these changes Apr 12, 2021

View reviewed changes

wuyunfeng merged commit 75db273 into apache:master Apr 12, 2021

wuyunfeng mentioned this pull request Apr 12, 2021

FE 和 BE 节点正常，使用 Doris on es ，不能正确链接至 ES 地址 #5389

Closed

morningman mentioned this pull request Oct 10, 2021

Release note 0.15.0 #6806

Closed

stalary deleted the es_ssl branch May 8, 2022 10:11

Conversation

stalary commented Jan 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Types of changes

Uh oh!

wuyunfeng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wuyunfeng left a comment

Choose a reason for hiding this comment

Uh oh!

wuyunfeng left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stalary commented Jan 30, 2021 •

edited

Loading