OpenSearch migration for SQL/PPL #1

dai-chen · 2021-04-07T17:45:43Z

Issue #, if available:

Description of changes: If nothing else miss, the only leftover is the ES doc links in source code comments and documentation.

1.Changing to OpenSearch Dependencies

build.gradle
- Plugin version → 1.15.0
- Add private Maven S3 repo -> 1.13.0.0-Snapshot
- Replace all org.elasticsearch, ElasticsearchException
- Organize imports to pass checkstyle

2.Renaming to OpenSearch

Module names: elasticsearch
Class names
- ElasticsearchXXX
- ESXXX
Package names
- ODFE [no changes]
- xxx.sql.elasticsearch
Source code
- Elasticsearch in comments
- ES, Es, es in variable/function name
- ES doc link
ANTLR
- grammar
Config files
- plugin-descriptor.xml
- plugin.config
- elasticsearch.yml
Test data
- Explain JSON file
- Test index name
Documentation
- README
- User manual

3.Testing

Checkstyle
Pass UT
Pass IT
Comparison test (Rerun after JDBC migrated)
Doctest (Rerun after CLI migrated)
GitHub action

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

dblock

See below, sorry to be late to this game.

README.md

build.gradle

Added Tableau Connector to OpenSearch SQL

) * # This is a combination of 9 commits. # This is the 1st commit message: Fix create_index/create_index_with_IOException issue caused by OpenSearch PR change (#1899) * Added setDefaultMediaType for create_index and create_index_with_IOException Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> (cherry picked from commit 7b932a7) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs 2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> cherry pick 60c0018 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Addressed PR comment for table format in AbstractExprValue. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from ExpressionReferenceOptimizer. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> fixed java doc in QualifiedName.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing checkstyle test for core and added spotless for relevant directories. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #2: Fixing spacing around headers in ExpressionReferenceOptimizer.java SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #3: Fix breaking changes. Disable some flaky tests in legacy. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> (cherry picked from commit 809e656) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #4: Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #5: running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #6: Fixed java doc spelling and improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #7: improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #8: Improving failure format on some functions. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> # This is the commit message #9: spotless apply and fix of build.gradle Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * parent 496fe8e author Mitchell Gale <Mitchell.Gale@improving.com> 1691080710 -0700 committer Mitchell Gale <Mitchell.Gale@improving.com> 1691080766 -0700 Fix create_index/create_index_with_IOException issue caused by OpenSearch PR change (#1899) * Added setDefaultMediaType for create_index and create_index_with_IOException Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> (cherry picked from commit 7b932a7) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs 2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> cherry pick 60c0018 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Addressed PR comment for table format in AbstractExprValue. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from ExpressionReferenceOptimizer. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> fixed java doc in QualifiedName.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing checkstyle test for core and added spotless for relevant directories. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixing spacing around headers in ExpressionReferenceOptimizer.java SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fix breaking changes. Disable some flaky tests in legacy. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> (cherry picked from commit 809e656) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixed java doc spelling and improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Improving failure format on some functions. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> spotless apply and fix of build.gradle Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing unused import in OpenSearchRestClientTest.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * running spotless apply. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Addressed comments in PR 5. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com>

) * GJF part 4 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * add build.gradle comment to mention why we are ignoring checkstyle failures for core. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Fix include spotless build gradle. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert astDSL.JAVA Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert ast changes as was covered in spotless #1 PR for GJF. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Reverting commits in ast folder attempt #2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert change to RaretopN.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * addressed PR comments. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Replacing removed include in spotless. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Signed-off-by: Mitchell Gale <Mitchell.gale@improving.com>

…in files #4 #1933 (#1995) * [Spotless] Applying Google Code Format for core/src/main files #4 (#1933) * GJF part 4 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * add build.gradle comment to mention why we are ignoring checkstyle failures for core. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Fix include spotless build gradle. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert astDSL.JAVA Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert ast changes as was covered in spotless #1 PR for GJF. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Reverting commits in ast folder attempt #2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * revert change to RaretopN.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * addressed PR comments. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Replacing removed include in spotless. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Signed-off-by: Mitchell Gale <Mitchell.gale@improving.com> * spotless apply Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * addressed PR comments Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * remove pitest Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Signed-off-by: Mitchell Gale <Mitchell.gale@improving.com>

…in files #1 #1930 (#1992) * [Spotless] Applying Google Code Format for core/src/main files #1 (#1930) * # This is a combination of 9 commits. Fix create_index/create_index_with_IOException issue caused by OpenSearch PR change (#1899) * Added setDefaultMediaType for create_index and create_index_with_IOException Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> (cherry picked from commit 7b932a7) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs 2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> cherry pick 60c0018 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Addressed PR comment for table format in AbstractExprValue. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from ExpressionReferenceOptimizer. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> fixed java doc in QualifiedName.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing checkstyle test for core and added spotless for relevant directories. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixing spacing around headers in ExpressionReferenceOptimizer.java SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fix breaking changes. Disable some flaky tests in legacy. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> (cherry picked from commit 809e656) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixed java doc spelling and improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Improving failure format on some functions. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> spotless apply and fix of build.gradle Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * parent 496fe8e author Mitchell Gale <Mitchell.Gale@improving.com> 1691080710 -0700 committer Mitchell Gale <Mitchell.Gale@improving.com> 1691080766 -0700 Fix create_index/create_index_with_IOException issue caused by OpenSearch PR change (#1899) * Added setDefaultMediaType for create_index and create_index_with_IOException Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> (cherry picked from commit 7b932a7) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs 2 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> cherry pick 60c0018 Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Addressed PR comment for table format in AbstractExprValue. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from ExpressionReferenceOptimizer. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from AstDSL.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removed pre tag from SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> fixed java doc in QualifiedName.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing checkstyle test for core and added spotless for relevant directories. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixing spacing around headers in ExpressionReferenceOptimizer.java SelectExpressionAnalyzer.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fix breaking changes. Disable some flaky tests in legacy. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> (cherry picked from commit 809e656) Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Fixed java doc spelling and improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> improving string concatination. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Improving failure format on some functions. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> spotless apply and fix of build.gradle Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Resolving merge conflicts for pre tag in java docs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> running spotless check on newly pre tagged javadocs. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Converts java doc table to proper java doc table. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Removing unused import in OpenSearchRestClientTest.java Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * running spotless apply. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * Addressed comments in PR 5. Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * spotless apply Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * removed pitest Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * remove pitest Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * add spotless Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> * remove pitest Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> --------- Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com> Signed-off-by: Mitchell Gale <Mitchell.gale@improving.com>

Map opensearch math functions to calcite implementations

…-plan Added support for average metric aggregation

dai-chen added 16 commits April 2, 2021 14:33

Bump plugin version to 1.15.0

5df435f

Migrate gradle and pass all UT

604a13a

Rename elasticsearch module

b047126

Rename all classes

bc1c6c0

Rename elasticsearch in source code and comments

b6c4f1b

Rename elasticsearch in source code and comments

ae6b0db

Rename ES in source code and comments

3d7c377

Rename elasticsearch module package

b5ca5e9

Rename missed elasticsearch and es

f9f1785

Rename ES class

662a537

Change test index names and pass IT

6142de7

Fix broken test UT

27565c3

Change doc and pass doctest

02eb66d

Rename variable and function name

247ca08

Rename es in antlr grammar

dc2ba95

Remove unused es config file

9cef37d

dai-chen self-assigned this Apr 7, 2021

Remove unused file

d4282d0

dai-chen marked this pull request as ready for review April 7, 2021 22:15

Final cleanup

20635a8

dai-chen requested a review from joshuali925 April 8, 2021 17:47

dai-chen merged commit 6340849 into develop Apr 8, 2021

dblock reviewed Apr 8, 2021

View reviewed changes

README.md Show resolved Hide resolved

build.gradle Show resolved Hide resolved

dai-chen mentioned this pull request May 18, 2021

ODBC folder, file and code renaming #62

Merged

6 tasks

dai-chen mentioned this pull request Jun 11, 2021

Aggregation support improvement #124

Open

joshuali925 referenced this pull request in joshuali925/sql Jan 4, 2022

Merge pull request #1 from Bit-Quill/migrate-jdbc-tableau-connector

c8ddfb9

Added Tableau Connector to OpenSearch SQL

forestmvey mentioned this pull request Aug 17, 2022

Support highlighting function in SQL/PPL query engine #636

Closed

6 tasks

dai-chen mentioned this pull request Nov 23, 2022

Improve pushdown optimization and logical to physical transformation #1091

Merged

6 tasks

forestmvey mentioned this pull request Nov 23, 2022

Validate Field and Fields Parameters in Relevance Search Functions #1067

Merged

6 tasks

MitchellGale mentioned this pull request Dec 23, 2022

SQL ingestion and ops - standup #791

Closed

songkant-aws referenced this pull request in songkant-aws/sql Mar 19, 2025

Merge pull request #1 from yaunchun/map2calcite

25d7d51

Map opensearch math functions to calcite implementations

selsong mentioned this pull request Aug 14, 2025

[RFC] Support timechart command in PPL #3965

Closed

expani pushed a commit to expani/sql that referenced this pull request Oct 13, 2025

Merge pull request opensearch-project#1 from expani/feature/substrait…

96ae16e

…-plan Added support for average metric aggregation

coderabbitai bot mentioned this pull request Nov 27, 2025

Remove unnecessary filter for DateHistogram aggregation #4877

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenSearch migration for SQL/PPL #1

OpenSearch migration for SQL/PPL #1

Uh oh!

dai-chen commented Apr 7, 2021 •

edited

Loading

Uh oh!

dblock left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

OpenSearch migration for SQL/PPL #1

OpenSearch migration for SQL/PPL #1

Uh oh!

Conversation

dai-chen commented Apr 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1.Changing to OpenSearch Dependencies

2.Renaming to OpenSearch

3.Testing

Uh oh!

dblock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dai-chen commented Apr 7, 2021 •

edited

Loading