Skip to content

CVE Fix: Update json-path version#15772

Merged
xvrl merged 13 commits intoapache:masterfrom
pagrawal10:pagrawal/updateJsonPath
Feb 15, 2024
Merged

CVE Fix: Update json-path version#15772
xvrl merged 13 commits intoapache:masterfrom
pagrawal10:pagrawal/updateJsonPath

Conversation

@pagrawal10
Copy link
Copy Markdown
Contributor

@pagrawal10 pagrawal10 commented Jan 29, 2024 •
edited
Loading

Description

Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.
This PR modifies pom file, license file, OrcReaderTest and OWASP Dependency Check suppression file.
Append function has been added to json-path and so the unit test to check for the append function not present has been removed.

Release Note:

JSON-Path upgrade now allows append functionality. However, we do not support json-path append function for ORC format (see #11722). Earlier, if the user tried to use JsonPath function append earlier, it would have not appended the item, but now the same function will return an exception. We are overriding the parse function used when append is invoked. So, row.getDimension("append") results in java.lang.UnsupportedOperationException: Unused coming from

druid/processing/src/main/java/org/apache/druid/java/util/common/parsers/FlattenerJsonProvider.java

This is a potential issue to watch out for when upgrading.


This PR has:
  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@pagrawal10 pagrawal10 force-pushed the pagrawal/updateJsonPath branch from 3ad578d to 7334c7f Compare January 30, 2024 08:19
xvrl
xvrl reviewed Jan 30, 2024
View reviewed changes
Comment thread extensions-core/orc-extensions/src/test/java/org/apache/druid/data/input/orc/OrcReaderTest.java Outdated
Comment on lines +317 to +318
//Support for append has been added in json-path-2.9.0
//Assert.assertEquals(Collections.emptyList(), row.getDimension("append"));
Copy link
Copy Markdown
Member

@xvrl xvrl Jan 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if append is now supported, then let's update the test to check that it does what it needs to do.

Copy link
Copy Markdown
Contributor Author

@pagrawal10 pagrawal10 Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

row.getDimension("append") results in java.lang.UnsupportedOperationException: Unused coming from

druid/processing/src/main/java/org/apache/druid/java/util/common/parsers/FlattenerJsonProvider.java

Line 118 in 0089f6b

throw new UnsupportedOperationException("Unused");
as we have overriden the parse function.

Copy link
Copy Markdown
Contributor Author

@pagrawal10 pagrawal10 Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Therefore, while the upstream has added the capability of append , we are not using it.

Copy link
Copy Markdown
Member

@xvrl xvrl Feb 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understood, but there are two issues here:

  1. as a principle we don't leave commented code unless there's a very good reason.
  2. This is potentially being breaking change. Any json-path expression using append today would have silently been ignored, but would now cause an error.

My suggestion would be to:

  1. update the test to check that we do get the exception
  2. make a note in the PR description that we should call this out in the release notes as a potential issue to watch out for when upgrading.

Copy link
Copy Markdown
Contributor Author

@pagrawal10 pagrawal10 Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

xvrl
xvrl reviewed Jan 30, 2024
View reviewed changes
Comment thread owasp-dependency-check-suppressions.xml Outdated
@pagrawal10 pagrawal10 requested a review from xvrl February 1, 2024 03:09
pagrawal10 and others added 2 commits February 9, 2024 09:17
@pagrawal10 @xvrl
Update extensions-core/orc-extensions/src/test/java/org/apache/druid/…
cf89b4f 
…data/input/orc/OrcReaderTest.java

Co-authored-by: Xavier Léauté <xl+github@xvrl.net>
@pagrawal10 @xvrl
Update extensions-core/orc-extensions/src/test/java/org/apache/druid/…
b5ce300 
…data/input/orc/OrcReaderTest.java

Co-authored-by: Xavier Léauté <xl+github@xvrl.net>
@pagrawal10 pagrawal10 requested a review from xvrl February 9, 2024 04:27
@pagrawal10 pagrawal10 mentioned this pull request Feb 12, 2024
Closed
@xvrl
Copy link
Copy Markdown
Member

xvrl commented Feb 15, 2024

I tried rerunning the flaky test. Didn’t seem to help. Since the failure doesn’t appear to related I’m ok merging this.

@xvrl xvrl merged commit 495e66f into apache:master Feb 15, 2024
pagrawal10 added a commit to confluentinc/druid that referenced this pull request Feb 15, 2024
@pagrawal10 @xvrl
CVE Fix: Update json-path version (apache#15772)
c9b61ad 
Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.

Append function has been added to json-path and so the unit test to check for the append function not present has been updated.

---------

Co-authored-by: Xavier Léauté <xvrl@apache.org>
pagrawal10 added a commit to confluentinc/druid that referenced this pull request Feb 15, 2024
@pagrawal10 @xvrl
CVE Fix: Update json-path version (apache#15772)
278a344 
Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.

Append function has been added to json-path and so the unit test to check for the append function not present has been updated.

---------

Co-authored-by: Xavier Léauté <xvrl@apache.org>
pagrawal10 added a commit to confluentinc/druid that referenced this pull request Feb 15, 2024
@pagrawal10 @xvrl
OBSDATA-3639 CVE Fix: Update json-path version (apache#15772) (#186)
da76b8d 
Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.

Append function has been added to json-path and so the unit test to check for the append function not present has been updated.

---------

Co-authored-by: Xavier Léauté <xvrl@apache.org>
sreemanamala pushed a commit to sreemanamala/druid that referenced this pull request Feb 20, 2024
@pagrawal10 @xvrl @sreemanamala
CVE Fix: Update json-path version (apache#15772)
4fdffec 
Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.

Append function has been added to json-path and so the unit test to check for the append function not present has been updated.

---------

Co-authored-by: Xavier Léauté <xvrl@apache.org>
pagrawal10 added a commit to confluentinc/druid that referenced this pull request Mar 8, 2024
@pagrawal10 @KeerthanaSrikanth @xvrl
Update Memcache SSLContext Protocol to TLSv1.2 (#190)
3746924 
* Upgrade pac4j-oidc to 4.5.7 to address CVE-2021-44878 (apache#15522)

* Upgrade org.pac4j:pac4j-oidc to 4.5.5 to address CVE-2021-44878
* add CVE suppression and notes, since vulnerability scan still shows this CVE
* Add tests to improve coverage

* CVE Fix: Update json-path version (apache#15772)

Apache Druid brings the dependency json-path which is affected by CVE-2023-51074.
Its latest version 2.9.0 fixes the above CVE.

Append function has been added to json-path and so the unit test to check for the append function not present has been updated.

---------

Co-authored-by: Xavier Léauté <xvrl@apache.org>

* Update protocol for MemcachedCache (apache#16035)

---------

Co-authored-by: Keerthana Srikanth <ksrikanth@confluent.io>
Co-authored-by: Xavier Léauté <xvrl@apache.org>
@adarshsanjeev adarshsanjeev added this to the 30.0.0 milestone May 6, 2024
@adarshsanjeev adarshsanjeev mentioned this pull request May 28, 2024
Closed
@nozjkoitop nozjkoitop mentioned this pull request Sep 20, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xvrl xvrl xvrl approved these changes

Assignees

No one assigned

Labels

Area - Batch Ingestion Area - Dependencies Area - Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying Area - Segment Format and Ser/De Release Notes

Projects

None yet

Milestone

30.0.0

Development

Successfully merging this pull request may close these issues.

4 participants

@pagrawal10 @xvrl @vogievetsky @adarshsanjeev