Skip to content

[Backport] FrameFile: Java 17 compatibility#13002

Merged
gianm merged 1 commit intoapache:24.0.0from
gianm:backport-12987-to-24
Aug 31, 2022
Merged

[Backport] FrameFile: Java 17 compatibility#13002
gianm merged 1 commit intoapache:24.0.0from
gianm:backport-12987-to-24

Conversation

@gianm
Copy link
Copy Markdown
Contributor

@gianm gianm commented Aug 30, 2022

Backport #12987 to 24.0.0.

Main reason: it's not just about Java 17 compatibility; this also eliminates the need for special JVM args on Java 11, which will help people that are upgrading.

* FrameFile: Java 17 compatibility.

DataSketches Memory.map is not Java 17 compatible, and from discussions
with the team, is challenging to make compatible with 17 while also
retaining compatibility with 8 and 11. So, in this patch, we switch away
from Memory.map and instead use the builtin JDK mmap functionality. Since
it only supports maps up to Integer.MAX_VALUE, we also implement windowing
in FrameFile, such that we can still handle large files.

Other changes:

1) Add two new "map" functions to FileUtils, which we use in this patch.
2) Add a footer checksum to the FrameFile format. Individual frames
   already have checksums, but the footer was missing one.

* Changes for static analysis.

* wip

* Fixes.
@gianm gianm added the Backport label Aug 30, 2022
@gianm gianm added this to the 24.0.0 milestone Aug 30, 2022
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented Aug 30, 2022

This pull request introduces 1 alert and fixes 1 when merging 21a7f50 into 4fa0d08 - view on LGTM.com

new alerts:

  • 1 for Uncontrolled data used in path expression

fixed alerts:

  • 1 for Uncontrolled data used in path expression

@gianm gianm merged commit 3db8967 into apache:24.0.0 Aug 31, 2022
@gianm gianm deleted the backport-12987-to-24 branch August 31, 2022 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants