Skip to content

Flink: Adds support for 1.18 version#9211

Merged
stevenzwu merged 4 commits intoapache:mainfrom
rodmeneses:adds_118_support
Dec 7, 2023
Merged

Flink: Adds support for 1.18 version#9211
stevenzwu merged 4 commits intoapache:mainfrom
rodmeneses:adds_118_support

Conversation

@rodmeneses
Copy link
Copy Markdown
Contributor

No description provided.

@Fokko Fokko requested a review from stevenzwu December 4, 2023 18:29
@stevenzwu stevenzwu changed the title Adds support for 1.18 version Flink: Adds support for 1.18 version Dec 5, 2023
ArrayData actualArrayData;
try {
actualArrayData = (ArrayData) actual;
actualArrayData = convertToArray(actual);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a behavior change from 1.18? it seems like a breaking change to me.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only on 1.18 and it only affects the unit tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you only updated the unit test. but it suggests that Flink RowData field has changed from ArrayData to the raw type array (like int[]). I am trying to understand if it has any impact on compatibility.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you only updated the unit test. but it suggests that Flink RowData field has changed from ArrayData to the raw type array (like int[]). I am trying to understand if it has any impact on compatibility.

Thanks! cc: @pvary

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on @PrabhuJoseph's comment the changes are caused by https://issues.apache.org/jira/browse/FLINK-33523.
Based on the jira, this is an expected general fix in Flink 1.18.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the clarify from the discussion thread that @pvary started, I am good with following up on this one separately when Flink 1.18.1 is released

@stevenzwu stevenzwu merged commit 22b95dc into apache:main Dec 7, 2023
@stevenzwu
Copy link
Copy Markdown
Contributor

thanks @rodmeneses for the contribution and @pvary for following up with the Flink community

@pvary
Copy link
Copy Markdown
Contributor

pvary commented Dec 13, 2023

@rodmeneses: Could you please update the docs in a follow-up PR too?

Here are the supported Flink versions: https://iceberg.apache.org/multi-engine-support/#apache-flink
And please check and if you find, fix at other places as well.
Thank you very much, Peter

@andrea-zanetti
Copy link
Copy Markdown

andrea-zanetti commented Feb 27, 2024

Hi, just to let you know that trying to download the runtime jar for the 1.18 version of Flink from https://iceberg.apache.org/multi-engine-support/#apache-flink returns a 404.

EDIT: Wrong link.

@fmorillo7694
Copy link
Copy Markdown

@andrea-zanetti Could you share the proper link? i cant find the Jar Runtime for 1.18

@andrea-zanetti
Copy link
Copy Markdown

Hi @FranMorilloAWS,

I have the same exact problem: cannot find it on maven central, got here via google search but the download link is broken.

@rodmeneses
Copy link
Copy Markdown
Contributor Author

Hi @FranMorilloAWS,

I have the same exact problem: cannot find it on maven central, got here via google search but the download link is broken.
@andrea-zanetti
1.18 version will be available when Iceberg 1.5 is released

@pvary
Copy link
Copy Markdown
Contributor

pvary commented Feb 28, 2024

@andrea-zanetti, @FranMorilloAWS: Iceberg 1.5.0 will be the first version which supports Flink 1.18.x. It will be released soon, as the release process is already started

@andrea-zanetti
Copy link
Copy Markdown

Hi,

Thanks for you answer. I'll wait for the release of the new version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants