[Bug] The change to file meta classes causes compatibility problem

[yuzelin]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

master (0.9-SNAPSHOT which contains #3363 and #3368 )

Compute Engine

flink

Minimal reproduce step

Stop a flink job which uses paimon-0.8, then restart with checkpoint using paimon-0.9-SNAPSHOT.

What doesn't meet your expectations?

The error is:

java.lang.UnsupportedOperationException: Unsupported byte value '100' for value kind.
at org.apache.paimon.manifest.FileSource.fromByteValue(FileSource.java:47)
at org.apache.paimon.io.DataFileMetaSerializer.fromRow(DataFileMetaSerializer.java:81)
at org.apache.paimon.io.DataFileMetaSerializer.fromRow(DataFileMetaSerializer.java:34)
at org.apache.paimon.utils.ObjectSerializer.deserialize(ObjectSerializer.java:81)
at org.apache.paimon.utils.ObjectSerializer.deserializeList(ObjectSerializer.java:104)
at org.apache.paimon.table.sink.CommitMessageSerializer.deserialize(CommitMessageSerializer.java:121)
at org.apache.paimon.table.sink.CommitMessageSerializer.deserializeList(CommitMessageSerializer.java:99)
at org.apache.paimon.manifest.ManifestCommittableSerializer.deserialize(ManifestCommittableSerializer.java:94)
at org.apache.paimon.manifest.ManifestCommittableSerializer.deserialize(ManifestCommittableSerializer.java:34)
at org.apache.paimon.flink.VersionedSerializerWrapper.deserialize(VersionedSerializerWrapper.java:48)
at org.apache.flink.core.io.SimpleVersionedSerialization.readVersionAndDeSerialize(SimpleVersionedSerialization.java:227)
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState$DeserializingIterator.next(SimpleVersionedListState.java:138)
at java.base/java.lang.Iterable.forEach(Iterable.java:74)
at org.apache.paimon.flink.sink.RestoreAndFailCommittableStateManager.initializeState(RestoreAndFailCommittableStateManager.java:75)
at org.apache.paimon.flink.sink.CommitterOperator.initializeState(CommitterOperator.java:131)
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:122)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:275)
at org.apache.paimon.flink.sink.AutoTagForSavepointCommitterOperator.initializeState(AutoTagForSavepointCommitterOperator.java:112)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:851)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:798)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:746)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:959)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:928)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:751)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:567)
at java.base/java.lang.Thread.run(Thread.java:991)

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[yuzelin]

#3368