Improve snapshot rollback failure when snapshots produced too frequent.

[puchengy]

Hi all, I am seeing the error below.

java.lang.IllegalArgumentException: [BUG] Expected sorted snapshot log entries.
	at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
	at org.apache.iceberg.TableMetadata.<init>(TableMetadata.java:306)
	at org.apache.iceberg.TableMetadata$Builder.build(TableMetadata.java:1040)

Does it mean we shouldn’t write the new snapshot too frequent (within a minute)? I wonder should an improvement be added to the writer side to prevent such error happen? https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/TableMetadata.java#L310

The check is introduced from #1110 #1109

[wypoon]

If you are getting that error, it means that your table's history contains snapshots with timestamps that are out of order. Iceberg has built in a large tolerance for what is "out of order", a one-minute tolerance. That means if you have timestamps that are out of order by sub-second or even seconds, Iceberg will not complain. (I think it is the opposite of what you were thinking, that snapshots occurring very close to each other will be problematic. Those will not cause this error even if the timestamps are out of order by sub-minute.)

There is a bug introduced in 0.13.0, where if you write a sequence of snapshots more than a minute apart, and then rollback to one of the earlier snapshots, an incorrect timestamp is written for the rollback snapshot, resulting in this error (because the rollback snapshot is now out of order by more than a minute). This bug is fixed by #4135 in the 0.13.x branch, and if we release a 0.13.2, the fix will be there. In master, it is fixed by #4089.

[jpugliesi]

@wypoon thank you for the explanation - is there any known workaround for this?

[wypoon]

@jpugliesi there is no workaround for this. However, I believe a 0.13.2 release is planned soon, which will have the fix for this. If you need the fix sooner, you can build Iceberg from the 0.13.x branch yourself.

[wypoon]

I suppose if you do need to rollback to a snapshot (and the rollback fails because of this bug), you can create a new table by selecting from the original table as of the snapshot you want, and switch to using the new table.

[puchengy]

I can confirm the fix is working for 0.13.x branch. So I am closing this diff. cc: @SinghAsDev