Docs: document the compareWithFileList parameter

[bitsondatadev]

We should document the awesome compareWithFileList parameter that lets you use inventory reports for remove_orphan_files in the docs. I had to find it after googling and stumbling on PR #4503.

https://apache-iceberg.slack.com/archives/C03LG1D563F/p1690322588417159

[fernandocast]

Hello @bitsondatadev, could you please assign me this issue?
I would like to contribute on this project

[bitsondatadev]

Done!

[Tavisca-vinayak-bhadage]

This compareWithFileList would be good solution for AWS S3 based iceberg tables also. As we are facing below exception with default remove orphan file implementation :

org.apache.iceberg.exceptions.RuntimeIOException: java.io.IOException: com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: Please reduce your request rate. (Service: Amazon S3; Status Code: 503; Error Code: SlowDown;)

Is there any way to use compareWithFileList parameter from pyspark or spark-sql. As we are having AWS Glue as spark engine which is not supporting java Data Set api.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'