Skip to content

How to move Iceberg table from one location to another #11645

@NotaSimpO

Description

@NotaSimpO

I'm working with Apache Iceberg on my local setup, using tools like Apache Spark, Nessie, and MinIO. I need to move Parquet data files from one location (standardize/location1) to another (standardize/location2). I've updated the manifest file to reflect the new data paths, and the table is partially working in the new location.

However, I'm facing a problem with Iceberg recognizing delete files. Specifically, the delete files (which track deleted file paths and the positions of deleted data) also contain the original file paths. Even after updating these paths to the new location, Iceberg still doesn't recognize the delete files correctly, and the table's behavior remains inconsistent.

How can I handle this issue effectively? Is there a way to move the table and its associated files (including delete files) properly so Iceberg recognizes them? I would prefer a solution in PySpark or SQL..

Thank

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions