Skip to content

Conversation

@nk1506
Copy link
Contributor

@nk1506 nk1506 commented Oct 3, 2023

added missing metadata tables all_delete_files all_entries entries position_deletes

Fixes #757

@ajantha-bhat
Copy link
Member

@nk1506: I think it is hard to review. Can you please keep the scope of this PR to add the missing metadata tables? Rearranging can be done in a followup PR.

```

| status | snapshot_id | sequence_number | file_sequence_number | data_file | readable_metrics |
|--------| -- |-----------------|----------------------| -- | -- |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this style is not same as existing tables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment for all new tables added.


### Entries

To show all the table's current manifest entries for both data and delete files.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this statement is confusing. "all the table + current" ?


### Positional Delete Files

To show all positional delete files from a table:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only from current snapshot? If it is not referenced by current snapshot, it won't show right? I think we need to clarify


#### All Delete Files

To show all the table's delete files and each file's metadata:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment for "all the table's"

do we need each file's metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

followed the same convention of previous tables like files

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. The existing convention is still confusing for me to read. But we can optimize in a follow up. Ok for me to keep it similar now.


#### All Entries

To show all the table's manifest entries from any reachable snapshot for both data and delete files:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from any reachable snapshot -> from all the snapshots

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also " all the table's" is confusing here too.

@ajantha-bhat ajantha-bhat requested a review from szehon-ho October 4, 2023 15:02
@nk1506
Copy link
Contributor Author

nk1506 commented Oct 5, 2023

@szehon-ho , Please review and share the feedback.

Copy link
Member

@ajantha-bhat ajantha-bhat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nastra nastra merged commit d8a07ff into apache:master Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document all metadata tables

3 participants