Skip to content

Support SST Decoder #284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
marsishandsome merged 2 commits into from
Oct 14, 2021
Merged

Support SST Decoder #284

marsishandsome merged 2 commits into from
Oct 14, 2021

Conversation

@marsishandsome
Copy link
Collaborator

@marsishandsome marsishandsome commented Oct 9, 2021 •
edited
Loading

Signed-off-by: marsishandsome marsishandsome@gmail.com

What problem does this PR solve?

close #285

support decode SST to Key-Value

subtask of tikv/tikv#11007

What is changed and how it works?

RocksDB provides a java API SstFileReader to decode SST to Key-Value pairs. We can wrapper this API in client-java.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to be included in the release note

@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch 6 times, most recently from 4bcdc15 to 6cb7646 Compare October 11, 2021 02:50
@marsishandsome
Support SST Decoder
e0c00f6 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from 6cb7646 to e0c00f6 Compare October 11, 2021 02:57
@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 11, 2021

/run-all-tests

@marsishandsome marsishandsome mentioned this pull request Oct 11, 2021
Merged
Little-Wallace
Little-Wallace approved these changes Oct 13, 2021
View reviewed changes
Copy link
Contributor

@Little-Wallace Little-Wallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andylokandy
Copy link
Contributor

andylokandy commented Oct 13, 2021 •
edited
Loading

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 14, 2021 •
edited
Loading

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

Cool. Can we do this job as a subtask of API V2?
We can update to API to pass the SST Metadata file and leave the implementation empty.

@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from d0dd3c8 to a6f13ea Compare October 14, 2021 08:44
@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome
add backupmeta decoder
5869fb8 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from a6f13ea to 5869fb8 Compare October 14, 2021 09:01
@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome marsishandsome merged commit bbe4f9b into tikv:master Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andylokandy andylokandy andylokandy approved these changes

@Little-Wallace Little-Wallace Little-Wallace approved these changes

@ti-srebot ti-srebot ti-srebot approved these changes

Assignees

No one assigned

Labels

status/LGT2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

support decode SST to Key-Value

4 participants

@marsishandsome @andylokandy @Little-Wallace @ti-srebot