Skip to content

import/export-tar: --tar-format=BORG: roundtrip ALL item metadata#6523

Merged
ThomasWaldmann merged 1 commit intoborgbackup:masterfrom
ThomasWaldmann:pax-borg-item-master
Apr 9, 2022
Merged

import/export-tar: --tar-format=BORG: roundtrip ALL item metadata#6523
ThomasWaldmann merged 1 commit intoborgbackup:masterfrom
ThomasWaldmann:pax-borg-item-master

Conversation

@ThomasWaldmann
Copy link
Copy Markdown
Member

fixes #5830.

export-tar: just msgpack and b64encode all item metadata and put that into a BORG specific PAX header. this is additional to the standard tar metadata.

import-tar: when detecting the BORG specific PAX header, just get all metadata from there (and ignore the standard tar metadata).

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 2, 2022

Codecov Report

Merging #6523 (e8069a8) into master (22fc6d1) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #6523   +/-   ##
=======================================
  Coverage   82.99%   83.00%           
=======================================
  Files          39       39           
  Lines       10531    10543   +12     
  Branches     2061     2063    +2     
=======================================
+ Hits         8740     8751   +11     
- Misses       1292     1294    +2     
+ Partials      499      498    -1     
Impacted Files Coverage Δ
src/borg/archive.py 81.77% <100.00%> (+0.06%) ⬆️
src/borg/archiver.py 78.47% <100.00%> (+0.05%) ⬆️
src/borg/repository.py 84.00% <0.00%> (-0.19%) ⬇️
src/borg/helpers/parseformat.py 90.19% <0.00%> (+0.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22fc6d1...e8069a8. Read the comment docs.

 borgbackup#5830

export-tar: just msgpack and b64encode all item metadata and
            put that into a BORG specific PAX header.
            this is *additional* to the standard tar metadata.

import-tar: when detecting the BORG specific PAX header, just get
            all metadata from there (and ignore the standard tar
            metadata).
@ThomasWaldmann
Copy link
Copy Markdown
Member Author

if noone objects or has review feedback, will merge this soon.

@ThomasWaldmann ThomasWaldmann merged commit 28fa9e0 into borgbackup:master Apr 9, 2022
@ThomasWaldmann ThomasWaldmann deleted the pax-borg-item-master branch April 9, 2022 18:22
@callegar
Copy link
Copy Markdown

Thanks for this addition! I see that this item is in the docs for 1.3 alpha. Is it planned to land in the 1.2 series as well?

@ThomasWaldmann
Copy link
Copy Markdown
Member Author

Well, the idea is that this is the enabler for crossing even a breaking release. For that to work, it needs to be available on both old and new side.

The code from this PR basically already solves that, but a remaining problem here is efficiency: just imagine you have 100 archives referencing 1TB data each. That would mean it would read 100TB from source repo and pump 100TB over the tar pipe and that would take quite a while. See my current PR which tries to optimize this.

@callegar
Copy link
Copy Markdown

callegar commented Apr 28, 2022

moved to #6643

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tar import/export: add --format option, pax-borg metadata

3 participants