borg diff should have --format option

[elho]

The diff sub-command should have a --format option like e.g. the list sub-command has.

Placeholders should include (but not be limited to), path, type (directory entry type, same as with list), change (giving the type of change, ie. whether added, deleted, modified (content change), owner, group, mode, etc. - speaking long or like AME, whatever is deemed preferable) and whatever fields to represent what else diff has at its hands that it can output, e.g. oldsize, newsize, deltasize, etc.
If several aspects of a directory entry were changed, multiple lines, each with the according change value would be printed. This does not sound nice at first, bit fits the format-string style output (as opposed to more template style with conditionals etc.) and suits working on the output with shell tools like grep.

This would allow to do stuff like borg diff --format "{change}{type}:{path}{NUL}" | sed --null-data --quiet --expression='s/(A|M)-://p' | xargs --null dosomethingwithchangedfiles, thus being able to workaround/solve e.g. #3514 and #4191.

[palao]

I'd like to implement this. Is it ok?

[elho]

I'd like to implement this. Is it ok?

Great, go ahead. 👍

Just note that it is not as trivial as it sounds, the current output is not generated by the diff function but deeper in the code where the actual changes between items are detected.

So based on what I remember from back then, when I did briefly look into the code to see whether I could just fix it quickly instead of opening the bug, I would imagine it to involve these steps:

1. Redo that internal part to return/communicate changes using some defined proper data structure (e.g. a nested dict of the attributes changed, each with old and new maybe?) instead of strings.
2. Make the existing diff function build the current output based on that data structure gathered from the internal diffing process.
3. Make any other part of the code that eventually uses those strings to use the new data structure.
4. Extent the existing diff function generate --format output based on those very same data structures

Further additions, such as some sort of JSON output would then be quite trivial.

[palao]

One question: what are the expected list of possible placeholders?

[diivi]

I would like to work on this. Please assign this issue to me, I'll ask for help when needed on the #borgbackup IRC channel.

[ThomasWaldmann]

@diivi you have it, thanks for helping!

[diivi]

Hey @elho, the current ItemDiff methods _something_diff always return a 2-tuple: infos_in_a_dict, infos_formatted_str do you want me to get rid of infos_formatted_str and just use the dict to format outputs?

[diivi]

Also, regarding the placeholders -
I'll have to return the same dict structure from every _diff_* function right? So that I have the right keys to access when I create my own DiffFormatter class later. This means when the file's owner was changed, I will still return all other keys- dict(mode=(st1.mode,st2.mode),user=(st1.user,st.user)...) even though st1.user and st2.user are the same.
Another option-
Just show a message saying key did not change?

[RF-Tar-Railt]

I'd like to get a try for solution of this

[ThomasWaldmann]

@RF-Tar-Railt maybe @diivi is still working on this? @diivi are you?

[diivi]

@RF-Tar-Railt maybe @diivi is still working on this? @diivi are you?

Nope, I couldn't start working on this as it required some more design discussions.
@RF-Tar-Railt can work on it if they want to.