'borg diff' does not report changed mtime

[xrbd]

Have you checked borgbackup docs, FAQ, and open Github issues?

YES

Is this a BUG / ISSUE report or a QUESTION?

1 BUG
1 ENHANCEMENT-REQUEST

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.2.1

Operating system (distribution) and version.

Debian bullseye x64

Hardware / network configuration, and filesystems used.

AMD x64 processor, 8GB RAM,
1Gbps ethernet
3 x 8TB drives layered as LUKS, LVM, BTRFS RAID1

How much data is handled by borg?

Repo1: 22 TB (un-deduplicated), 2TB (deduplicated), 105 archives in repo (most files are hardlinked together).

Full borg commandline that lead to the problem (leave away excludes and passwords)

See bash script below.

Describe the problem you're observing.

1. BUG: 'borg diff' does not report changed mtime for files within a repository, even though 'borg list --json-lines' shows that the file metadata have changed. (Other metadata than mtime may have same problem...haven't checked).
2. ENHANCEMENT REQUEST: 'borg list --json-lines' sometimes renders attributes (eg size, mtime) in different order; this causes a 'diff' of the directory entry against another instance to to mis-match, even though the attribute values are identical. Proposed change: please render attributes in a stable order (alphabetic sort order is suggested, but what is important is that it be the same order across all archives within a single repository).

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Yes. See script which follows:

#!/bin/bash

# This script demonstrates that 'borg diff' (version 1.2.1, at least) does
# not report a difference in mtime of a file.
# This issue may also apply to atime and ctime as well, and also to
# directories, and to other metadata attributes (such as permissions),
# but isn't tested by this script.

dir=$(mktemp -d)
pushd "$dir" > /dev/null

borg init -e none repo

mkdir work
echo "hello" > work/hello
borg create repo::archive01 work
borg list --json-lines repo::archive01 > list1
sleep 4      
touch work/hello
borg create repo::archive02 work
borg list --json-lines repo::archive02 > list2

# Enhancement request: please emit attributes (eg, mtime, size) in same
# order each time, so that a diff of unchanged elements (eg, a directory
# entry) will match:
# Note if 'sleep 4' above is change to 'sleep 2', the 'borg list --json-lines'
# attributes are rendered in same order.  Why?
diff list1 list2

# Bug: borg diff does not report the change in mtime, even though it is
# visible in 'borg list'
borg diff repo::archive01 archive02

popd

[Michael-Girma]

@ThomasWaldmann can I pick this up?

[ThomasWaldmann]

@Michael-Girma sure, all open and unassigned stuff can be taken. thanks!

[Michael-Girma]

@ThomasWaldmann After making my changes and seeing my test pass, I realized that the previous diff test cases will need to be modified. Seeing how this results in an output that includes files that only have metadata changes (due to 'mtime', 'ctime'), should we make this feature accessible through a flag?

[ThomasWaldmann]

Yes, maybe there could be an option --content-only that makes it only compare the file contents and ignore all metadata changes. Guess the default should be to compare everything (metadata and data).