Excluding / doesn't work with --patterns-from

[edgimar]

In issue #1406 the patterns file format which was chosen should allow something like the following to work, if I'm not mistaken:

+ /home/foo/bar/**
+ /etc/**
- /**

This should backup the contents of /home/foo/bar and /etc, and nothing else on the filesystem. Presently using a patterns file like this results in no files getting selected at all. Regardless of whether it make sense to have a patterns file like this (some might argue that two roots should be specified on the command-line, but there are use-cases when it is preferable to have just a single root of /), the above should still work.

For the record, I am using borg (1.1.0b4.dev223+ng10d4c97c) from the command-line as follows:

borg-backup create --exclude-if-present .nobackup --keep-exclude-tags --patterns-from patt_file backupserver:/home/backupuser/backups/borgrepo::myarchive /

Backing up without the --patterns-from option works fine.

[edgimar]

One other note: I realize now that I can to some extent work around this problem by just specifying roots within the patterns file instead of on the command-line. But I still do consider the reported behavior (no files getting backed up) to be a bug.

[edgimar]

Oops, I take that back -- it really can't be worked around in a reasonable way using roots -- there is a bug that prevents standard use-cases. For example, say I have a directory tree (beneath a home directory) that looks like:

mydir/
mydir/x
mydir/x/a
mydir/x/b
mydir/y

Now let's say I want to backup everything under mydir, except that within the x subdirectory, I only want to back up a, and no other folders within x. So, I attempt to get this behavior with the following pattern file:

R /home/user/mydir
+ /home/user/mydir/x/a/**
- /home/user/mydir/x/**

Unfortunately with this patterns file, and using the command-line below, mydir/x does not get backed up at all. If it were working correctly, mydir/x/a should get backed up (and nothing else besides that within mydir/x).

Command line used:

borg-backup create --exclude-if-present .nobackup --keep-exclude-tags --patterns-from patt_file backupserver:/home/backupuser/backups/borgrepo::myarchive

(this is the same as in the original issue, except that it doesn't specify a root on the commandline)

[ThomasWaldmann]

I guess for a sane handling of this, we need to push the decision about "recursion or not" to the user.

So it could be something like:

+ include and recurse into
- exclude, but recurse into, searching for includes
! exclude, do not recurse into

[enkore]

That sounds about right. Maybe swap -/! meanings, as --exclude (~ -) should not be changed to "exclude, but recurse into". And ! = danger zone -- or in this case "slow zone".

[ThomasWaldmann]

Well, I guess some users will just use + and - and expect that it'll work correct under all circumstances. if - does not recurse, it won't.

But also, being inconsistent with --exclude isn't nice....

[edgimar]

If it matters, using + and - with recursion is also consistent with rdiff-backup's --exclude-filelist syntax.

[enkore]

Can we get some votes in here please?

Thumbs up here for

+ include and recurse into
- exclude, but recurse into, searching for includes
! exclude, do not recurse into

(- slow, but respects + in a directory if the directory was excluded)

[enkore]

And thumbs up here for

+ include and recurse into
- exclude, do not recurse into
! exclude, but recurse into, searching for includes

(- fast, but does not respect + in a directory if the directory was excluded)

[enkore]

Since this is then just a matter of swapping the two characters later if necessary, this can already be implemented.