speed up xattrs / acls / bsdflags back up

[ThomasWaldmann]

this is summarizing what ideas came up regarding to this:

• PR xattr: move to platform pkg, use cython, use bytes, use fd #3918, ACLs fd-based #3960, make bsdflags linux code fd-based #3961 seem to speed up processing a bit

• if we would put xattrs / acls / bsdflags into the files cache (we already have the chunks list there), we would not need to read them from the filesystem IF we can reliably detect changes. is comparing ctime and inode_number enough for this? mtime and size are obviously not helpful with that.

• if we put that stuff into the files cache, we might produce a resource usage issue for some systems that have a lot of xattrs/acls, but not a lot of memory. so there should be an option to not put it into the files cache and fallback to reading it from disk in that case.

• there could be another option not to read and back up xattrs / acls at all. we already have that for bsdflags. if there are technical issues (see freebsd 12 FUSE xattr issues #3952) or you're sure there are no acls/xattrs or you don't need them for restore (but you have a megaton of small files), this likely is useful even if there is faster xattr/acl code.

• keep in mind that xattrs might be big, see extend item data structure to better deal with large xattr-like things? #1681 - we shouldn't store big stuff into files cache.

[ThomasWaldmann]

A minor speedup was achieved by #3918, #3960, #3961 (about 5% for total time, for a 2nd backup of lots of small files (/usr/share) using a cold cache).

[FabioPedretti]

Thanks for the improvements!
Are these and following related commits (syscall reduction) too intrusive for backporting to 1.1 branch?

[ThomasWaldmann]

@FabioPedretti The overall speed improvement isn't big, but the change is rather big, so I guess it is better not to backport, but test it a while in 1.2 alpha, beta, rc testing before it enters a production version.

[ThomasWaldmann]

How big the bsdflags overhead is has to be re-evaluated after the PR #4043 is merged (we open the file there rather early anyway to work with the FD).

[ThomasWaldmann]

#4043 is merged now into master, so someone could do benchmarks on many small files, with and without --nobsdflags and see if there is any significant difference now.

There is no --noacls or --noxattrs (yet?), but commenting the stuff out in the code should be relatively easy (search for stat_attrs).