hashindex: flags / set operations

[ThomasWaldmann]

i am thinking about adding another 32 reserved bits field to the hashindex entries, see #6689.

it is a pain to change the entry layout (usually that would need an upgrade process), so maybe just have some bits reserved for misc purposes, e.g.:

• flags:
  • todo/done flag for bookkeeping of long running repo operations (like chunk conversions, compression, crypto, chunk checks)
• maybe remove the in-band signalling EMPTY/DELETED the hashtable does in the first uint32 value to mark empty buckets, tombstones, etc. - guess that would slightly simplify the code and make it more general purpose. this could be done just using some reserved bits of the flags field for hashtable internal purposes.
• other ideas?

for the flags there could be a nice set-based api:

• query all / N members of a set (return chunkids of entries that have a specific flag set)
• add member(s) to a set (set a specific flag)
• remove member(s) from a set (reset a specific flag)

[ThomasWaldmann]

In borg2 branch, there is:

• Repository.flags to query or set flags in the extra 32bit repoindex entry
• Repository.iteritems extended to only yield chunkids with flags & mask == value.

[ThomasWaldmann]

done (and merged to master recently). not used yet though.