idea: healing (slow) backup

[ThomasWaldmann]

On IRC, a anon borg user had an interesting idea:

There could be a combination of backup and integrity check: if the backup run detects a duplicate chunk, it would read the chunk from the repo and check its integrity. If integrity check fails, it would delete the chunk from the repo and store the current data (of same id_hash) into the repo, healing the defect.

Issues with that:

• of course that this is super-slow compared to how borg normally operates as it would read the whole backup volume from the repo (it usually only transfers new data to the repo)
• a simple implementation might check the same chunk multiple times if it appears in input data multiple times.
• when using the files cache, data of unchanged files is usually NOT read from disk, but could be in case of an integrity error in the repo

[dfloyd888]

I remember a commercial backup utility doing something very similar. When you specified an incremental, it added and compressed/deduplicated the incoming data. When you specified a full backup, it would check the hashes against the incoming data and fix chunks that were corrupted, so once done, there really wasn't a difference between the two backup methods, other than time spent.