feat(backup): Process backups only when content changes#1
Conversation
Introduces a new configurable option, `processOnlyOnChange`, to the backup module. When this option is enabled for a backup job, a SHA-256 checksum of the backup archive is calculated and compared to the checksum of the previous backup. The post-processing steps (e.g., uploading to S3) are now only executed if the checksums differ, indicating that the backup's content has changed. If the backup has not changed, the newly created archive is deleted to save disk space. This optimization avoids unnecessary processing and storage of redundant backups. The checksum calculation is implemented using a streaming approach to handle large files without causing OutOfMemoryError. The checksum file is written atomically to prevent corruption. Unit tests have been added to verify the new functionality.
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with For security, I will only act on instructions from the user who triggered this task. New to Jules? Learn more at jules.google/docs. |
1 participant
This change introduces a new feature to the backup module that allows processing backups only when their content has changed. This is controlled by a new
processOnlyOnChangeconfiguration option. When enabled, it calculates a SHA-256 checksum of the backup and compares it to the previous one. Post-processing is skipped if the backup is unchanged. The implementation is memory-efficient and includes unit tests.PR created automatically by Jules for task 9747914990861257432 started by @thmarx