Don't unpack again if unchanged

[everettraven]

Creating this issue to continue discussion from #30 (comment)

Currently the CatalogSource controller will attempt to unpack the catalog contents and recreate the metadata on cluster every reconciliation, even if it has already been unpacked previously. This causes some unnecessary overhead by running the unpacking logic every single reconciliation loop. Instead, we should explore ways to signal that the unpacking process has already been completed and that the unpacking logic does not need to be run.

Some potential solutions:

• Update the CatalogSource status to also communicate the observed generation and/or the hash of the catalog contents (maybe image sha if we can easily fetch that info?). If there is a change in generation / catalog contents then unpack again
  • Downside: We can't determine the hash of the catalog contents without running the unpack job again so that would still be a process that is run every time (although maybe we gate it with a generation change?)
• Add a label/annotation to signal it has been unpacked and doesn't need to run the unpack process again

[anik120]

@everettraven this is probably stale at this point right?

[everettraven]

@anik120 I think this is still relevant. This is getting at having a check when the call to Unpack() has returned that we don't attempt to process/store the returned contents unless they have actually changed

[everettraven]

I believe this was implemented as part of the direct image registry client work. Closing this issue, but anyone should feel free to re-open this if it is closed insufficiently