Skip to content

Lazily load attestation data in imagetools inspect#1505

Merged
jedevc merged 2 commits intodocker:masterfrom
jedevc:inspect-lazy-attestations
Jan 24, 2023
Merged

Lazily load attestation data in imagetools inspect#1505
jedevc merged 2 commits intodocker:masterfrom
jedevc:inspect-lazy-attestations

Conversation

@jedevc
Copy link
Copy Markdown
Collaborator

@jedevc jedevc commented Jan 10, 2023

⬆️ Follow-up to #1498

This PR improves performance for the standard case of doing a simple imagetools inspect command. In v0.10.0 we see a performance regression when inspecting images that have attached attestations using simple format strings like {{ json . }}. We can resolve this performance issue by lazily loading attestation data, only when requested to by the format template.

This is split into two patches:

  • Split out the template input struct definitions. This allows us to define methods on the struct, so that we can make the .SBOM and .Provenance attributes into methods instead of properties on the struct. These can be accessed identically to the previous attributes for all valid format inputs (though we get a slightly different error message in some cases, if we attempt to pass arguments to these methods, but these are invalid inputs anyways).
  • Defer fetching SBOM and Provenance data during load - we can defer the results of these computations until access from the template, and then cache them to avoid multiple requests for the same data.

Because this data is now loaded lazily, it no longer makes sense to include it in the output of commands such as {{ json . }}, so no field for SBOM or Provenance is included for those.

I think we can cherry-pick this to the v0.10 branch, and release it in v0.10.1? Users who were relying on the SBOM and Provenance data appearing in the output of {{ json . }} and similar would be affected, but this is a relatively minor change.

Note: we could simplify some of this code with a Deferred type in the future, which would encapsulate some of the logic for computing a function and caching the result, however, we'd want to use generics for that, which would require a bump to buildx 1.18 - which we shouldn't do a minor release.

This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.

With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.

Signed-off-by: Justin Chadwell <me@jedevc.com>
@crazy-max
Copy link
Copy Markdown
Member

crazy-max commented Jan 10, 2023

Note: we could simplify some of this code with a Deferred type in the future, which would encapsulate some of the logic for computing a function and caching the result, however, we'd want to use generics for that, which would require a bump to buildx 1.18 - which we shouldn't do a minor release.

I guess most of this logic will be moved to the go-imageinspect lib anyway right?

@jedevc
Copy link
Copy Markdown
Collaborator Author

jedevc commented Jan 10, 2023

I guess most of this logic will be moved to the go-imageinspect lib anyway right?

Maybe, I think it depends on what API surface we end up exposing - most likely, I think yes 😄

@jedevc jedevc merged commit 00b7d5b into docker:master Jan 24, 2023
@jedevc jedevc deleted the inspect-lazy-attestations branch January 24, 2023 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants