Skip to content

Sourcepolicy#3332

Merged
tonistiigi merged 15 commits intomoby:masterfrom
cpuguy83:sourcepolicy
Dec 13, 2022
Merged

Sourcepolicy#3332
tonistiigi merged 15 commits intomoby:masterfrom
cpuguy83:sourcepolicy

Conversation

@cpuguy83
Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 commented Nov 28, 2022

Add source policy in LLB as described in #2943 (comment)

Replaces #2943
Closes #2943

@cpuguy83 cpuguy83 force-pushed the sourcepolicy branch 3 times, most recently from e8f8a03 to 4e308ac Compare November 28, 2022 23:20
Comment thread docs/build-repro.md Outdated
@tonistiigi tonistiigi added this to the v0.11.0 milestone Nov 29, 2022
@cpuguy83 cpuguy83 force-pushed the sourcepolicy branch 6 times, most recently from 1ad024c to 43e8ffb Compare November 29, 2022 22:32
@cpuguy83 cpuguy83 marked this pull request as ready for review November 29, 2022 22:49
@cpuguy83
Copy link
Copy Markdown
Member Author

Marking this ready, looks like it's gonna be all 💚

Comment thread sourcepolicy/matcher.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/engine_test.go
Comment thread sourcepolicy/pb/policy.proto
Comment thread solver/llbsolver/solver.go Outdated
}

func (s *Solver) Solve(ctx context.Context, id string, sessionID string, req frontend.SolveRequest, exp ExporterRequest, ent []entitlements.Entitlement, post []Processor) (_ *client.SolveResponse, err error) {
func (s *Solver) Solve(ctx context.Context, id string, sessionID string, req frontend.SolveRequest, exp ExporterRequest, ent []entitlements.Entitlement, post []Processor, srcPol *sourcepolicypb.Policy) (_ *client.SolveResponse, err error) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SolverResponse should contain a (boolean?) value that indicates whether the policy was consumed or not

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consumed in what way? Policy applied to sources?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a boolean here is not very useful except for just 1-off testing cases.
Probably it would be best to look at the build metadata to see what sources were actually used... maybe we could add something to trace policy usage in that metadata, but this will require a bit more effort.

Comment thread docs/build-repro.md Outdated
Comment thread solver/llbsolver/vertex.go Outdated
@cpuguy83 cpuguy83 force-pushed the sourcepolicy branch 3 times, most recently from f491e5f to 9502353 Compare December 2, 2022 19:04
@cpuguy83
Copy link
Copy Markdown
Member Author

cpuguy83 commented Dec 2, 2022

This is updated with suggestions.

@cpuguy83 cpuguy83 force-pushed the sourcepolicy branch 2 times, most recently from 32a6668 to 2bce7d0 Compare December 2, 2022 20:02
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/pb/policy.proto Outdated
Comment thread sourcepolicy/pb/policy.proto Outdated
Comment thread sourcepolicy/matcher.go Outdated
Comment thread sourcepolicy/matcher.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread docs/build-repro.md Outdated
Comment thread docs/build-repro.md Outdated
Comment thread hack/dockerfiles/generated-files.Dockerfile Outdated
Comment thread solver/llbsolver/vertex.go Outdated
Comment thread sourcepolicy/engine.go Outdated
Comment thread sourcepolicy/engine.go Outdated
@cpuguy83 cpuguy83 force-pushed the sourcepolicy branch 2 times, most recently from 87725f9 to 385f4f0 Compare December 13, 2022 20:14
Comment thread sourcepolicy/engine.go Outdated
AkihiroSuda and others added 15 commits December 13, 2022 13:01
Alternative to PR 2816 ("dockerfile: support Dockerfile.pin for pinning sources")

This version is implemented on the llbsolver side and agnostic to the LLB frontends.
See `solver/llbsolver/vertex.go:loadLLB()`.

See `docs/build-repro.md` for the usage.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This defines a better type for wildcard matching as well as a more
appropritate package (contributed by Akihiro) for doing wildcard
matching.

Also some improvements to caching state such as compiled regexes.

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This makes destination more symetrical with sources.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Allows frontends to pass in a list of policies in the solve request.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
These were just there while figuring out how things would work and are
not needed now.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Per our discussion on github, each policy should be evaluated on it's
own.
ie. an "allow" in one policy should be able to change to a "deny" in
another policy.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This makes it so a deny is always a deny regardless of if it was allowed
in a prior rule.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This turns the closure into a proper recursive funciton.
It also ensures there isn't a chance of infinite recursion due to
changes to op inputs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

fix
Updates the field names for source policies to be a little less
confusing: `Source` -> `Selector`, `Destination` -> `Updates`.

Also collapse `Type` into the `Identifier` which matches how the rest of
buildkit works and greatly simplifies matching.. and generally makes it
so the policy engine doesn't need to care about schemes.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants