Support pre-built function runtimes and per-language schema generation#24
Open
negz wants to merge 3 commits into
Open
Support pre-built function runtimes and per-language schema generation#24negz wants to merge 3 commits into
negz wants to merge 3 commits into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Crossplane projects today discover embedded functions by convention: every subdirectory of paths.functions is treated as a function, and the CLI auto-detects the language and builds the runtime image. This works well for simple projects but blocks projects that have outgrown the built-in builders or that need to coordinate function builds with an existing build system (make, nix, Bazel, CI pipelines). Per crossplane#21, users want to supply pre-built OCI runtime images alongside source-based functions, so the CLI handles packaging while the user owns the build. This commit adds an optional functions list to ProjectSpec. When the list is present it disables auto-discovery and is the sole source of truth for which functions to build. Each entry uses a Source discriminator (Directory or Tarball) and a corresponding sub-field: spec: architectures: [amd64, arm64] functions: - source: Directory directory: name: function-a - source: Tarball tarball: name: function-b pathPrefix: build/function-b Directory-source functions follow the existing build path. Tarball- source functions skip language detection and load one pre-built single-platform OCI image tarball per target architecture, following the naming convention `<pathPrefix>-<arch>.tar`. So the example above loads `build/function-b-amd64.tar` and `build/function-b-arm64.tar`. Per-architecture tarballs match what build tools naturally produce without bundling: `docker save`, Nix's dockerTools.buildImage, Bazel's oci_tarball, `ko build --tarball`, etc. all emit one single-platform tarball at a time. Packaging is inherently per- architecture too — each runtime image gets its own crossplane.yaml layer before they're tied together into a multi-arch package index — so the CLI would have to split a multi-arch input apart anyway. The CLI verifies that each tarball's image config records the architecture its filename promises, and adds the package metadata layer (crossplane.yaml) before assembling the multi-arch package index. The on-disk output is identical to a CLI-built function. When the functions list is omitted, the existing auto-discovery behaviour is preserved unchanged. Fixes crossplane#21. Signed-off-by: Nic Cope <nicc@rk0n.org>
Nix's dockerTools.buildImage produces gzipped tarballs by default. Some other build tools (Bazel rules_oci's oci_load, certain ko invocations) do the same. With only plain .tar accepted, users of these tools had to add a decompress step to their build pipeline just to feed images to the Crossplane CLI. This commit teaches the function tarball loader to fall back to `<pathPrefix>-<arch>.tar.gz` when `<pathPrefix>-<arch>.tar` is not present, preferring the plain tar when both exist. The gzipped tarball is streamed through gzip.NewReader into go-containerregistry's tarball.Image; no temporary files are written. Signed-off-by: Nic Cope <nicc@rk0n.org>
By default crossplane project build and crossplane dependency update-cache generate schemas for all four supported languages (Go, JSON, KCL, Python). Per crossplane#29 this is wasteful for projects that only consume some of them: every build generates language bindings the project never imports. This commit adds an optional schemas block to ProjectSpec: spec: schemas: languages: [python] When languages is set, schema generation is restricted to the listed languages. The filter applies both to the project's own XRD schemas and to its declared dependencies, and flows through project build/run and dependency update-cache/clean-cache. When schemas is omitted (the default), all languages are generated as before. The schemas block is nested rather than flat to leave room for future schema-related knobs (output paths, generator-specific options) without scattering schema config across ProjectSpec. The supported language identifiers are defined as constants (SchemaLanguageGo, SchemaLanguageJSON, SchemaLanguageKCL, SchemaLanguagePython) in the API package, with SupportedSchemaLanguages returning the canonical set. The schema generator package consumes these constants directly so the two cannot drift, and a test in the generator package asserts that AllLanguages covers exactly the API's declared set. Fixes crossplane#29. Signed-off-by: Nic Cope <nicc@rk0n.org>
negz
changed the title
📝 WalkthroughWalkthroughThis PR enables Crossplane projects to declare embedded functions using either directory sources (built by the CLI) or pre-built tarball images (provided by users), while also adding per-language schema generation configuration to skip unnecessary schema artifacts. ChangesFunction sources and schema language configuration
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
internal/project/build.go (1)
607-642: 💤 Low valueMinor: Consider returning both errors from
gzipReadCloser.Close().Currently, if
g.Reader.Close()succeeds butg.file.Close()fails, we return the file error. But if both fail, we only return the gzip error and lose the file error. This is probably fine in practice since file close errors are rare, but I wanted to mention it.Would it be worth using
errors.Jointo return both errors when they both occur? That said, if you've considered this and decided the current behavior is sufficient, I'm happy to defer to your judgment.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/project/build.go` around lines 607 - 642, The Close method on gzipReadCloser currently returns only the first non-nil error (g.Reader.Close) or the file error (g.file.Close), losing the other error if both fail; change gzipReadCloser.Close to combine both errors when present (use errors.Join(gerr, ferr) or equivalent) so callers receive both failures, keeping existing return behavior when only one error exists; update the gzipReadCloser.Close implementation to import and use errors.Join while preserving the current call ordering and semantics used in gzipOpener and gzipReadCloser.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apis/dev/v1alpha1/project_types.go`:
- Around line 272-286: The Function.Name method should be nil-safe like
GetLanguages: add a nil-receiver guard at the start of Function.Name (check if f
== nil and return an empty string) so callers can safely call
(*Function)(nil).Name() without panicking; update the Function.Name method to
return "" immediately when f is nil and keep the existing switch logic
unchanged.
In `@apis/dev/v1alpha1/validate.go`:
- Around line 121-122: Update the user-facing validation messages that are
currently appended to errs for unsupported schema languages and invalid function
names: replace technical phrasing like "is not a supported schema language" with
clear, actionable text that states what the user tried to provide, what valid
options are, and exactly what to change and retry (e.g., "The schema language
'X' is not supported. Please choose one of [A,B,C] and update schemas.languages
to one of these values, then retry."). Apply the same style to the other related
error appends (the ones that reference schema languages, the variable supported,
lang, and the checks for invalid function names) so each error suggests the
corrective action and shows valid examples or accepted patterns.
- Around line 241-247: The validation error wrapping uses fmt.Errorf("...: %w",
err) for the Directory and Tarball cases; replace those with
crossplane-runtime's errors.Wrap to match project conventions: import
"github.com/crossplane/crossplane-runtime/pkg/errors" (or add to existing
imports) and change the two append lines to errs = append(errs, errors.Wrap(err,
"directory")) for f.Directory.Validate() and errs = append(errs,
errors.Wrap(err, "tarball")) for f.Tarball.Validate(), leaving the rest of the
switch logic unchanged.
---
Nitpick comments:
In `@internal/project/build.go`:
- Around line 607-642: The Close method on gzipReadCloser currently returns only
the first non-nil error (g.Reader.Close) or the file error (g.file.Close),
losing the other error if both fail; change gzipReadCloser.Close to combine both
errors when present (use errors.Join(gerr, ferr) or equivalent) so callers
receive both failures, keeping existing return behavior when only one error
exists; update the gzipReadCloser.Close implementation to import and use
errors.Join while preserving the current call ordering and semantics used in
gzipOpener and gzipReadCloser.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 6a7b1fe0-a0d4-41ce-9bbf-203928089248
⛔ Files ignored due to path filters (1)
apis/dev/v1alpha1/zz_generated.deepcopy.gois excluded by!**/zz_generated*.goand included by**/*.go
📒 Files selected for processing (15)
apis/dev/v1alpha1/project_types.goapis/dev/v1alpha1/validate.goapis/dev/v1alpha1/validate_test.gocmd/crossplane/dependency/cache.gocmd/crossplane/project/build.gocmd/crossplane/project/run.gointernal/project/build.gointernal/project/build_test.gointernal/schemas/generator/go.gointernal/schemas/generator/interface.gointernal/schemas/generator/interface_test.gointernal/schemas/generator/json.gointernal/schemas/generator/kcl.gointernal/schemas/generator/python.gointernal/schemas/manager/manager.go
Comment on lines
+272
to
+286
| func (f *Function) Name() string { | ||
| switch f.Source { | ||
| case FunctionSourceDirectory: | ||
| if f.Directory == nil { | ||
| return "" | ||
| } | ||
| return f.Directory.Name | ||
| case FunctionSourceTarball: | ||
| if f.Tarball == nil { | ||
| return "" | ||
| } | ||
| return f.Tarball.Name | ||
| } | ||
| return "" | ||
| } |
There was a problem hiding this comment.
Guard Name() against a nil receiver for safer API usage
Nice addition overall—could we make this method nil-safe as well (like GetLanguages) so external callers don’t panic on (*Function)(nil).Name()?
Suggested patch
func (f *Function) Name() string {
+ if f == nil {
+ return ""
+ }
switch f.Source {
case FunctionSourceDirectory:
if f.Directory == nil {
return ""📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func (f *Function) Name() string { | |
| switch f.Source { | |
| case FunctionSourceDirectory: | |
| if f.Directory == nil { | |
| return "" | |
| } | |
| return f.Directory.Name | |
| case FunctionSourceTarball: | |
| if f.Tarball == nil { | |
| return "" | |
| } | |
| return f.Tarball.Name | |
| } | |
| return "" | |
| } | |
| func (f *Function) Name() string { | |
| if f == nil { | |
| return "" | |
| } | |
| switch f.Source { | |
| case FunctionSourceDirectory: | |
| if f.Directory == nil { | |
| return "" | |
| } | |
| return f.Directory.Name | |
| case FunctionSourceTarball: | |
| if f.Tarball == nil { | |
| return "" | |
| } | |
| return f.Tarball.Name | |
| } | |
| return "" | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apis/dev/v1alpha1/project_types.go` around lines 272 - 286, The Function.Name
method should be nil-safe like GetLanguages: add a nil-receiver guard at the
start of Function.Name (check if f == nil and return an empty string) so callers
can safely call (*Function)(nil).Name() without panicking; update the
Function.Name method to return "" immediately when f is nil and keep the
existing switch logic unchanged.
Comment on lines
+121
to
+122
| errs = append(errs, errors.Errorf("schemas.languages[%d]: %q is not a supported schema language, must be one of %v", i, lang, supported)) | ||
| } |
There was a problem hiding this comment.
Make new validation errors more user-actionable
Great coverage expansion. Could we rephrase these messages to be less technical and include a clear next step (e.g., what to change and retry), especially around unsupported languages and invalid function names?
Example wording direction
- schemas.languages[%d]: %q is not a supported schema language, must be one of %v
+ schemas.languages[%d]: %q is not supported. Choose one of %v and run the command again.
- name %q is not a valid DNS-1123 subdomain: %v
+ name %q is invalid for a function name. Use lowercase letters, numbers, '-' or '.', then try again.As per coding guidelines: "CRITICAL: Ensure all error messages are meaningful to end users, not just developers - avoid technical jargon, include context about what the user was trying to do, and suggest next steps when possible."
Also applies to: 269-270, 287-293
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apis/dev/v1alpha1/validate.go` around lines 121 - 122, Update the user-facing
validation messages that are currently appended to errs for unsupported schema
languages and invalid function names: replace technical phrasing like "is not a
supported schema language" with clear, actionable text that states what the user
tried to provide, what valid options are, and exactly what to change and retry
(e.g., "The schema language 'X' is not supported. Please choose one of [A,B,C]
and update schemas.languages to one of these values, then retry."). Apply the
same style to the other related error appends (the ones that reference schema
languages, the variable supported, lang, and the checks for invalid function
names) so each error suggests the corrective action and shows valid examples or
accepted patterns.
Comment on lines
+241
to
+247
| if err := f.Directory.Validate(); err != nil { | ||
| errs = append(errs, fmt.Errorf("directory: %w", err)) | ||
| } | ||
| case FunctionSourceTarball: | ||
| if err := f.Tarball.Validate(); err != nil { | ||
| errs = append(errs, fmt.Errorf("tarball: %w", err)) | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify wrapper style in validation files.
rg -nP --type=go 'fmt\.Errorf\(".*: %w"' apis/dev/v1alpha1Repository: crossplane/cli
Length of output: 580
Switch validation error wrapping to crossplane-runtime/pkg/errors.Wrap
apis/dev/v1alpha1/validate.go currently wraps directory/tarball validation errors with fmt.Errorf("...: %w", err). For consistency with Crossplane’s wrapping patterns (this same fmt.Errorf(...: %w, err) approach is used for other sources in this file too), consider switching to errors.Wrap.
Suggested patch
case FunctionSourceDirectory:
if err := f.Directory.Validate(); err != nil {
- errs = append(errs, fmt.Errorf("directory: %w", err))
+ errs = append(errs, errors.Wrap(err, "directory"))
}
case FunctionSourceTarball:
if err := f.Tarball.Validate(); err != nil {
- errs = append(errs, fmt.Errorf("tarball: %w", err))
+ errs = append(errs, errors.Wrap(err, "tarball"))
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apis/dev/v1alpha1/validate.go` around lines 241 - 247, The validation error
wrapping uses fmt.Errorf("...: %w", err) for the Directory and Tarball cases;
replace those with crossplane-runtime's errors.Wrap to match project
conventions: import "github.com/crossplane/crossplane-runtime/pkg/errors" (or
add to existing imports) and change the two append lines to errs = append(errs,
errors.Wrap(err, "directory")) for f.Directory.Validate() and errs =
append(errs, errors.Wrap(err, "tarball")) for f.Tarball.Validate(), leaving the
rest of the switch logic unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
This PR bundles two small improvements I wanted while taking the new Crossplane CLI for a test drive on a complex project. Each is in its own commit.
1. Pre-built function runtime images (fixes #21)
Crossplane projects today discover embedded functions by convention: every subdirectory of
paths.functionsis treated as a function, and the CLI auto-detects the language and builds the runtime image. This works well for simple projects but blocks projects that have outgrown the built-in builders or that need to coordinate function builds with an existing build system (make, nix, Bazel, CI pipelines).This PR adds an optional
functionslist toProjectSpec. When the list is present it disables auto-discovery and is the sole source of truth for which functions to build. Each entry uses asourcediscriminator (DirectoryorTarball):Directory-source functions follow the existing build path. Tarball-source functions skip language detection and load one pre-built single-platform OCI image tarball per target architecture, following the naming convention
<pathPrefix>-<arch>.taror<pathPrefix>-<arch>.tar.gz(preferring the plain.tarwhen both exist). Per-architecture tarballs match what build tools naturally produce without bundling:docker save, Nix'sdockerTools.buildImage, Bazel'soci_tarball,ko build --tarball, etc. all emit one single-platform tarball at a time. Packaging is inherently per-architecture too — each runtime image gets its owncrossplane.yamllayer before they're tied together into a multi-arch package index — so the CLI would have to split a multi-arch input apart anyway. The gzipped variant is split into a separate commit; it's needed because Nix's image builders emit gzipped tarballs by default.2. Per-language schema generation (fixes #29)
By default
crossplane project buildandcrossplane dependency update-cachegenerate schemas for all four supported languages (Go, JSON, KCL, Python). For a project that only consumes one of them, every build generates language bindings the project never imports.This commit adds an optional
schemasblock toProjectSpec:When
languagesis set, schema generation is restricted to the listed languages, both for the project's own XRDs and for its declared dependencies. The filter flows throughproject build/runanddependency update-cache/clean-cache. The block is nested rather than flat to leave room for future schema-related knobs.Reviewers may want to focus on
loadTarballRuntimeandloadRuntimeImageininternal/project/build.go(the new tarball loading path) and onProjectSchemas.Validateinapis/dev/v1alpha1/validate.gotogether withgenerator.Filterininternal/schemas/generator/interface.go(the schema language filter). The language identifiers are now defined as constants in the API package and consumed by the generators directly, so the two can't drift.I have:
./nix.sh flake checkto ensure this PR is ready for review.Linked a PR or a docs tracking issue to document this change.Addedbackport release-x.ylabels to auto-backport this PR.