Add Radius.Core/terraformConfigs and bicepConfigs resources#11780
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #11780 +/- ##
==========================================
+ Coverage 51.47% 51.69% +0.22%
==========================================
Files 716 725 +9
Lines 45155 45595 +440
==========================================
+ Hits 23242 23572 +330
- Misses 19705 19798 +93
- Partials 2208 2225 +17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
a99b027 to
371a010
Compare
There was a problem hiding this comment.
Pull request overview
Adds new Radius.Core/terraformConfigs and Radius.Core/bicepConfigs resources, wires them into Radius.Core/environments, and threads their data into recipe execution and generated API artifacts.
Changes:
- Adds new TypeSpec/OpenAPI/resource-provider models, clients, converters, and manifests for
terraformConfigsandbicepConfigs. - Extends
Radius.Core/environmentsand the config loader/terraform executor to resolve and consume the new config resources. - Adds functional/unit tests and a new architecture document describing the feature.
Reviewed changes
Copilot reviewed 43 out of 46 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
typespec/Radius.Core/terraformConfigs.tsp |
Defines new Terraform config resource schema. |
typespec/Radius.Core/main.tsp |
Imports new Radius.Core resource specs. |
typespec/Radius.Core/environments.tsp |
Adds environment references to shared config resources. |
typespec/Radius.Core/bicepConfigs.tsp |
Defines new Bicep config resource schema. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/tfbicep-combined-test.bicep |
Combined functional scenario for both config resources. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/terraformconfig-redis-test.bicep |
Functional Terraform config scenario. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/bicepconfig-test.bicep |
Functional Bicep config scenario. |
test/functional-portable/dynamicrp/noncloud/resources/terraformconfig_bicepconfig_test.go |
Functional tests for new resources and combined flow. |
swagger/specification/radius/resource-manager/Radius.Core/preview/2025-08-01-preview/openapi.json |
Regenerated OpenAPI for new resources and environment fields. |
pkg/rp/util/config.go |
Adds fetch helpers for new config resources. |
pkg/recipes/terraform/execute.go |
Wires Terraform CLI config generation into execution paths. |
pkg/recipes/terraform/execute_test.go |
Adds tests around Terraform CLI config/env handling. |
pkg/recipes/terraform/cliconfig.go |
Renders .terraformrc from provider installation settings. |
pkg/recipes/terraform/cliconfig_test.go |
Tests .terraformrc rendering/writing. |
pkg/recipes/configloader/environment.go |
Resolves environment config references into recipe config. |
pkg/recipes/configloader/environment_test.go |
Updates loader tests for new function signature. |
pkg/corerp/setup/setup.go |
Registers new Radius.Core resources. |
pkg/corerp/setup/setup_test.go |
Extends setup routing tests for new resources. |
pkg/corerp/setup/operations.go |
Adds RBAC operation entries for new resources. |
pkg/corerp/frontend/controller/environments/v20250801preview/createorupdateenvironment.go |
Validates referenced config IDs during environment PUT/PATCH. |
pkg/corerp/datamodel/terraformconfig.go |
Adds Terraform config datamodel. |
pkg/corerp/datamodel/recipe_types.go |
Extends shared recipe config with provider installation support. |
pkg/corerp/datamodel/environment_v20250801preview.go |
Adds new environment reference fields. |
pkg/corerp/datamodel/converter/terraformconfig_converter.go |
Adds Terraform config versioned/datamodel converter. |
pkg/corerp/datamodel/converter/bicepconfig_converter.go |
Adds Bicep config versioned/datamodel converter. |
pkg/corerp/datamodel/bicepconfig.go |
Adds Bicep config datamodel. |
pkg/corerp/api/v20250801preview/zz_generated_terraformconfigs_client.go |
Generated Terraform config client. |
pkg/corerp/api/v20250801preview/zz_generated_responses.go |
Generated response types for new clients. |
pkg/corerp/api/v20250801preview/zz_generated_options.go |
Generated option types for new clients. |
pkg/corerp/api/v20250801preview/zz_generated_models.go |
Generated API models for new resources and env fields. |
pkg/corerp/api/v20250801preview/zz_generated_models_serde.go |
Generated serde for new models. |
pkg/corerp/api/v20250801preview/zz_generated_constants.go |
Generated enum/constants for Bicep auth methods. |
pkg/corerp/api/v20250801preview/zz_generated_client_factory.go |
Exposes new generated clients from client factory. |
pkg/corerp/api/v20250801preview/zz_generated_bicepconfigs_client.go |
Generated Bicep config client. |
pkg/corerp/api/v20250801preview/terraformconfig_conversion.go |
Converts Terraform config API models to/from datamodel. |
pkg/corerp/api/v20250801preview/fake/zz_generated_terraformconfigs_server.go |
Generated fake Terraform config server. |
pkg/corerp/api/v20250801preview/fake/zz_generated_server_factory.go |
Wires new fake servers into factory. |
pkg/corerp/api/v20250801preview/fake/zz_generated_bicepconfigs_server.go |
Generated fake Bicep config server. |
pkg/corerp/api/v20250801preview/environment_conversion.go |
Converts new environment reference fields. |
pkg/corerp/api/v20250801preview/bicepconfig_conversion.go |
Converts Bicep config API models to/from datamodel. |
hack/bicep-types-radius/generated/radius/radius.core/2025-08-01-preview/types.json |
Regenerated Bicep type metadata. |
hack/bicep-types-radius/generated/index.json |
Adds new Bicep type index entries. |
docs/architecture/terraform-bicep-config.md |
New architecture doc for the feature. |
deploy/manifest/built-in-providers/self-hosted/radius_core.yaml |
Registers new resource types in self-hosted manifest. |
deploy/manifest/built-in-providers/dev/radius_core.yaml |
Registers new resource types in dev manifest. |
.devcontainer/devcontainer-lock.json |
Formatting-only lockfile change. |
Resolves several review comments on PR #11780. Schema changes: - BicepConfigProperties.registryAuthentication (single value, no host) is replaced with registryAuthentications: Record<...> keyed by registry hostname. The Bicep driver looks up credentials by the host parsed from the recipe template path; the previous shape gave the user no way to declare which registry the credentials applied to and the loader was hard-coding 'default' as the key, so credentials were silently dropped. - TerraformrcConfig.credentials documentation clarifies it is for HTTP Terraform CLI registry auth (rendered as native credentials "host" {} blocks with a 'token' key), not Git module-source PAT auth. Wiring changes: - Terraform driver now renders credentials "host" { token = ... } blocks in the generated .terraformrc, resolving the token value from the fetched-secret map at execution time. Drops the broken bridge that forwarded credentials into the legacy Git PAT path (which expects the 'pat' key, not 'token'). - terraform driver FindSecretIDs reports the new credentials secret stores with the 'token' key so the engine fetches them before the driver runs. - Loader uses the new host-keyed Bicep map to populate the legacy Authentication map (BasicAuth only; AzureWI/AwsIrsa stay schema-only follow-ups). Validation: - Environment controller's referenced-config-id checks now also verify the resource type matches the expected Radius.Core/{terraformConfigs, bicepConfigs}, instead of accepting any resource ID that happens to exist (Copilot review #1, #2). - New bicepconfigs.ValidateRequest hook (UpdateFilter) enforces that authenticationMethod=BasicAuth requires basicAuthSecretId, AzureWI requires both Wi ids, and AwsIrsa requires the role ARN. TypeSpec cannot express these conditional-required-field rules without a discriminated- union restructure (Copilot review #6). Tests: - New TestWriteTerraformCLIConfig_Credentials exercises native credentials rendering: success, deterministic ordering, missing secret reference, missing secret data, missing 'token' key, and embedded-quote escaping. - TestWriteTerraformCLIConfig_BothBlocks asserts provider_installation and credentials co-render in one .terraformrc. - New bicepconfigs.TestValidateRequest covers the controller-level conditional validation matrix. - bicepconfig-test.bicep and tfbicep-combined-test.bicep updated to the new schema shape with a real SecretStore reference (was previously a no-op BasicAuth without a secret, which would now be rejected). Docs: - docs/architecture/terraform-bicep-config.md updated for the new schema, the credentials/token rendering, and the architecture README index now links the page (Copilot review #5).
Resolves several review comments on PR #11780. Schema changes: - BicepConfigProperties.registryAuthentication (single value, no host) is replaced with registryAuthentications: Record<...> keyed by registry hostname. The Bicep driver looks up credentials by the host parsed from the recipe template path; the previous shape gave the user no way to declare which registry the credentials applied to and the loader was hard-coding 'default' as the key, so credentials were silently dropped. - TerraformrcConfig.credentials documentation clarifies it is for HTTP Terraform CLI registry auth (rendered as native credentials "host" {} blocks with a 'token' key), not Git module-source PAT auth. Wiring changes: - Terraform driver now renders credentials "host" { token = ... } blocks in the generated .terraformrc, resolving the token value from the fetched-secret map at execution time. Drops the broken bridge that forwarded credentials into the legacy Git PAT path (which expects the 'pat' key, not 'token'). - terraform driver FindSecretIDs reports the new credentials secret stores with the 'token' key so the engine fetches them before the driver runs. - Loader uses the new host-keyed Bicep map to populate the legacy Authentication map (BasicAuth only; AzureWI/AwsIrsa stay schema-only follow-ups). Validation: - Environment controller's referenced-config-id checks now also verify the resource type matches the expected Radius.Core/{terraformConfigs, bicepConfigs}, instead of accepting any resource ID that happens to exist (Copilot review #1, #2). - New bicepconfigs.ValidateRequest hook (UpdateFilter) enforces that authenticationMethod=BasicAuth requires basicAuthSecretId, AzureWI requires both Wi ids, and AwsIrsa requires the role ARN. TypeSpec cannot express these conditional-required-field rules without a discriminated- union restructure (Copilot review #6). Tests: - New TestWriteTerraformCLIConfig_Credentials exercises native credentials rendering: success, deterministic ordering, missing secret reference, missing secret data, missing 'token' key, and embedded-quote escaping. - TestWriteTerraformCLIConfig_BothBlocks asserts provider_installation and credentials co-render in one .terraformrc. - New bicepconfigs.TestValidateRequest covers the controller-level conditional validation matrix. - bicepconfig-test.bicep and tfbicep-combined-test.bicep updated to the new schema shape with a real SecretStore reference (was previously a no-op BasicAuth without a secret, which would now be rejected). Docs: - docs/architecture/terraform-bicep-config.md updated for the new schema, the credentials/token rendering, and the architecture README index now links the page (Copilot review #5).
23ab261 to
27b6f97
Compare
|
Filed #11856 for the Delete env/secrets asymmetry — leaving it out of this PR to keep the scope focused. |
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
Define TerraformConfigResource and BicepConfigResource in Radius.Core TypeSpec namespace. Add terraformConfig/bicepConfig reference fields to EnvironmentProperties. Regenerate OpenAPI spec and Go SDK (v20250801preview). Add design doc at docs/architecture/terraform-bicep-config.md.
Add TerraformConfig and BicepConfig datamodel structs reusing existing recipe_types.go sub-types. Add bidirectional API-to-datamodel converters for v20250801preview. Add datamodel/converter bridge files for version routing. Update environment converter to handle terraformConfig/bicepConfig reference fields.
Add resource registrations in SetupRadiusCoreNamespace using default sync CRUD controllers. Add route test entries for Put and Patch operations.
Add FetchTerraformConfig and FetchBicepConfig utilities in pkg/rp/util/. Update getConfigurationV20250801 in the recipe config loader to resolve referenced config resources and populate RecipeConfig. Add PUT-time validation in the environment controller to verify referenced config resource IDs exist.
Test_TerraformConfig_Redis: deploys a Terraform recipe (Redis on K8s) via a Radius.Core/terraformConfigs resource referenced by a Radius.Core/environments resource. Validates env var injection via the new config path. Test_BicepConfig_CRUD: validates CRUD and environment reference wiring for Radius.Core/bicepConfigs without requiring a private registry. Both tests follow existing dynamicrp test patterns alongside recipepacks_test.go.
- TerraformConfig: replace authentication/providers/envSecrets with terraformrc (providerInstallation: networkMirror/direct, credentials) + plain env map. Per spec, provider secrets and envSecrets are handled by recipe parameters, not config. - BicepConfig: replace inline authentication map with structured registryAuthentication (authenticationMethod enum: BasicAuth/AzureWI/ AwsIrsa, plus identity fields and basicAuthSecretId). - Loader bridges new shape into the legacy RecipeConfig consumed by the shared driver: terraformrc.credentials -> Authentication.Git.PAT, env -> EnvironmentVariables, registryAuthentication.basicAuthSecretId -> Bicep.Authentication["default"]. - Updates architecture doc and bicepconfig functional test fixture.
Add ProviderInstallation field to the shared TerraformConfigProperties datamodel and wire it through to the Terraform driver. The driver writes a .terraformrc file in the working directory and points Terraform at it via TF_CLI_CONFIG_FILE during both Deploy and Delete (so terraform init can resolve providers from a network mirror on destroy). The new field is optional and only populated by the Radius.Core path. Applications.Core leaves it nil, so the driver behavior is unchanged for the legacy code path consumed by existing recipeConfig users. Tests: - pkg/recipes/terraform/cliconfig_test.go: HCL rendering, file mode, edge cases, error handling. - TestSetEnvironmentVariables: new case for providerInstallation. - TestApplyTerraformCLIConfig: Delete-path helper. - Fixes pre-existing test compile failure in environment_test.go where getConfigurationV20250801 had a stale call signature.
Test_TerraformConfig_BicepConfig_Combined deploys a single Radius.Core/environments referencing both a terraformConfig (with terraformrc.providerInstallation and env vars) and a bicepConfig, then runs a Terraform recipe end-to-end. This exercises the new .terraformrc rendering path without requiring a network mirror in the test cluster by using providerInstallation.direct (default registry behavior).
- Remove the Implementation plan section (superseded by actual code). - Replace the aspirational Delete protection section with a Status and known limitations section that lists what is wired vs. follow-ups. - Expand the Runtime flow to call out the .terraformrc rendering path added in pkg/recipes/terraform/cliconfig.go. - Add a Code map pointing at the relevant files.
TerraformCredentialConfig and SecretConfig both have a single Secret string field; staticcheck (S1016) prefers a type conversion over a struct literal.
- Regenerate Bicep type definitions for radius.core 2025-08-01-preview to pick up the terraformConfigs/bicepConfigs schema refactor. - gofmt fix on terraformconfig_conversion.go indentation. Generated with mockgen v0.4.0 (matches CI version pinned in .github/workflows/lint.yaml).
…on failure (EOF on 127.0.0.1:45533) affecting 16 unrelated tests
Functional tests were failing with 'resource type not found' because the new types were defined in TypeSpec/codegen and wired into the corerp REST router, but never registered in the UCP manifest files that get baked into the ucpd image at build time. Adds the resource types to both the self-hosted (production) and dev manifests, plus the corresponding RBAC operation entries in pkg/corerp/setup/operations.go (mirroring the recipePacks pattern). Fixes Test_TerraformConfig_Redis, Test_BicepConfig_CRUD, and Test_TerraformConfig_BicepConfig_Combined.
Resolves several review comments on PR #11780. Schema changes: - BicepConfigProperties.registryAuthentication (single value, no host) is replaced with registryAuthentications: Record<...> keyed by registry hostname. The Bicep driver looks up credentials by the host parsed from the recipe template path; the previous shape gave the user no way to declare which registry the credentials applied to and the loader was hard-coding 'default' as the key, so credentials were silently dropped. - TerraformrcConfig.credentials documentation clarifies it is for HTTP Terraform CLI registry auth (rendered as native credentials "host" {} blocks with a 'token' key), not Git module-source PAT auth. Wiring changes: - Terraform driver now renders credentials "host" { token = ... } blocks in the generated .terraformrc, resolving the token value from the fetched-secret map at execution time. Drops the broken bridge that forwarded credentials into the legacy Git PAT path (which expects the 'pat' key, not 'token'). - terraform driver FindSecretIDs reports the new credentials secret stores with the 'token' key so the engine fetches them before the driver runs. - Loader uses the new host-keyed Bicep map to populate the legacy Authentication map (BasicAuth only; AzureWI/AwsIrsa stay schema-only follow-ups). Validation: - Environment controller's referenced-config-id checks now also verify the resource type matches the expected Radius.Core/{terraformConfigs, bicepConfigs}, instead of accepting any resource ID that happens to exist (Copilot review #1, #2). - New bicepconfigs.ValidateRequest hook (UpdateFilter) enforces that authenticationMethod=BasicAuth requires basicAuthSecretId, AzureWI requires both Wi ids, and AwsIrsa requires the role ARN. TypeSpec cannot express these conditional-required-field rules without a discriminated- union restructure (Copilot review #6). Tests: - New TestWriteTerraformCLIConfig_Credentials exercises native credentials rendering: success, deterministic ordering, missing secret reference, missing secret data, missing 'token' key, and embedded-quote escaping. - TestWriteTerraformCLIConfig_BothBlocks asserts provider_installation and credentials co-render in one .terraformrc. - New bicepconfigs.TestValidateRequest covers the controller-level conditional validation matrix. - bicepconfig-test.bicep and tfbicep-combined-test.bicep updated to the new schema shape with a real SecretStore reference (was previously a no-op BasicAuth without a secret, which would now be rejected). Docs: - docs/architecture/terraform-bicep-config.md updated for the new schema, the credentials/token rendering, and the architecture README index now links the page (Copilot review #5).
Test_BicepConfig_CRUD and Test_TerraformConfig_BicepConfig_Combined were failing because the new test bicep files declared Applications.Core/secretStores resources at the default (global) scope without a properties.resource value. The secretStore controller rejects this with: $.properties.resource cannot be empty for global scoped resource. Bind each test secretStore to a Kubernetes secret in the env namespace (matching the pattern used by corerp-resources-terraform-postgres.bicep and corerp-resources-terraform-private-git-repo-redis.bicep).
…d tests Two bugs surfaced by Brooke's PR review: 1. validateConfigRef never rejected missing references. Operation.GetResource clears database.ErrNotFound (returning nil err with a nil resource), so the existing 'if err != nil' guard always passed. Now inspect the returned resource pointer too: nil => BadRequest 'does not exist'; transport errors => InternalServerError. New unit tests in validateconfigref_test.go cover invalid-id, wrong-type, not-found, db-error, and happy-path cases for both terraformConfig and bicepConfig. 2. In recipes/terraform/execute.go Deploy, generateConfig (which calls terraform get to download the module) ran before setEnvironmentVariables wrote .terraformrc and exported TF_CLI_CONFIG_FILE. Module fetches from authenticated registries therefore happened without the configured credentials block in effect. Move setEnvironmentVariables ahead of generateConfig so the rc file is in scope for the entire init/get/apply pipeline. Also fills in the missing test coverage Brooke called out: - pkg/corerp/api/v20250801preview/terraformconfig_conversion_test.go: nil, NetworkMirror-only, Direct-only, both, multi-host Credentials, nil-entry skipped, wrong-type, round-trip identity, pointer-aliasing guard. - pkg/corerp/api/v20250801preview/bicepconfig_conversion_test.go: empty, BasicAuth, AzureWI, AwsIrsa, nil-entry skipped, wrong-type, round-trip identity, two-distinct-entries pointer-aliasing guard for the fromBicepRegistryAuthDataModel loop. - pkg/recipes/configloader/environment_v20250801_bridge_test.go: terraform Credentials/Env/ProviderInstallation mapping, Bicep BasicAuth mapping, silent-skip behavior for entries without BasicAuthSecretId, no synthesized empty Authentication map, and error wrapping for both fetch failures.
…ltipleCredentialHosts
58aa321 to
b304290
Compare
Radius functional test overviewClick here to see the test run details
Test Status |
Description
Adds two new resources in the
Radius.Corenamespace —terraformConfigsandbicepConfigs— that environments reference by resource ID. Platform teams can now define private registry authentication, Terraform CLI configuration (provider_installation,credentials, env vars), and Bicep registry authentication once and share it across multiple environments.This implements a scoped subset of design-notes#107. Out of scope: Terraform binary lifecycle (
rad terraform install), backend state stores, installer pipeline. See docs/architecture/terraform-bicep-config.md for the full design.What's included
Radius.Core/terraformConfigsandRadius.Core/bicepConfigs(regenerated SDK).terraformConfig?andbicepConfig?properties onRadius.Core/environments, validated for existence at PUT time.pkg/corerp/setup/setup.go.pkg/recipes/configloader/environment.gothat resolves the new config resources at recipe execution time and populatesRecipeConfigconsumed by the shared driver.ProviderInstallationfield on the sharedTerraformConfigPropertiesdatamodel; the Terraform driver writes a.terraformrcto the working dir and points Terraform at it viaTF_CLI_CONFIG_FILEon both Deploy and Delete. Field is optional and only populated by the Radius.Core path — the legacyApplications.CorerecipeConfigflow is unchanged..terraformrcwriter, the env-var setter, and the Delete-path helper.Test_TerraformConfig_BicepConfig_Combined) deploying both configs in a single environment and running a Terraform recipe end-to-end.Known follow-ups (filed separately)
bicepConfig.registryAuthentication.{AzureWI,AwsIrsa}are accepted by the schema but not yet wired in the Bicep driver (onlyBasicAuthflows through today).referencedByon the config resources is exposed in the schema but not yet populated.Type of change
Fixes: #10615
Contributor checklist
Please verify that the PR meets the following requirements, where applicable: