feat: make host mode directory paths configurable#7225
Conversation
|
license check fails due to not indexed yet. I'll fix the other things later https://deps.dev/go/github.com%2Ftetratelabs%2Ffunc-e |
6383591 to
5398829
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7225 +/- ##
==========================================
+ Coverage 71.97% 71.99% +0.01%
==========================================
Files 229 230 +1
Lines 33201 33280 +79
==========================================
+ Hits 23897 23960 +63
- Misses 7573 7579 +6
- Partials 1731 1741 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
🤞 build is green now |
|
@arkodg @nacx @missBerg fyi I am off next week but wanted to eliminate this nagging tech debt for good. This PR uses XDG conventions (like prometheus, goose etc), which map into the same change made in func-e. The result is no more hard-coded /tmp/envoy-gateway and flexibility to change things in docker etc. Most importantly, envoy-ai-gateway is blocked on this because the certs directory is hard-coded here. This causes problems as when you run tests that update certs it breaks your host configuration, as there's no way to overwrite it before this change. So, TL;DR; this isn't just a random TODO fix, this is a big deal for sustainability forward and we should merge soon, especially as I can finish up the whole thing soup-to-nuts including ai gateway if folks can prioritize it this week! |
|
OIDC_with_BackendCluster is a very flaky test.
Please kick the build. Maybe we can prioritize deflaking this after merge, especially as you can use a clean certs directory after. The certs being hard-coded directory smashed by other tests, caused flakes in ai-gateway also. |
|
incidentally this flake reminded me certs should be in the ConfigHome as they are configuration bound (for exactly the same reason as flakes when one test needs certs for its config). changing this now |
5398829 to
66871b3
Compare
|
updated wrt cert location |
This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.
The implementation introduces four configurable directory paths:
- configHome: certificates and configuration files (default: ~/.config/envoy-gateway)
- dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway)
- stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway)
- runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID})
Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.
These paths are propagated to func-e which creates subdirectories as needed:
- dataHome/envoy-versions/ for Envoy binaries
- stateHome/envoy-runs/{runID}/ for per-run logs
- runtimeDir/{runID}/ for per-run runtime files
The changes include path resolution helpers, updated runners to use configurable paths, comprehensive test coverage, and updated documentation.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
84779d3 to
935470e
Compare
|
/retest |
|
/retest |
|
@codefromthecrypt can you fix the conflict? |
|
adrian is ooo until next week i think |
|
if the conflicts are not huge then you maintainer folks can push directly to the branch then merge i guess |
Signed-off-by: Arko Dasgupta <arkodg@users.noreply.github.com>
#1368) **Description** This refactors aigw to use distinct directories following XDG Base Directory Specification conventions: - --config-home/$AIGW_CONFIG_HOME: Configuration files (default: ~/.config/aigw) - --data-home/$AIGW_DATA_HOME: Envoy binaries via func-e (default: ~/.local/share/aigw) - --state-home/$AIGW_STATE_HOME: Run logs and state (default: ~/.local/state/aigw) - --runtime-dir/$AIGW_RUNTIME_DIR: Ephemeral files like UDS (default: /tmp/aigw-${UID}) This separation aligns with XDG principles where configuration, data, state, and runtime files are independently configurable for different storage tiers. This is particularly useful for Docker deployments to map volumes appropriately. This also adds --run-id/$AIGW_RUN_ID to override the default YYYYMMDD_HHMMSS_UUU timestamp format with a custom identifier. Setting this to '0' enables predictable paths for Docker/Kubernetes single-run scenarios. **Related Issues/PRs (if applicable)** once envoyproxy/gateway#7225 is merged we have some maintenance to remove the /tmp/envoy-gateway/certs tech debt --------- Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.
The implementation introduces four configurable directory paths:
- configHome: certificates and configuration files (default: ~/.config/envoy-gateway)
- dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway)
- stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway)
- runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID})
Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.
These paths are propagated to func-e which creates subdirectories as needed:
- dataHome/envoy-versions/ for Envoy binaries
- stateHome/envoy-runs/{runID}/ for per-run logs
- runtimeDir/{runID}/ for per-run runtime files
The changes include path resolution helpers, updated runners to use configurable paths, comprehensive test coverage, and updated documentation.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Lin Moskovitch <lin.moskovitch@sap.com>
| cmd.PersistentFlags().BoolVarP(&local, "local", "l", false, | ||
| "Generate all the certificates locally.") | ||
| cmd.PersistentFlags().StringVar(&dataHome, "data-home", "", | ||
| "Directory for certificates (defaults to ~/.local/share/envoy-gateway)") |
There was a problem hiding this comment.
shouldn't this be ~/.config/envoy-gateway given that CertDir is written like this? cc @codefromthecrypt
// CertDir returns the certificate directory path (under ConfigHome).
func (p *Paths) CertDir(component string) string {
return filepath.Join(p.ConfigHome, "certs", component)
}
There was a problem hiding this comment.
the other way works too (changing CertDir to use ~/.local/share/envoy-gateway)
envoyproxy#1368) **Description** This refactors aigw to use distinct directories following XDG Base Directory Specification conventions: - --config-home/$AIGW_CONFIG_HOME: Configuration files (default: ~/.config/aigw) - --data-home/$AIGW_DATA_HOME: Envoy binaries via func-e (default: ~/.local/share/aigw) - --state-home/$AIGW_STATE_HOME: Run logs and state (default: ~/.local/state/aigw) - --runtime-dir/$AIGW_RUNTIME_DIR: Ephemeral files like UDS (default: /tmp/aigw-${UID}) This separation aligns with XDG principles where configuration, data, state, and runtime files are independently configurable for different storage tiers. This is particularly useful for Docker deployments to map volumes appropriately. This also adds --run-id/$AIGW_RUN_ID to override the default YYYYMMDD_HHMMSS_UUU timestamp format with a custom identifier. Setting this to '0' enables predictable paths for Docker/Kubernetes single-run scenarios. **Related Issues/PRs (if applicable)** once envoyproxy/gateway#7225 is merged we have some maintenance to remove the /tmp/envoy-gateway/certs tech debt --------- Signed-off-by: Adrian Cole <adrian@tetrate.io> Signed-off-by: Erica Hughberg <erica.sundberg.90@gmail.com>
What this PR does / why we need it:
Before, all paths were hard coded to start with /tmp/envoy-gateway, making parallel runs impractical, especially when certs change.
This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.
The implementation introduces four configurable directory paths:
Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.
These paths are propagated to func-e which creates subdirectories as needed: