Skip to content

feat: pivot psql-stack to EKS Auto Mode defaults #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
patrickleet merged 44 commits into from
May 8, 2026
Merged

feat: pivot psql-stack to EKS Auto Mode defaults #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
39e3ea6
feat: optional NodePool for dedicated psql workloads
patrickleet Apr 23, 2026
46fe4c9
feat: StorageClass + ExternalSecrets composed resources
patrickleet Apr 23, 2026
3a2946f
fix: default StorageClass name to 'psql' (per-stack naming)
patrickleet Apr 23, 2026
622fae5
feat: externalSecrets.connections[] composes password + config → URL
patrickleet Apr 23, 2026
ff1aec4
refactor: drop externalSecrets from PSQLStack
patrickleet Apr 23, 2026
7358566
feat: swap StackGres operator for CloudNativePG (phase 1 of CNPG pivot)
patrickleet Apr 24, 2026
8debdb6
feat: three-profile storage layer — mayastor + lvm + ebs (phase 2)
patrickleet Apr 25, 2026
cc258eb
feat: split NodePool into branches + primary sub-pools (phase 3)
patrickleet Apr 25, 2026
d6b5648
feat: install cnpg-i-scale-to-zero plugin (phase 4)
patrickleet Apr 25, 2026
5d6c3ff
docs/test: refresh README, examples, and render tests for CNPG pivot …
patrickleet Apr 25, 2026
2f6f2c8
feat: node-prep DaemonSet for Mayastor/LVM prereqs (phase 3b)
patrickleet Apr 25, 2026
0b7e221
refactor: drop LVM + EBS storage profiles, Mayastor-only
patrickleet Apr 25, 2026
e242e9a
nodepool + node-prep: spot primary, drop cluster prefix, DS creates D…
patrickleet Apr 25, 2026
dd8ad07
feat: stack-wide HA mode (replicaCount + topology spread by zone)
patrickleet Apr 25, 2026
da8d852
fix: skip Mayastor's bundled VolumeSnapshotClass CRDs
patrickleet Apr 25, 2026
4f33ca9
fix: confine Mayastor csi-node to workload-type=psql nodes
patrickleet Apr 25, 2026
727dc74
feat: pivot storage from Mayastor to Longhorn V2
patrickleet Apr 25, 2026
1ec04e6
feat: pivot psql-stack to EKS Auto Mode defaults
patrickleet Apr 25, 2026
3190483
fix: default snapshot driver to ebs.csi.eks.amazonaws.com
patrickleet Apr 27, 2026
38c2b7c
docs: flag snapshot-controller as a prerequisite
patrickleet Apr 27, 2026
b7a522f
docs: point snapshot-controller prereq at volume-snapshot-stack
patrickleet Apr 28, 2026
d4340db
feat: merge psql-cluster + psql-branch APIs into psql-stack package
patrickleet Apr 28, 2026
cdb8d5d
test(e2e): upgrade unified psql e2e to full Ready integration
patrickleet Apr 29, 2026
ee3373e
fix(e2e): drop StackGres-era spec fields from PSQLStack manifest
patrickleet Apr 29, 2026
2e08b7e
docs(psqlstacks): correct stale CSI driver and VSC defaults
patrickleet Apr 29, 2026
525fba0
test(psqlstacks): assert Atlas Release in scale-to-zero-disabled test
patrickleet Apr 29, 2026
e68d592
fix(psqlstacks): create CNPG/Atlas teardown-order Usage on existence,…
patrickleet Apr 29, 2026
03d233c
feat(psqlcluster)!: rework credentials around app/superuser, ESO-shap…
patrickleet Apr 29, 2026
5d8a092
build(make): derive xrd/composition/api_dir per example for multi-API…
patrickleet Apr 29, 2026
b789ac7
feat(psqlcluster): auto-gen app secret when neither field set
patrickleet Apr 30, 2026
990ddfd
feat(psqlstack)!: compose paired psql StorageClass + pivot e2e to AWS
patrickleet May 7, 2026
877c9c1
ci(e2e): install cert-stack alongside volume-snapshot-stack
patrickleet May 8, 2026
e537e26
ci: pin reusable workflows to hops-ops/workflows-crossplane@v3.0.0
patrickleet May 8, 2026
663b5e7
fix(psqlbranch): drop default postgres version, gate imageName on exp…
patrickleet May 8, 2026
b4c6e62
test(psqlcluster): lock in monitoring.enabled=false honors explicit off
patrickleet May 8, 2026
18d3115
fix(psqlbranch): prefer source-snapshot content over empty branch con…
patrickleet May 8, 2026
dd72818
test(psqlbranch): cross-namespace state-status with observed fixtures
patrickleet May 8, 2026
ed61c1d
fix(psqlbranch): prefix source-ns VolumeSnapshot name with branch nam…
patrickleet May 8, 2026
e1ad9ec
fix(psqlbranch): fall back to source.storage.size when branch size empty
patrickleet May 8, 2026
f3d3034
fix(psqlcluster): match ExternalSecret status lookup to actual resour…
patrickleet May 8, 2026
7f79901
fix(make): use defined Makefile vars in validate:% recipe
patrickleet May 8, 2026
45619d5
test(psqlcluster): assert full ExternalSecret data mapping
patrickleet May 8, 2026
e039995
fix(psqlcluster): bump ExternalSecret to external-secrets.io/v1
patrickleet May 8, 2026
6b8e83c
docs(psqlcluster): mirror externalSecret descriptions on superuser block
patrickleet May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 51 additions & 7 deletions .github/workflows/on-pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,28 +27,72 @@ permissions:

jobs:
validate:
uses: unbounded-tech/workflows-crossplane/.github/workflows/validate.yaml@v2.20.0
uses: hops-ops/workflows-crossplane/.github/workflows/validate.yaml@v3.0.0
with:
examples: |
[
{ "example": "examples/psqlstacks/minimal.yaml" },
{ "example": "examples/psqlstacks/standard.yaml" }
{ "example": "examples/psqlstacks/minimal.yaml", "api_path": "apis/psqlstacks" },
{ "example": "examples/psqlstacks/standard.yaml", "api_path": "apis/psqlstacks" },
{ "example": "examples/psqlclusters/minimal.yaml", "api_path": "apis/psqlclusters" },
{ "example": "examples/psqlclusters/standard.yaml","api_path": "apis/psqlclusters" },
{ "example": "examples/psqlbranches/same-namespace.yaml", "api_path": "apis/psqlbranches" }
]
api_path: apis/psqlstacks
error_on_missing_schemas: true

test:
uses: unbounded-tech/workflows-crossplane/.github/workflows/test.yaml@v2.20.0
uses: hops-ops/workflows-crossplane/.github/workflows/test.yaml@v3.0.0

e2e:
uses: unbounded-tech/workflows-crossplane/.github/workflows/e2e.yaml@v2.20.0
uses: hops-ops/workflows-crossplane/.github/workflows/e2e.yaml@v3.0.0
with:
# E2E provisions a real EKS Auto Mode cluster (mirror of aws-observe-stack)
# so the `psql` StorageClass + VolumeSnapshotClass land on a real
# ebs.csi.eks.amazonaws.com driver — same code path that runs on
# pat-local. Kind has no snapshot-capable CSI, so the prior kind-only
# e2e couldn't exercise PSQLBranch's snapshot/fork chain.
aws: true
aws-use-oidc: true
aws-account-id: "034489662075"
aws-region: us-east-2
aws-role-duration-seconds: 7200
write-env-files: true
env-vars: |
{
"ADMIN_ROLE_ARN": "${{ vars.ADMIN_ROLE_ARN }}",
"PRIVATE_SUBNET_ID_A": "${{ vars.PRIVATE_SUBNET_ID_A }}",
"PRIVATE_SUBNET_ID_B": "${{ vars.PRIVATE_SUBNET_ID_B }}"
}
debug-resource-types: |
[
"psqlstacks.hops.ops.com.ai",
"psqlclusters.hops.ops.com.ai",
"psqlbranches.hops.ops.com.ai",
"volumesnapshotstacks.hops.ops.com.ai",
"certstacks.hops.ops.com.ai",
"autoeksclusters.aws.hops.ops.com.ai"
]
# AutoEKSCluster outlives the manifests' cascade delete; the workflow
# tears it down separately. VolumeSnapshotStack + CertStack are
# colocated with the cluster — drop them together so their Helm
# releases on the EKS cluster are uninstalled before the cluster goes.
delete-extra-resources: |
[
"volumesnapshotstacks.hops.ops.com.ai",
"certstacks.hops.ops.com.ai",
"autoeksclusters.aws.hops.ops.com.ai"
]
# 90 min: ~10–15 min EKS Auto Mode cold start, then platform install +
# CNPG bootstrap + snapshot fork.
timeout-minutes: 90
cleanup-timeout-minutes: 30
delete-extra-resources-timeout-minutes: 30

publish:
needs:
- validate
- test
- e2e
uses: unbounded-tech/workflows-crossplane/.github/workflows/publish.yaml@v2.20.0
uses: hops-ops/workflows-crossplane/.github/workflows/publish.yaml@v3.0.0
secrets: inherit
with:
tag: pr-${{ github.event.pull_request.number }}-${{ github.sha }}
18 changes: 12 additions & 6 deletions .github/workflows/on-push-main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,27 @@ permissions:

jobs:
validate:
uses: unbounded-tech/workflows-crossplane/.github/workflows/validate.yaml@v2.20.0
uses: unbounded-tech/workflows-crossplane/.github/workflows/validate.yaml@feat/multi-api-support
with:
examples: |
[
{ "example": "examples/psqlstacks/minimal.yaml" },
{ "example": "examples/psqlstacks/standard.yaml" }
{ "example": "examples/psqlstacks/minimal.yaml", "api_path": "apis/psqlstacks" },
{ "example": "examples/psqlstacks/standard.yaml", "api_path": "apis/psqlstacks" },
{ "example": "examples/psqlclusters/minimal.yaml", "api_path": "apis/psqlclusters" },
{ "example": "examples/psqlclusters/standard.yaml","api_path": "apis/psqlclusters" },
{ "example": "examples/psqlbranches/same-namespace.yaml", "api_path": "apis/psqlbranches" }
]
api_path: apis/psqlstacks
error_on_missing_schemas: true

test:
uses: unbounded-tech/workflows-crossplane/.github/workflows/test.yaml@v2.20.0
uses: unbounded-tech/workflows-crossplane/.github/workflows/test.yaml@feat/multi-api-support

e2e:
uses: unbounded-tech/workflows-crossplane/.github/workflows/e2e.yaml@v2.20.0
uses: unbounded-tech/workflows-crossplane/.github/workflows/e2e.yaml@feat/multi-api-support
Comment on lines +26 to +42
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/on-push-main.yaml | head -50

Repository: hops-ops/psql-stack

Length of output: 1893

🏁 Script executed:

cat -n .github/workflows/on-push-main.yaml

Repository: hops-ops/psql-stack

Length of output: 2013

Pin reusable workflows to immutable refs before merging.

The validate, test, and e2e jobs reference @feat/multi-api-support, a mutable branch that can change without review. This breaks mainline CI reproducibility—different runs may execute different code. Pin these to immutable release tags (like the v1.21.0 used for version-and-tag).

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/on-push-main.yaml around lines 26 - 42, The workflow
currently references mutable branch refs for the reusable workflows in the
validate, test, and e2e jobs (the lines using
unbounded-tech/workflows-crossplane/.github/workflows/validate.yaml@feat/multi-api-support,
test.yaml@feat/multi-api-support, and e2e.yaml@feat/multi-api-support); update
those refs to an immutable release tag (for example replace
`@feat/multi-api-support` with a specific tag like `@v1.21.0` or another pinned tag)
so validate, test, and e2e always run the exact released workflow code.

with:
# See on-pr.yaml — 90 min for the full Ready chain.
timeout-minutes: 90
cleanup-timeout-minutes: 30

version-and-tag:
name: Version and Tag
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ _output/
# Temporary files
.tmp/

# E2E test credentials (never commit secrets)
# E2E test credentials + per-run env (never commit secrets)
**/aws-creds
tests/**/secrets/
tests/**/env/
apis/**/configuration.yaml
33 changes: 24 additions & 9 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
SHELL := /bin/bash

PACKAGE ?= psql-stack
# Default XRD_DIR for legacy single-API targets; multi-API targets derive per-example.
XRD_DIR := apis/psqlstacks
COMPOSITION := $(XRD_DIR)/composition.yaml
DEFINITION := $(XRD_DIR)/definition.yaml
EXAMPLE_DEFAULT := examples/psqlstacks/standard.yaml
RENDER_TESTS := $(wildcard tests/test-*)
E2E_TESTS := $(wildcard tests/e2etest-*)

# Multi-API support: examples/<apiplural>/<example>.yaml maps to apis/<apiplural>/.
# Helper macro: api_dir_for(example_path) → apis/<dirname>
api-dir = apis/$(word 2,$(subst /, ,$(1)))

clean:
rm -rf _output
rm -rf .up
Expand All @@ -17,9 +22,13 @@ build:

# Examples list - mirrors GitHub Actions workflow
# Format: example_path::observed_resources_path (observed_resources_path is optional)
# api_path is derived from example_path via the api-dir macro (examples/<x>/... → apis/<x>/).
EXAMPLES := \
examples/psqlstacks/minimal.yaml:: \
examples/psqlstacks/standard.yaml::
examples/psqlstacks/standard.yaml:: \
examples/psqlclusters/minimal.yaml:: \
examples/psqlclusters/standard.yaml:: \
examples/psqlbranches/same-namespace.yaml::
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# Render all examples (parallel execution, output shown per-job when complete)
render\:all:
Expand All @@ -28,14 +37,17 @@ render\:all:
for entry in $(EXAMPLES); do \
example=$${entry%%::*}; \
observed=$${entry#*::}; \
api_dir=$$(echo "$$example" | awk -F/ '{print "apis/" $$2}'); \
composition="$$api_dir/composition.yaml"; \
definition="$$api_dir/definition.yaml"; \
outfile="$$tmpdir/$$(echo $$entry | tr '/:' '__')"; \
( \
if [ -n "$$observed" ]; then \
echo "=== Rendering $$example with observed-resources $$observed ==="; \
up composition render --xrd=$(DEFINITION) $(COMPOSITION) $$example --observed-resources=$$observed; \
up composition render --xrd=$$definition $$composition $$example --observed-resources=$$observed; \
else \
echo "=== Rendering $$example ==="; \
up composition render --xrd=$(DEFINITION) $(COMPOSITION) $$example; \
echo "=== Rendering $$example (api=$$api_dir) ==="; \
up composition render --xrd=$$definition $$composition $$example; \
fi; \
echo "" \
) > "$$outfile" 2>&1 & \
Expand All @@ -58,18 +70,21 @@ validate\:all:
for entry in $(EXAMPLES); do \
example=$${entry%%::*}; \
observed=$${entry#*::}; \
api_dir=$$(echo "$$example" | awk -F/ '{print "apis/" $$2}'); \
composition="$$api_dir/composition.yaml"; \
definition="$$api_dir/definition.yaml"; \
outfile="$$tmpdir/$$(echo $$entry | tr '/:' '__')"; \
( \
if [ -n "$$observed" ]; then \
echo "=== Validating $$example with observed-resources $$observed ==="; \
up composition render --xrd=$(DEFINITION) $(COMPOSITION) $$example \
up composition render --xrd=$$definition $$composition $$example \
--observed-resources=$$observed --include-full-xr --quiet | \
crossplane beta validate $(XRD_DIR) --error-on-missing-schemas -; \
crossplane beta validate $$api_dir --error-on-missing-schemas -; \
else \
echo "=== Validating $$example ==="; \
up composition render --xrd=$(DEFINITION) $(COMPOSITION) $$example \
echo "=== Validating $$example (api=$$api_dir) ==="; \
up composition render --xrd=$$definition $$composition $$example \
--include-full-xr --quiet | \
crossplane beta validate $(XRD_DIR) --error-on-missing-schemas -; \
crossplane beta validate $$api_dir --error-on-missing-schemas -; \
fi; \
echo "" \
) > "$$outfile" 2>&1 & \
Expand Down
Loading
Loading