perf(exchange): replace peerFamilyGroups allowlist + per-rec offering API with cached Cost Explorer recommendations

[cristim]

Summary

The RI Exchange / reshape pipeline enumerates cross-family alternatives via a hand-curated peerFamilyGroups allowlist plus a per-recommendation DescribeReservedInstancesOfferings API call. CUDly already caches AWS Cost Explorer purchase recommendations in Postgres — the reshape page should pair underutilised convertible RIs against that cache instead.

Captured in: known_issues/24_exchange_offering_cache.md

Current behaviour

In pkg/exchange/reshape.go:

1. ListConvertibleReservedInstances surfaces underutilised convertible RIs.
2. peerFamilyGroups (hardcoded map) picks candidate target families.
3. FindConvertibleOfferings (providers/aws/services/ec2/client.go) calls DescribeReservedInstancesOfferings per recommendation to enrich pricing.
4. passesDollarUnitsCheck filters the surviving candidates.

Consequences today:

• New EC2 families are invisible until someone updates peerFamilyGroups by hand.
• Every reshape page load fans out N×M offering-enumeration API calls.
• Specialty/legacy alternatives exist only because of hand-curated group lists.

Expected behaviour

The recommendations table already stores Cost Explorer's GetReservationPurchaseRecommendation output (providers/aws/recommendations/client.go). Pair underutilised convertibles against those cached recommendations — AWS's own advice is strictly more relevant than enumerating every offering in the family, and the data is already in Postgres.

Steps to reproduce

1. Launch a reshape page load while tailing backend logs.
2. Observe one DescribeReservedInstancesOfferings call per convertible RI, gated by the static peerFamilyGroups allowlist.
3. Confirm the recommendations table already has cached Cost Explorer suggestions (SELECT provider,service,region,resource_type FROM recommendations WHERE provider='aws' AND service='ec2').

Proposed fix

• Delete peerFamilyGroups, candidateFamilies, and alternativesForTarget from pkg/exchange/reshape.go.
• Replace the OfferingLookup signature with a recommendation-driven PurchaseRecLookup(ctx, region, currencyCode) ([]OfferingOption, error) that reads store.ListStoredRecommendations(...) and maps each row to an OfferingOption (effective monthly = UpfrontCost/termMonths + MonthlyCost).
• Rename AnalyzeReshapingWithOfferings → AnalyzeReshapingWithRecs; the existing passesDollarUnitsCheck gate is unchanged.
• Update internal/api/handler_ri_exchange.go::convertToExchangeTypes and internal/server/handler_ri_exchange.go::convertForAutoExchange to thread the new adapter.
• Delete providers/aws/services/ec2/client.go::FindConvertibleOfferings if no other callers (pre-flight grep).

No new schema, no new store API, no scheduler hook needed — the recommendations table is already populated on every scheduler tick.

Edge cases

• Cache empty / cold start: no alternatives — same UX as a region with no AWS-recommended buys.
• No Cost Explorer recs in region: correct zero-alternatives display; AWS hasn't recommended anything.
• Term or currency mismatch: passesDollarUnitsCheck rejects the pair; filter by currency at SQL layer too.
• Cross-account leak: RecommendationFilter must include source account ID — reshape is per-account.
• auto.go path: uses base AnalyzeReshaping (no lookup) → AlternativeTargets=nil. Verify with grep -rn "AlternativeTargets" internal/ pkg/.

Test plan

Unit tests in pkg/exchange/:

• TestAnalyzeReshapingWithRecs_RecommendationDrivenAlternatives — fake lookup spanning m5/c5/r5, asserts cross-family alternatives surface.
• TestAnalyzeReshapingWithRecs_EmptyLookupReturnsNoAlternatives — cold-start UX.
• TestAnalyzeReshapingWithRecs_AppliesDollarUnitsFilter — filter gate works.
• Migrate existing TestAnalyzeReshapingWithOfferings_* to the new signature.
• Delete TestCandidateFamilies_*.

Handler tests:

• TestPurchaseRecLookupFromStore_RegionFilter / AccountFilter / NoRecsReturnsEmpty.

References

• Commit cd440d9ea — feat(exchange): cross-family RI alternatives for specialty + legacy (the hand-curated fix this replaces).
• pkg/exchange/reshape.go, providers/aws/services/ec2/client.go::FindConvertibleOfferings.
• internal/config/store_postgres_recommendations.go::ListStoredRecommendations.
• Known-issue doc: known_issues/24_exchange_offering_cache.md.

Effort

Small — single refactor commit, no schema changes, no new failure surfaces. ~0.5–1 day including test updates.

Verification

go build ./...
go test -short -count=1 -race ./pkg/exchange/... ./internal/api/... ./internal/server/...

Post-deploy: reshape page surfaces alternatives for instance types launched after the last hand-curated allowlist update; zero DescribeReservedInstancesOfferings calls fire during a reshape load.

[cristim]

Verification

Original concern: The RI Exchange / reshape pipeline used a hand-curated peerFamilyGroups allowlist plus a per-recommendation DescribeReservedInstancesOfferings API call. The asks were: drop the allowlist, drop the per-rec API fan-out, and source cross-family alternatives from the cached AWS Cost Explorer purchase recommendations the scheduler already populates.

What merged: PR #79 (perf(exchange): rec-driven cross-family alternatives via cached Cost Explorer recommendations, merged 2026-04-28). Replaces peerFamilyGroups / candidateFamilies / alternativesForTarget with a recommendations-driven PurchaseRecLookup closure; renames AnalyzeReshapingWithOfferings → AnalyzeReshapingWithRecs; deletes providers/aws/services/ec2/client.go::FindConvertibleOfferings; adds internal/api/exchange_lookup.go::purchaseRecLookupFromStore reading store.ListStoredRecommendations. Term-matching, primary-target exclusion, dollar-units pre-filter all preserved. PR #94 was a separate Savings Plans normalisation chore — not related to this issue, despite the parent task hint.

Code state today (branch feat/multicloud-web-frontend):

• pkg/exchange/reshape.go:99 — PurchaseRecLookup type defined.
• pkg/exchange/reshape.go:305 — AnalyzeReshapingWithRecs exported.
• internal/api/exchange_lookup.go:41 — purchaseRecLookupFromStore(store, accountID) adapter calls store.ListStoredRecommendations.
• internal/api/handler_ri_exchange.go:282 — handler threads the new lookup through.
• git grep peerFamilyGroups on the branch returns zero hits — the allowlist is gone.
• git grep FindConvertibleOfferings returns zero hits — the per-rec offerings API is deleted.
• Test coverage: internal/api/exchange_lookup_test.go (region/account/empty/error/mapping/3y-term cases) + pkg/exchange/reshape_crossfamily_test.go (recommendation-driven, cold cache, dollar-units filter, lookup error, primary-target exclusion, legacy m4).

Recommendation: close. PR #79's Closes #40 reflects reality — the allowlist and per-rec API call have both been deleted, the cache-driven path is wired end to end with regression tests, and the AnalyzeReshaping (no-alternatives) path used by auto.go is preserved.

[cristim]

Live verification — SKIPPED

#40 is a backend perf change (peerFamilyGroups allowlist + per-rec offering API replaced by cached Cost Explorer recommendations). The behavioural delta is "fewer/smaller AWS API calls per cross-family alternatives request" and "richer alternatives sourced from cached recs" — neither of which is externally observable from a logged-in session against the deployed Lambda URL.

What was checked: the deployed /api/recommendations returns 33 recs and the bulk-purchase + reshape surfaces are functional, so the perf change didn't regress the listing path. But that doesn't prove the new code path is the one taking effect (vs a fallback to the old path).

Not externally verifiable from a logged-in session because: AWS API call counts and the specific code path used to source cross-family alternatives are server-side observability, not reflected in client-visible response shape.

Relying on PR #79 test suite — the Cost-Explorer-driven cross-family alternatives via cached Cost Explorer recommendations change passed full root go test ./... on merge (2026-04-29).

Tab: tabId 1893022023, time UTC 2026-05-03T20:30Z