Summary
The current test suite is entirely unit-level — all CloudFormation compilation is tested by inspecting generated JSON/YAML objects. This means a whole class of bugs can pass unit tests but still fail at deploy time.
The problem
CloudFormation-specific issues like circular resource dependencies, invalid ARN references, or misconfigured IAM policies are only caught when CloudFormation actually tries to resolve the template. No amount of unit testing can reliably catch these, because the bug only manifests when the full dependency graph is evaluated.
A concrete example: #470 (circular dependency when using !Ref to a Lambda function as a Task state resource). The logic looks reasonable in isolation but CloudFormation rejects the generated template. We cannot confidently fix — or even confirm — this bug without being able to deploy a real template.
Proposed solution
Use Serverless Compose to orchestrate a set of minimal fixture services, each representing a specific edge case or known bug scenario.
Structure:
fixtures/
serverless-compose.yml # orchestrates all fixtures
circular-dependency/
serverless.yml # reproduces #470
notifications-policy-merge/
serverless.yml # reproduces #275
lambda-arn-iam/
serverless.yml # reproduces #302
...
Running the full suite:
Targeting a single fixture:
sls compose deploy --service circular-dependency
Teardown:
Each fixture is a real serverless.yml that documents the exact scenario — self-explanatory to anyone reading the repo.
CI setup
CI runs the fixtures against LocalStack in a Docker container. The GitHub Actions workflow pattern is already proven — see BANCS-Norway/serverless-offline-sns — integration.yml as the reference implementation.
This keeps CI self-contained, fast, and free — no AWS costs, no credential management.
Why now
Several open bugs are difficult to reproduce, fix, or verify without this infrastructure. Fixing them without integration tests risks introducing regressions or shipping fixes that don't actually solve the problem.
Related issues
Summary
The current test suite is entirely unit-level — all CloudFormation compilation is tested by inspecting generated JSON/YAML objects. This means a whole class of bugs can pass unit tests but still fail at deploy time.
The problem
CloudFormation-specific issues like circular resource dependencies, invalid ARN references, or misconfigured IAM policies are only caught when CloudFormation actually tries to resolve the template. No amount of unit testing can reliably catch these, because the bug only manifests when the full dependency graph is evaluated.
A concrete example: #470 (circular dependency when using
!Refto a Lambda function as a Task state resource). The logic looks reasonable in isolation but CloudFormation rejects the generated template. We cannot confidently fix — or even confirm — this bug without being able to deploy a real template.Proposed solution
Use Serverless Compose to orchestrate a set of minimal fixture services, each representing a specific edge case or known bug scenario.
Structure:
Running the full suite:
Targeting a single fixture:
Teardown:
Each fixture is a real
serverless.ymlthat documents the exact scenario — self-explanatory to anyone reading the repo.CI setup
CI runs the fixtures against LocalStack in a Docker container. The GitHub Actions workflow pattern is already proven — see BANCS-Norway/serverless-offline-sns — integration.yml as the reference implementation.
This keeps CI self-contained, fast, and free — no AWS costs, no credential management.
Why now
Several open bugs are difficult to reproduce, fix, or verify without this infrastructure. Fixing them without integration tests risks introducing regressions or shipping fixes that don't actually solve the problem.
Related issues
!Reflambda resourceFn::Sub