-
Notifications
You must be signed in to change notification settings - Fork 218
test: add post-deploy template assertion suite for integration fixtures #748
Description
Background
The current integration test suite (added in #745) deploys all fixtures to LocalStack and verifies that deployment succeeds. This catches runtime errors — malformed IAM policies, invalid state machine definitions, etc.
However, some bugs produce structurally incorrect CloudFormation templates that LocalStack accepts without error but that cause silent failures in production. Two examples already fixed:
- Using notifications overwrites exisiting resouce permissions. #275 — multiple
AWS::SNS::TopicPolicyresources targeting the same topic. CloudFormation silently overwrites the first with the second. LocalStack deploys green; the bug is only visible by inspecting the generated template. - Circular dependency calling intrinsic function #470 —
{ Ref: LambdaFunction }in an IAM policy resource resolves to the function name, not its ARN. LocalStack's IAM validates this and rejects it — but only because IAM happens to validate it. A future similar bug might not be caught this way.
Proposed solution
After sls compose deploy runs (which internally packages each fixture and writes the compiled template to .serverless/cloudformation-template-update-stack.json), run a second step that executes verify.test.js files co-located with each fixture:
fixtures/
notifications/
serverless.yml
verify.test.js ← asserts exactly one SNS TopicPolicy per topic, etc.
circular-dependency/
serverless.yml
verify.test.js ← asserts no { Ref: Lambda } in IAM policy resources, etc.
basic-state-machine/
serverless.yml
# no verify.test.js needed — deploy success is sufficient
Runner added to the CI workflow after the deploy step:
npx mocha fixtures/**/verify.test.jsThe verify.test.js files use the same Mocha + Chai setup as the rest of the test suite and read the already-generated .serverless/cloudformation-template-update-stack.json for their assertions.
Why this matters
This closes the gap between "deploys without error" and "generates a correct template". It also gives future contributors a clear pattern for adding template-level assertions when fixing compilation bugs.