Reference AWS ARNs across multiple CloudFormation stacks
This article will enable you to dynamically cross reference AWS ARNs across multiple CloudFormation stacks without hardcoding any values. This is possible by using theOutputsproperty in one stack combined with!Importfeature in another. This is useful when running multiple CloudFormation stacks in the same workload.
The workload
Consider a workload consisting of two CloudFormation stacks:
Stack A produces events through user actions (aka "the events producer").
Stack B consumes events via worker Lambdas (aka "the events consumer").
Workload description
Stack A is responsible for user management flows (sign-up, get profile data etc) and emits DynamoDB StreamEvents when new rows are inserted into the Users table. This in turn triggers an account activation email to be sent to the user in a Lambda function provisioned in Stack B.
To make this work, Stack B needs to know the ARN of the DynamoDB stream provisioned in Stack A.
In addition, we need an SNS topic to trigger a password reset email, when the user wants to reset their account password.
To summarise, we have two event triggers in Stack A:
- DynamoDB StreamEvent: New user signup triggering activation email
- SNS Password Reset topic: Triggered using
sns.publish
DynamoDB StreamEvent
This behaviour is defined using the StreamSpecification property on the table resource:
Resources
UserTable
...
...
StreamSpecification:
StreamViewType: NEW_IMAGE
When you deploy the stack, you see the Stream ARN being associated with your table:
arn:aws:dynamodb:ap-southeast-2:XXXXXXXXXXXX:table/dev.id-api.users/stream/2022-11-26T12:14:37.573
Make notice of the last bit of the ARN 2022-11-26T12:14:37.573. An ISO date string, not anything we can guess or calculate using CloudFormation syntax e.g Fn::GetAtt (more on that later).
Creating the Output reference
In my CloudFormation template for Stack A I'm defining 4 outputs each consisting of Description, Value and Export:
resources:
Resources:
UserTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:custom.usersTableName}
...
...
StreamSpecification:
StreamViewType: NEW_IMAGE ## NEW_AND_OLD_IMAGES
TopicPasswordReset:
Type: AWS::SNS::Topic
Properties:
TopicName: ${sls:stage}-password-reset
DisplayName: Trigger password reset flow
Outputs:
UserTableName:
Description: User table
Value:
Ref: UserTable
Export:
Name: ${sls:stage}-UserTableName
UserTableArn:
Description: The ARN of the user table
Value: { "Fn::GetAtt": ["UserTable", "Arn"] }
Export:
Name: ${sls:stage}-UserTableArn
UserTableStreamArn:
Description: The ARN of the user table event stream
Value: { "Fn::GetAtt": ["UserTable", "StreamArn"] }
Export:
Name: ${sls:stage}-UserTableStreamArn
PasswordResetTopicArn:
Description: Password reset SNS topic ARN
Value:
Ref: TopicPasswordReset
Export:
Name: ${sls:stage}-PasswordResetTopicArnAfter deploying the stack, in AWS Console > CloudFormation > Outputs you can see the created export names. We are going to use those names to reference the ARN in Stack B.
And the SNS topic was also created:
Reference resources in Stack B provisioned in Stack A
There are a number of ways to do this.
Attempt #1 Copy/paste ARN
Manually maintaining the ARN string is something we'd want to avoid. It will work and you can easily copy it from AWS Console, however this is not recommended due to maintainability issues.
arn:aws:dynamodb:ap-southeast-2:XXXXXXXXXXXX:table/dev.id-api.users/stream/2022-11-26T12:14:37.573
functions:
worker_accountCreated:
handler: src/workers/account/account-created.handler
events:
- stream:
type: dynamodb
arn: arn:aws:dynamodb:ap-southeast-2:XXXXXXXXXXXX:table/dev.id-api.users/stream/2022-11-26T12:14:37.573
filterPatterns:
- eventName: [INSERT]
batchSize: 1
worker_passwordReset:
handler: src/workers/account/password-reset.handler
events:
- sns:
arn: arn:aws:sns:${aws:region}:${aws:accountId}:${sls:stage}-password-reset
topicName: ${sls:stage}-password-resetAttempt #2 Using variables
Using the CloudFormation variable syntax is much better:
arn:aws:dynamodb:${aws:region}:${aws:accountId}:table/${self:custom.usersTableName}/stream/2022-11-26T12:14:37.573
However, this is still not good enough, since the last part of the ARN is a timestamp for which there is no variable, leaving this approach as useless as the first.
functions:
worker_accountCreated:
handler: src/workers/account/account-created.handler
events:
- stream:
type: dynamodb
arn: arn:aws:dynamodb:${aws:region}:${aws:accountId}:table/${self:custom.usersTableName}/stream/2022-11-26T12:14:37.573
filterPatterns:
- eventName: [INSERT]
batchSize: 1
worker_passwordReset:
handler: src/workers/account/password-reset.handler
events:
- sns:
arn: arn:aws:sns:${aws:region}:${aws:accountId}:${sls:stage}-password-reset
topicName: ${sls:stage}-password-resetAttempt #3 Using the !ImportValue method
The most flexible way is pointing directly to the exported CloudFormation value in Stack A:
arn: !ImportValue ${sls:stage}-UserTableStreamArn
Where UserTableStreamArn is a direct reference to Outputs > UserTableStreamArn in serverless.yml
This final working snippet:
functions:
worker_accountCreated:
handler: src/workers/account/account-created.handler
events:
- stream:
arn: !ImportValue ${sls:stage}-UserTableStreamArn
filterPatterns:
- eventName: [INSERT]
batchSize: 1
worker_passwordReset:
handler: src/workers/account/password-reset.handler
events:
- sns:
arn: !ImportValue ${sls:stage}-PasswordResetTopicArn
topicName: ${sls:stage}-password-resetCaveats
Order of deployment matters.
If you first deploy Stack B and reference ARNs in Stack A, the deployment will fail because the resources you are pointing to have not been created yet.
In addition the !ImportValue syntax will only work for resources in the same account and region.
If you have needs exceeding the capabilities of AWS SNS, I recommend using AWS EventBridge.
Jeff Barr