Static Documentation

AWS SNS and CloudWatch

Version latest · Updated 2026-09-01
Interactive docs View on GitHub

AWS SNS and CloudWatch integration#

IncidentRelay can receive signed Amazon SNS messages and CloudWatch alarm notifications through a dedicated inbound integration. Messages are accepted only after the SNS signature, signing certificate URL, route source, route state, and exact Topic ARN have been validated.

Endpoint#

POST /api/integrations/aws-sns/{route_id}

Example:

https://incidentrelay.example.com/api/integrations/aws-sns/17

This endpoint does not use a route intake bearer token. Amazon SNS authenticates requests with its message signature, while IncidentRelay also requires the TopicArn to exactly match the value stored in the route.

Create an IncidentRelay route#

  • Open Routes.
  • Create a route.
  • Select AWS SNS / CloudWatch as the source.
  • Select the owning team.
  • Enter the exact SNS Topic ARN.
  • Configure matchers and grouping.
  • Select a rotation or escalation policy.
  • Enable and save the route.
  • Open route details and copy the SNS webhook URL.

Example Topic ARN:

arn:aws:sns:eu-west-1:123456789012:incidentrelay-alerts

Recommended grouping:

[
  "cloudwatch_alarm_arn"
]

This keeps all state changes for one CloudWatch alarm in the same IncidentRelay group.

Create the SNS subscription#

In Amazon SNS:

  • Open the topic whose ARN is configured in the route.
  • Create a subscription.
  • Select HTTPS as the protocol.
  • Set the endpoint to the IncidentRelay webhook URL.
  • Create the subscription.

Amazon SNS sends a signed SubscriptionConfirmation message. IncidentRelay validates it and confirms the subscription automatically.

Configure a CloudWatch alarm#

Attach the SNS topic to the CloudWatch alarm notification actions. For a complete lifecycle, send notifications for at least:

ALARM
OK

ALARM creates or updates a firing IncidentRelay alert. OK resolves the existing alert because both notifications use the same CloudWatch alarm ARN.

INSUFFICIENT_DATA is treated as firing with warning severity.

AWS CLI configuration example#

The same setup can be created from the AWS CLI. First subscribe the IncidentRelay route endpoint to the SNS topic:

TOPIC_ARN='arn:aws:sns:eu-west-1:123456789012:incidentrelay-alerts'

aws sns subscribe \
  --topic-arn "$TOPIC_ARN" \
  --protocol https \
  --notification-endpoint 'https://incidentrelay.example.com/api/integrations/aws-sns/17'

IncidentRelay validates the signed SubscriptionConfirmation request and confirms the subscription automatically.

Then create or update a CloudWatch alarm and send both alarm and recovery actions to the same SNS topic:

aws cloudwatch put-metric-alarm \
  --alarm-name HighCPU \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --statistic Average \
  --period 300 \
  --evaluation-periods 2 \
  --threshold 90 \
  --comparison-operator GreaterThanThreshold \
  --alarm-actions "$TOPIC_ARN" \
  --ok-actions "$TOPIC_ARN"

Using the same topic in both --alarm-actions and --ok-actions is what gives IncidentRelay the complete firing → resolved lifecycle.

CloudWatch state mapping#

CloudWatch stateIncidentRelay statusDefault severity
ALARMfiringcritical
INSUFFICIENT_DATAfiringwarning
OKresolvedinfo

A severity SNS message attribute overrides the default severity.

Deduplication#

IncidentRelay uses AlarmArn as both the external identifier and deduplication key.

Example:

arn:aws:cloudwatch:eu-west-1:123456789012:alarm:HighCPU

The same ARN must be present in ALARM and OK notifications so that the existing alert is updated instead of duplicated.

Normalized labels#

Common labels include:

IncidentRelay labelSource
alertnameAlarmName
severitySNS attribute or state mapping
aws_servicecloudwatch
aws_account_idAWSAccountId
aws_regionRegion
cloudwatch_alarm_arnAlarmArn
cloudwatch_stateNewStateValue
cloudwatch_previous_stateOldStateValue
cloudwatch_metric_nameTrigger.MetricName
cloudwatch_namespaceTrigger.Namespace
cloudwatch_statisticTrigger.Statistic
cloudwatch_unitTrigger.Unit
cloudwatch_periodTrigger.Period
cloudwatch_evaluation_periodsTrigger.EvaluationPeriods
cloudwatch_datapoints_to_alarmTrigger.DatapointsToAlarm
cloudwatch_comparison_operatorTrigger.ComparisonOperator
cloudwatch_thresholdTrigger.Threshold
cloudwatch_treat_missing_dataTrigger.TreatMissingData
sns_topic_arnSNS TopicArn
sns_message_idSNS MessageId

CloudWatch dimensions use the dimension_ prefix. For example:

dimension_instanceid=i-0123456789abcdef0

SNS message attributes#

String SNS message attributes are converted into labels.

{
  "team": {
    "Type": "String",
    "Value": "sre"
  },
  "environment": {
    "Type": "String",
    "Value": "production"
  },
  "severity": {
    "Type": "String",
    "Value": "critical"
  }
}

These become:

team=sre
environment=production
severity=critical

The team and oncall_team attributes can provide a team hint, but route matching remains authoritative.

Example route matchers#

Production only:

{
  "environment": "production"
}

One AWS account:

{
  "aws_account_id": "123456789012"
}

EC2 CPU alarms:

{
  "cloudwatch_namespace": "AWS/EC2",
  "cloudwatch_metric_name": "CPUUtilization"
}

One EC2 instance:

{
  "dimension_instanceid": "i-0123456789abcdef0"
}

All configured matchers must match.

Example SNS notification#

The Message field contains a JSON-encoded CloudWatch alarm payload.

{
  "Type": "Notification",
  "MessageId": "sns-message-1",
  "TopicArn": "arn:aws:sns:eu-west-1:123456789012:incidentrelay-alerts",
  "Subject": "ALARM: HighCPU",
  "Message": "{\"AlarmName\":\"HighCPU\",\"AWSAccountId\":\"123456789012\",\"NewStateValue\":\"ALARM\",\"NewStateReason\":\"Threshold crossed\",\"Region\":\"EU (Ireland)\",\"AlarmArn\":\"arn:aws:cloudwatch:eu-west-1:123456789012:alarm:HighCPU\",\"OldStateValue\":\"OK\"}",
  "Timestamp": "2026-06-21T10:00:01.000Z",
  "SignatureVersion": "2",
  "Signature": "base64-signature",
  "SigningCertURL": "https://sns.eu-west-1.amazonaws.com/SimpleNotificationService-example.pem"
}

A manually created payload with a placeholder signature is rejected. The signature must be generated by Amazon SNS.

Composite alarms#

Composite alarm payloads can contain:

  • AlarmRule;
  • TriggeringChildren.

IncidentRelay stores these values in annotations and preserves the full CloudWatch payload. Deduplication still uses the alarm ARN.

Generic SNS notifications#

When Message is not recognized as a CloudWatch alarm, IncidentRelay creates a generic SNS alert using:

  • Subject as the title;
  • Message as the message;
  • MessageId as the external identifier and deduplication key;
  • string message attributes as labels;
  • warning as the default severity;
  • firing as the status.

Stored payload#

IncidentRelay stores:

payload.sns
payload.cloudwatch

The SNS signature is removed before the payload is stored. The rest of the SNS envelope and CloudWatch alarm body remain available for diagnostics and auditing.

Signature validation#

Before accepting a request, IncidentRelay checks:

  • the route exists;
  • the route source is aws_sns;
  • the route, team, and group are active;
  • the SNS envelope is valid;
  • optional SNS headers match the JSON body;
  • TopicArn exactly matches the route configuration;
  • SigningCertURL uses HTTPS;
  • the certificate host and path belong to Amazon SNS;
  • the certificate is currently valid;
  • the RSA signature matches the canonical SNS signing string.

Signature algorithms:

Signature versionDigest
1SHA-1
2SHA-256

Subscription responses#

Successful confirmation:

{
  "status": "confirmed",
  "message_id": "sns-confirmation-1",
  "topic_arn": "arn:aws:sns:eu-west-1:123456789012:incidentrelay-alerts"
}

Unsubscribe confirmation:

{
  "status": "unsubscribed",
  "message_id": "sns-confirmation-2",
  "topic_arn": "arn:aws:sns:eu-west-1:123456789012:incidentrelay-alerts"
}

Successful notification response#

[
  {
    "created": true,
    "alert_id": 123,
    "group_id": 45,
    "status": "firing",
    "team_id": 2,
    "team_slug": "sre",
    "route_id": 17,
    "routing_error": null,
    "trace_id": "..."
  }
]

HTTP responses#

StatusMeaning
200Notification processed or subscription state confirmed
202Processing accepted but not fully completed synchronously
207Mixed ingest outcomes
400Invalid envelope, headers, route source, or confirmation data
403Signature, Topic ARN, route, team, or group validation failed
404Route not found
502Signing certificate download or subscription confirmation failed

Troubleshooting#

Subscription remains pending#

Check that:

  • the IncidentRelay endpoint is publicly reachable through HTTPS;
  • the route source is aws_sns;
  • the route is active;
  • the configured Topic ARN exactly matches the SNS topic;
  • IncidentRelay can reach Amazon SNS certificate and confirmation URLs;
  • the reverse proxy forwards POST bodies unchanged.

Topic ARN mismatch#

Check the AWS partition, region, account ID, topic name, and optional .fifo suffix. The comparison is exact.

Signature verification failed#

Do not edit or manually replay a signed SNS body. Any change to a signed field invalidates the signature.

Also verify:

  • outbound HTTPS access from IncidentRelay;
  • system clock synchronization;
  • that the body is not rewritten by a proxy;
  • that the message came from the configured topic.

Alarm does not resolve#

Make sure CloudWatch sends a notification when the alarm enters OK. The ALARM and OK messages must contain the same AlarmArn.

Alert did not match a route#

Use the returned trace_id to inspect routing. Verify route state, team state, assignment target, and every configured matcher.

Multiple alarms are grouped together#

Use:

[
  "cloudwatch_alarm_arn"
]

as the route grouping configuration.

Security recommendations#

  • Use HTTPS only.
  • Use a dedicated SNS topic per environment or trust boundary.
  • Configure the exact Topic ARN on each route.
  • Do not disable certificate URL validation.
  • Do not accept redirects when loading certificates or confirming subscriptions.
  • Keep IncidentRelay system time synchronized.
  • Review routing traces and logs after rejected requests.