Scenario
Say you’re performing an operation using Terraform and you run into an AWS permission-related error e.g:
Error: error listing tags for Route53 Resolver rule (arn:aws:route53resolver:eu-west-1:<acctID>:resolver-rule/rslvr-rr-xxxx):
InvalidParameterException: [RSLVR-00907] You are not authorized to do this operation on shared resource
{
RespMetadata: {
StatusCode: 400,
RequestID: "xxxx"
},
Message_: "[RSLVR-00907] You are not authorized to do this operation on shared resource"
}
How do you troubleshoot it?
Since we have the StatusCode
and affected AWS service displayed the error, the first thing you can do is to check for error code on the AWS website.
Googling for “route53 resolver rule error code 400” as an example will lead you to the Common Errors section of the Amazon Route53 API reference documentation - where we see that the status code 400 means “You do not have sufficient access to perform this action.”
The good thing is that we know the exact resolver rule affected arn:aws:route53resolver:eu-west-1:<acctID>:resolver-rule/rslvr-rr-xxxx
, and that we are not authorized to perform the operation. But what particular part of our access model is the problem?
This is where terraform debugging comes into play. For that, we’re going to use the most-detailed logging level available to us - the trace log. The trace log shows the API operations behind your Terraform command. To learn how to gather a trace log for the terraform run that produced the error, please see this guide here.
In the trace log, search using the affected resource ARN arn:aws:route53resolver:eu-west-1:<acctID>:resolver-rule/rslvr-rr-xxxx
to see the exact API call that caused the error.
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: route53resolver.eu-west-1.amazonaws.com
User-Agent: aws-sdk-go/1.36.19 (go1.15.5; windows; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.13.6 (+https://www.terraform.io)
Content-Length: 105
Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20210120/eu-west-1/route53resolver/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-target, Signature=xxxx
Content-Type: application/x-amz-json-1.1
X-Amz-Date: 20210120T112258Z
X-Amz-Security-Token: xxxx
X-Amz-Target: Route53Resolver.ListTagsForResource
Accept-Encoding: gzip
{"ResourceArn":"arn:aws:route53resolver:eu-west-1:<acctID>:resolver-rule/rslvr-rr-xxxx"}
Here you can see that the exact API call is the Route53Resolver.ListTagsForResource
- which is the permission we didn’t have/was denied.
To fix, check the IAM permissions for your AWS credential and verify that you have the right scope of access for the action Route53Resolver.ListTagsForResource
and resource arn:aws:route53resolver:eu-west-1:<acctID>:resolver-rule/rslvr-rr-xxxx
you’re applying changes to.
Additional Reading/References:
- Managing IAM Permissions: https://aws.amazon.com/iam/features/manage-permissions/
- Common AWS API Errors: https://docs.aws.amazon.com/awssupport/latest/APIReference/CommonErrors.html