[AWS] A duplicate Security Group rule was found – HashiCorp Help Center

Introduction

Problem

Consider the following code to create a security group rule in AWS with multiple cidr blocks.

resource "aws_security_group_rule" "example" {
    description = "Allow LDAP"
    from_port = 636
    protocol = "tcp"
    security_group_id = aws_security_group.example.id
    to_port = 636
    cidr_blocks = ["18.0.0.0/24","19.0.0.0/24", "17.0.0.0/24"]
    type = "ingress"
}

A second apply suddenly fails with the following error:

│ Error: [WARN] A duplicate Security Group rule was found on (sg-0b7c95ffefd890db1). This may be
│ a side effect of a now-fixed Terraform issue causing two security groups with
│ identical attributes but different source_security_group_ids to overwrite each
│ other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
│ information and instructions for recovery. Error: InvalidPermission.Duplicate: the specified rule "peer: 19.0.0.0/24, TCP, from port: 636, to port: 636, ALLOW" already exists
│ status code: 400, request id: 6aadf5a9-4e06-46f1-9828-a3f881b60efe
│ 
│ with aws_security_group_rule.example,
│ on main.tf line 25, in resource "aws_security_group_rule" "example":
│ 25: resource "aws_security_group_rule" "example" {

Cause

The security group rules were probably changed manually in the AWS Console. For example, the rule with cidr block "17.0.0.0/24" was removed manually.

Terraform sees that a rule is missing and wants to create it again. However, because all the cidr blocks are defined in one resource, terraform decides to delete the entire resource and recreate all the rules.

It will now want to recreate the rule with cidr block "19.0.0.0/24", but this rule still exists in AWS and a duplicate error is generated, as can be seen in the error shown above.

(A bug report has been filed for this issue.)

Overview of possible solutions (if applicable)

Solutions:

Remove the security group from AWS Console and run terraform apply
Remove all the duplicate rules from AWS Console and run terraform apply
Import the security group rules back in to your configuration

Example import:

- On the command line

terraform import 'aws_security_group_rule.example' 'sg-0e90ef2bf8c599bd9_ingress_tcp_636_636_19.0.0.0/24_18.0.0.0/24'

The id string is build up as follows:

The id from the security group
The type of rule, here ingress
The protocol, here tcp
The from_port, here 636
The to_port, here 636
The string of cidr blocks, separated by an _

- With an import block

import {
    to = aws_security_group_rule.example
    id = "sg-0e90ef2bf8c599bd9_ingress_tcp_636_636_19.0.0.0/24_18.0.0.0/24"
}

Preferred Solution:

To prevent similar problems in the future, it is best to start using a for_each meta-argument to create each rule, stored with it's own key. This way every aws_security_group_ruleis it's own resource instead of all stored in one.

Replace your code with the following

resource "aws_security_group_rule" "example" {
    for_each = toset(["18.0.0.0/24", "19.0.0.0/24"])
    description = "Allow LDAP"
    from_port = 636
    protocol = "tcp"
    security_group_id = aws_security_group.example.id
    to_port = 636
    cidr_blocks = [each.value]
    type = "ingress"
}

Import the security rules back in to your configuration

Example import:

On the command line

terraform import 'aws_security_group_rule.example["19.0.0.0/24"]' 'sg-0fa4e55b76036112d_ingress_tcp_636_636_19.0.0.0/24'
terraform import 'aws_security_group_rule.example["18.0.0.0/24"]' 'sg-0fa4e55b76036112d_ingress_tcp_636_636_18.0.0.0/24'

The id string is build up as follows:

The id from the security group
The type of rule, here ingress
The protocol, here tcp
The from_port, here 636
The to_port, here 636
One cidr block, here 19.0.0.0/24

With an import block

import {
    for_each = ["18.0.0.0/24", "19.0.0.0/24"]
    to = aws_security_group_rule.example[each.value]
    id = "sg-0fa4e55b76036112d_ingress_tcp_636_636_${each.value}"
}

Make sure that the values in the for_each of the import block are also present in the for_each of your aws_security_group_rule resource.

Outcome

Able to run a terraform apply again with multiple cidr blocks defined in your aws_security_group_rule resource.

It would even be better to use the aws_vpc_security_group_ingress_rule for your ingress rules. Likewise there is the aws_vpc_security_group_egress_rule for egress rules. Please see the documentation linked in additional information below.

Additional Information