Recently I ran into a few circular dependency problems when creating resources with AWS CloudFormation. While creating the stack AWS will throw a “Circular dependency between resources: [ ResourceA, Resource B, …]”.

The error message is not very specific on where exactly this happened. The message can even set you on the wrong track while solving the problem.

Whats going on is that CloudFormation is trying to determine the order in which to create the resources based on the dependencies between those. ResourceA might depend on ResourceB while at the same time ResourceB depends on ResourceA. Of course this circle might also consist of more than two resources or it might even be one resource that depends on itself.

Dependencies can be explicit (defined by the DependsOn Attribute) or implicit (for example if ResourceA uses a !Ref to ResourceB). But I’ve only run into hard to debug circular dependencies with implicit dependencies.

Common scenarios with tightly coupled resources are different EC2 Hosts that want to reference each other (for example HostA needs to know HostB’sĀ Elastic IP and vice versa) and Security Groups that reference itself (for example to define a rule that allows all hosts in a Security Group to access each other).

Luckily, in most situations you can work around the problem by extracting the component that is causing the circular dependency.

So instead of defining a SecurityGroupIngress inside of the SecurityGroup:

MySecurityGroup:
  Type: AWS::EC2::SecurityGroup
  Properties:
    GroupDescription: Allow all hosts in this group to access each other on port 80
    GroupName: My Security Group
    VpcId: ...
    SecurityGroupIngress:
    - IpProtocol: TCP
      FromPort: 80
      ToPort: 80
      # this will cause a circular dependency
      SourceSecurityGroupId: !Ref MySecurityGroup

you can define it as separate SecurityGroupIngress resource:

MySecurityGroup:
  Type: AWS::EC2::SecurityGroup
  Properties:
    GroupDescription: Allow all hosts in this group to access each other on port 80
    GroupName: My Security Group
    VpcId: ...

MySecurityGroupIngress:
  Type: AWS::EC2::SecurityGroupIngress
  Properties:
    GroupId: !Ref MySecurityGroup
    IpProtocol: TCP
    FromPort: 80
    ToPort: 80
    SourceSecurityGroupId: !Ref MySecurityGroup

This works with other resources as well.

So for example for referencing the Elastic IPs of different hosts to each other you cloud extract the network interfaces and then reference those:

InterfaceHostA:
  Type: AWS::EC2::NetworkInterface
  Properties:
    SubnetId:
      Ref: MySubnet
      ...
InterfaceHostB:
  ...

HostA:
  Type: AWS::EC2::Instance
  Properties:
    ...
    NetworkInterfaces:
    - NetworkInterfaceId: !Ref InterfaceHostA
      DeviceIndex: 0
    UserData:
      Fn::Base64:
      - |
        #!/bin/bash -x
        echo "Host B IP is: ${HostBip}"
      - HostBip: !GetAtt [ InterfaceHostB, PrimaryPrivateIpAddress ]
HostB:
  ...
  Properties:
    NetworkInterfaces:
    - NetworkInterfaceId: !Ref InterfaceHostB
      DeviceIndex: 0
    UserData:
      Fn::Base64:
      - |
        #!/bin/bash -x
        echo "Host A IP is: ${HostAip}"
      - HostAip: !GetAtt [ InterfaceHostA, PrimaryPrivateIpAddress ]
Avoiding Circular Dependency Problems in AWS CloudFormation
Tagged on: