Skip to content

This repository demonstrates a scalable, segregated, secured AWS network hub for multi-account organizations using Terraform.

License

Notifications You must be signed in to change notification settings

aws-samples/aws-network-hub-for-terraform

Network Hub Account with Terraform

This repository demonstrates a scalable, segregated, secured AWS network for multi-account organizations. Using Transit Gateway to separate production, non-production and shared services traffic, it deploys an advanced AWS networking pattern using centralized ingress and egress behind Network Firewall, centralizes private VPC endpoints to share across all VPCs, and manages IP address allocation using Amazon VPC IPAM.

  • Perfect for a central networking hub account, potentially alongside Account Factory for Terraform
  • Solution itself can be deployed into nonprod, test and production deployments for safe iteration and testing.
  • Written using clean, composable modules: the solution is easily extended and customised.

Spoke VPCs for organization members can be created using the provided sister example in this repo.

The following resources will be deployed by this example:

  • VPC Transit Gateway
  • VPC Endpoints
  • AWS Network Firewall
  • Route 53 Resolver
  • Amazon VPC IP Address Manager

The resources deployed and the architectural pattern they follow are provided for demonstration and testing purposes but are based on and inspired by AWS best practice and articles.

Table of Contents

Overview

Diagrams

Solution Diagram diagram

Transit Gateway tgw

VPC Endpoints vpc_endpoints

Network Firewall nfw

Route 53 Resolver dns

References


Prerequisites

Minimal tooling is required for this solution. However, there are hard requirements around AWS configuration.

Tooling

  • Terraform ~> 1.1
    • AWS provider >= 4.4.0
  • AWS CLI
  • Git CLI

AWS account configuration

  • AWS Organization
  • Centralised network account
  • IAM role with required permissions
  • RAM sharing enabled for the Organisation
aws ram enable-sharing-with-aws-organization

ram

Troubleshooting tip

If you experience any issue with the RAM share disable and re-enable RAM.

aws organizations disable-aws-service-access --service-principal ram.amazonaws.com
  • IPAM delegated from the master account to the Centralised network account
aws ec2 enable-ipam-organization-admin-account \
    --delegated-admin-account-id <Network-Account-ID>

ipam

Customisation

If you do not define a remote backend Terraform will use the local directory to store the backend files including tfstate. Examples of how to customise the Terraform backend are included but commented out. Usual caveats around safe storage of Terraform state must be considered.

backend

Example GitLab HTTP backend for use with GitLab CI.

http_backend

Variables

Type Variable Name Description Notes
Global variables environment Environment to deploy into. Accepted values * dev, test, preprod, prod
aws_region Region to deploy in.
vpc_endpoints List of centralised VPC endpoints to be deployed
Environment specific variables ipam_cidr CIDR to be allocated to the IP Address Manager
tgw_route_tables Transit Gateway Router Tables to create
root_domain Root DNS domain to create private hosted zone and resolver rules

Input Variable - config.auto.tfvars

aws_region    = "eu-west-2"
vpc_endpoints = ["ec2", "rds", "sqs", "sns", "ssm", "logs", "ssmmessages", "ec2messages", "autoscaling", "ecs", "athena"]

env_config = {
  dev = {
    ipam_cidr        = "10.0.0.0/10"
    tgw_route_tables = ["prod", "dev", "shared"]
    root_domain      = "network-dev.internal"
  }
  test = {
    ipam_cidr        = "10.64.0.0/10"
    tgw_route_tables = ["prod", "dev", "shared"]
    root_domain      = "network-test.internal"
  }
  preprod = {
    ipam_cidr        = "10.128.0.0/10"
    tgw_route_tables = ["prod", "dev", "shared"]
    root_domain      = "network-preprod.internal"
  }
  prod = {
    ipam_cidr        = "10.192.0.0/10"
    tgw_route_tables = ["prod", "dev", "shared"]
    root_domain      = "network-prod.internal"
  }
}

Quick Start

Deploy from client machine

When deploying from your local machine having configured the TF Backend in the code you need to ensure you have access to read and write to the backend - possible backends include HTTP, Consul, Postgres, Artifactory, S3 or S3 + DynamoDB. We initialise the Terraform, complete the validate and format. Review the plan and then apply.

  • terraform init
  • terraform validate
  • set environment for deployment
    • export TF_VAR_environment="$ENV"
    • Set-Item -Path env:TF_VAR_environment -Value "$ENV" (Possible $ENV values - dev, test, preprod, prod)
  • terraform plan
  • terraform apply or terraform apply --auto-approve

Tagging

Tags are added to all AWS resources through use of the tag configuration of the AWS Provider.

As not all AWS resources support default tags passed from the provider (EC2 Auto-Scaling Group + Launch Template) We pass the tags as a variable (Map(string) - these are defined in the root locals.tf file.

provider

Example Tags - locals.tf

tags = {
  Product    = "Network_Automation"
  Owner      = "GitHub"
  Project_ID = "12345"
}

Clean Up

Remember to clean up after your work is complete. You can do that by doing terraform destroy.

Note that this command will delete all the resources previously created by Terraform.

Terraform Docs

Terraform Deployment

Requirements

Name Version
terraform ~> 1.1
aws >= 4.4.0

Providers

Name Version
aws 4.5.0

Modules

Name Source Version
dns ./modules/dns n/a
ipam ./modules/ipam n/a
network_firewall_vpc ./modules/network_firewall_vpc n/a
tgw ./modules/tgw n/a
vpc_endpoints ./modules/vpc_endpoints n/a

Resources

Name Type
aws_iam_policy.central_network resource
aws_iam_policy_attachment.central_network resource
aws_iam_role.central_network resource
aws_iam_role.flow_logs resource
aws_iam_role_policy.flow_logs resource
aws_kms_key.log_key resource
aws_availability_zones.available data source
aws_caller_identity.current data source
aws_iam_policy_document.policy_kms_logs_document data source
aws_organizations_organization.main data source

Inputs

Name Description Type Default Required
aws_region AWS region being deployed to string n/a yes
env_config Map of objects for per environment configuration
map(object({
ipam_cidr = string
tgw_route_tables = list(string)
root_domain = string
}))
n/a yes
environment Deployment environment passed as argument or environment variable string n/a yes
tags Default tags to apply to all resources map(string) n/a yes
vpc_endpoints Which VPC endpoints to use list(string) n/a yes

Outputs

No outputs.

TGW Module

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_ec2_transit_gateway.org_tgw resource
aws_ec2_transit_gateway_route.blackhole_route resource
aws_ec2_transit_gateway_route.default_route resource
aws_ec2_transit_gateway_route.default_route_ipv6 resource
aws_ec2_transit_gateway_route_table.org_tgw resource
aws_ram_principal_association.org resource
aws_ram_resource_association.tgw resource
aws_ram_resource_share.main resource

Inputs

Name Description Type Default Required
az_names A list of the Availability Zone names available to the account list(string) n/a yes
cidr Corporate CIDR range for use with blackholing traffic between production and development environments string n/a yes
environment Deployment environment passed as argument or environment variable string n/a yes
inspection_attachment Inspection VPC attachment for default route string n/a yes
org_arn The ARN of the AWS Organization this account belongs to string n/a yes
tgw_route_tables List of route tables to create for the transit gateway list(string) n/a yes

Outputs

Name Description
tgw TGW ID for VPC attachments
tgw_route_table Map of route tables used for association and propagation

IPAM Module

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_ram_principal_association.org resource
aws_ram_resource_association.ipam resource
aws_ram_resource_share.main resource
aws_ssm_parameter.ipam_pool_id resource
aws_vpc_ipam.org_ipam resource
aws_vpc_ipam_pool.private_org_ipam_pool resource
aws_vpc_ipam_pool_cidr.private_org_ipam_pool resource
aws_vpc_ipam_scope.private_org_ipam_scope resource

Inputs

Name Description Type Default Required
aws_region AWS region being deployed to string n/a yes
ipam_cidr CIDR block assigned to IPAM pool string n/a yes
org_arn The ARN of the AWS Organization this account belongs to string n/a yes

Outputs

Name Description
org_ipam Org IPAM ID
org_ipam_pool Org IPAM pool ID

VPC Endpoint Module

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_cloudwatch_log_group.flow_logs resource
aws_default_security_group.default resource
aws_ec2_transit_gateway_route_table_association.shared resource
aws_ec2_transit_gateway_route_table_propagation.org resource
aws_ec2_transit_gateway_vpc_attachment.vpc_endpoint resource
aws_flow_log.vpc resource
aws_route.default_route resource
aws_route.default_route_ipv6 resource
aws_route53_record.dev_ns resource
aws_route53_zone.interface_phz resource
aws_route_table.endpoint_vpc resource
aws_route_table_association.attachment_subnet resource
aws_route_table_association.endpoint_subnet resource
aws_security_group.allow_vpc_endpoint resource
aws_security_group_rule.org_cidr resource
aws_subnet.attachment_subnet resource
aws_subnet.endpoint_subnet resource
aws_vpc.endpoint_vpc resource
aws_vpc_dhcp_options.endpoint_vpc resource
aws_vpc_dhcp_options_association.endpoint_vpc resource
aws_vpc_endpoint.interface resource

Inputs

Name Description Type Default Required
az_names A list of the Availability Zone names available to the account list(string) n/a yes
cidr Corporate CIDR range for use with blackholing traffic between production and development environments string n/a yes
environment Deployment environment passed as argument or environment variable string n/a yes
iam_role_arn IAM role to allow VPC Flow Logs to write to CloudWatch string n/a yes
interface_endpoints Object representing the region and services to create interface endpoints for map(string) n/a yes
kms_key_id VPC Flow Logs KMS key to encrypt logs string n/a yes
org_ipam_pool IPAM pool ID to allocate CIDR space string n/a yes
tgw TGW ID for VPC attachments string n/a yes
tgw_route_tables TGW route tables for VPC association and propagation map(string) n/a yes

Outputs

No outputs.

DNS Module

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_cloudwatch_log_group.flow_logs resource
aws_default_security_group.default resource
aws_ec2_transit_gateway_route_table_association.shared resource
aws_ec2_transit_gateway_route_table_propagation.org resource
aws_ec2_transit_gateway_vpc_attachment.vpc_dns resource
aws_flow_log.vpc resource
aws_ram_principal_association.org resource
aws_ram_resource_association.r53r resource
aws_ram_resource_share.main resource
aws_route.default_route resource
aws_route.default_route_ipv6 resource
aws_route53_resolver_endpoint.inbound resource
aws_route53_resolver_endpoint.outbound resource
aws_route53_resolver_rule.fwd resource
aws_route53_resolver_rule_association.org_dns resource
aws_route53_zone.root_private resource
aws_route_table.dns_vpc resource
aws_route_table_association.attachment resource
aws_route_table_association.privatesubnet resource
aws_security_group.allow_dns resource
aws_security_group_rule.dns_tcp resource
aws_subnet.attachment_subnet resource
aws_subnet.endpoint_subnet resource
aws_vpc.dns_vpc resource
aws_vpc_dhcp_options.dns_vpc resource
aws_vpc_dhcp_options_association.dns_vpc resource

Inputs

Name Description Type Default Required
az_names A list of the Availability Zone names available to the account list(string) n/a yes
cidr Corporate CIDR range for use with blackholing traffic between production and development environments string n/a yes
environment Deployment environment passed as argument or environment variable string n/a yes
iam_role_arn IAM role to allow VPC Flow Logs to write to CloudWatch string n/a yes
interface_endpoints Object representing the region and services to create interface endpoints for map(string) n/a yes
kms_key_id VPC Flow Logs KMS key to encrypt logs string n/a yes
org_arn The ARN of the AWS Organization this account belongs to string n/a yes
org_ipam_pool IPAM pool ID to allocate CIDR space string n/a yes
root_domain Root domain for private hosted zone delegation string n/a yes
tgw TGW ID for VPC attachments string n/a yes
tgw_route_tables TGW route tables for VPC association and propagation map(string) n/a yes

Outputs

No outputs.

Network Firewall Module

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_cloudwatch_log_group.flow_logs resource
aws_cloudwatch_log_group.network_firewall_alert_log_group resource
aws_cloudwatch_log_group.network_firewall_flow_log_group resource
aws_default_security_group.default resource
aws_ec2_transit_gateway_route_table_association.shared resource
aws_ec2_transit_gateway_route_table_propagation.org resource
aws_ec2_transit_gateway_vpc_attachment.vpc_inspection resource
aws_egress_only_internet_gateway.eigw resource
aws_eip.internet_vpc_nat resource
aws_flow_log.vpc resource
aws_internet_gateway.igw resource
aws_nat_gateway.internet resource
aws_networkfirewall_firewall.inspection_vpc_network_firewall resource
aws_networkfirewall_firewall_policy.anfw_policy resource
aws_networkfirewall_logging_configuration.network_firewall_alert_logging_configuration resource
aws_networkfirewall_rule_group.block_domains resource
aws_route.default_route resource
aws_route.default_route_ipv6 resource
aws_route.egress_route resource
aws_route.egress_route_ipv6 resource
aws_route.ingress_route resource
aws_route.inspection_route resource
aws_route.inspection_route_ipv6 resource
aws_route.inspection_route_natgw_ipv6 resource
aws_route.internal_route resource
aws_route_table.attachment resource
aws_route_table.inspection resource
aws_route_table.internet resource
aws_route_table_association.attachment_subnet_rt_association resource
aws_route_table_association.inspection_subnet resource
aws_route_table_association.internet_subnet resource
aws_subnet.attachment_subnet resource
aws_subnet.inspection_subnet resource
aws_subnet.internet_subnet resource
aws_vpc.inspection_vpc resource
aws_vpc_dhcp_options.inspection_vpc resource
aws_vpc_dhcp_options_association.inspection_vpc resource

Inputs

Name Description Type Default Required
aws_region AWS region being deployed to string n/a yes
az_names A list of the Availability Zone names available to the account list(string) n/a yes
cidr Corporate CIDR range for use with blackholing traffic between production and development environments string n/a yes
environment Deployment environment passed as argument or environment variable string n/a yes
iam_role_arn IAM role to allow VPC Flow Logs to write to CloudWatch string n/a yes
kms_key_id VPC Flow Logs KMS key to encrypt logs string n/a yes
org_ipam_pool IPAM pool ID to allocate CIDR space string n/a yes
tgw TGW ID for VPC attachments string n/a yes
tgw_route_tables TGW route tables for VPC association and propagation map(string) n/a yes

Outputs

Name Description
eni_map Output ENI map
firewall_info Info of network firewall for routing
inspection_attachment Inspection TGW attachment ID for default route in TGW
route_table Output route tables used for NFW
rt_map Output RT map

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.