ANS Exercise 1.1: Subnetting and Routing with IPv6

December 13, 2019

ANS Exercise 1.1: Subnetting and Routing with IPv6

In this exercise we will build a VPC with IPv6 enabled and explore the differences between routing in IPv4 and IPv6.

This is exercise 1.1 for the AWS Advanced Networking Specialty training. For an explanation and overview of all exercises, see the overview post.

The exercises are built on the assumption that you’re already familiar with the AWS basics and have achieved at least one associate level AWS certification.

Create a VPC

Open the AWS Web Console, navigate to the VPC section, and click the Create VPC button.

Create VPC

Fill in the fields and make sure you select “Amazon provided IPv6 CIDR block”. Here you will see the first differences between IPv4 and IPv6:

  • You’re free to choose the IPv4 range, but AWS decides which IPv6 range you will be using.
  • You can set the IPv4 netmask (/16 in our example), but for IPv6 it will always be a /56.

The limit of choice has to do with the nature of IPv4 vs. IPv6. The IPv4 range is most likely an RFC 1918 range, meaning it consists of private IP addresses that can be used in many private networks. Because these addresses are private, i.e. invisible to the outside world, anyone can use them without risk of address collisions. IPv6 also has a concept of private addresses, called “Unique Local IPv6 Unicast Addresses”, defined in RFC 4193. However, because the amount of addresses available in IPv6 is so massive, in general every device gets its own public IPv6 address, and every AWS subnet gets its own publicly routable /56 range. These subnets are provided from AWS’ own pool of IPv6 addresses, and that’s why you don’t get to freely choose a range.

create-vpc-details

Create a public subnet

When the VPC has been created, move to the Subnets tab and click the “Create subnet” button.

create-subnet

On the next screen, fill in some details and select Custom IPv6 in the last field.

create-subnet-public-1a

As you can see, we don’t get to choose the IPv6 prefix or netmask here either; it’s set to 2a05:d018:0810:54 <xx> ::/64. This means every subnet will have a /64 range, and we can have a maximum of 2^(64-56) = 256 IPv6 enabled subnets in a VPC.

Connecting to the internet

In the previous step we’ve created a public subnet that should be able to connect directly to the internet. However, before a resource in the subnet could actually connect to the outside world, we need an Internet Gateway.

In the VPC console, click the Internet Gateways tab, followed by the “Create internet gateway” button.

create-internet-gateway

Give the IGW a name and click Create. Next, attach it to our VPC.

attach-internet-gateway

attach-internet-gateway-2

Then move to the Route Tables tab and add both IPv4 and IPv6 routes to the IGW.

route-tables

When this is complete, we have achieved the following setup:

overview-1

Adding an IPv4 routed instance

Let’s add an instance to this subnet. Go to the EC2 console and click the “Launch instance” button.

launch-instance

Select the Amazon Linux 2 AMI and a cheap instance (eg. t2.micro). In Step 3: Configure Instance Details, enable a public IPv4 address. Keep the IPv6 address disabled for now.

launch-instance-details

Click next and skip storage and tags (unless you want to change them), then create a new security group that allows port 22 (SSH) for your local IPv4 address.

security-group

Review your settings, launch the instance and select an SSH key pair. Then view the instance in the EC2 console, and as expected you will see that it has a public IPv4 address.

ec2-public-address

In the background, what has actually happened is that the internet gateway has attached this public IP address (52.212.75.150), and that the router will forward any traffic to the private IP address (10.0.0.28). Or in other words, the router performs one-on-one NAT between the public address and the instance (see the docs). The result is that any inbound traffic to the public IP address will be forwarded to the instance, and any outbound traffic from the instance will be translated as originating from the public IP address. This is displayed in the following diagram.

overview-2

You should open an SSH connection to this instance to verify it works. On the instance, run ip addr, and you should get output like this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 0a:8c:da:81:98:5c brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.28/24 brd 10.0.0.255 scope global dynamic eth0
       valid_lft 3590sec preferred_lft 3590sec
    inet6 fe80::88c:daff:fe81:985c/64 scope link 
       valid_lft forever preferred_lft forever

As you can see, there is no mention of the public IPv4 address, which confirms that NAT is taking place. When you’re done, feel free to terminate the instance.

Adding an IPv6 routed instance

Next up, let’s add an instance that has no public IPv4 address, but instead uses a public IPv6 address.

Create a new instance, but when you get to Step 3: Configure Instance Details, enable “Auto-assign IPv6 IP”.

launch-instance-details-ipv6

When you get to the security group, open port 22 to your IPv6 address (or ::0 if you don’t know your address).

security-group-ipv6

Continue to launch the instance with an SSH key pair, as you have before, then verify the instance in the EC2 console. You will see that the instance still has a private IPv4 address (they always do), but now an IPv6 address is displayed as well.

ec2-public-address-ipv6

If everything has been configured correctly, you can now SSH to the instance with ssh ec2-user@2a05:d018:810:5400:3f50:acbc:e3da:6912.

When you run ip addr now, you should get output like this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 0a:02:d8:5d:c0:76 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.80/24 brd 10.0.0.255 scope global dynamic eth0
       valid_lft 3369sec preferred_lft 3369sec
    inet6 2a05:d018:810:5400:3f50:acbc:e3da:6912/128 scope global dynamic 
       valid_lft 447sec preferred_lft 147sec
    inet6 fe80::802:d8ff:fe5d:c076/64 scope link 
       valid_lft forever preferred_lft forever

As you can see, this does mention the IPv6 address. From this, we can conclude that the public IPv6 address is assigned to the instance itself, and no NAT is taking place. To view that in a diagram:

overview-3

In other words, 2a05:d018:810:5400::/56 is the publicly routable CIDR for the entire VPC, 2a05:d018:810:5400::/64 is the publicly routable CIDR for the public subnet, and 2a05:d018:810:5400:3f50:acbc:e3da:6912/128 is the publicly routable address for the instance itself.

Leave this instance running for now, we will use it as a bastion in the next section.

Private IPv4 servers

A proper cloud infrastructure ‘hides’ instances that should not be publicly accessible in private subnets. A common example is to put web servers behind a load balancer, where the load balancer is publicly accessible, but the web servers can only be accessed by the load balancer. This might look like this:

overview-4

In this example, the instance can only be reached by the load balancer, and the instance has no way to initiate a connection to the internet. If the instance needs an internet connection, we need to add a NAT Gateway:

overview-5

The function of the NAT Gateway is to accept traffic from the private instances and route it to the internet. While doing so, it will apply its own public IP address as the ‘source address’ of the traffic. When a response to an outbound request comes back, it will route the traffic back to the originating instance. As such, from the perspective of the outside internet any traffic (from any amount of instances) routed through the NAT Gateway will seem to originate from a single source; the NAT Gateway’s public address.

But as we’ve learned earlier in this post, all IPv6 addresses are public. So how do we hide our IPv6 servers from the outside world? We’ll discuss that in the next section.

Private IPv6 servers

Before we look at how to set up private IPv6 servers, we should mention that currently there aren’t many arguments to implement non-public servers running IPv6. A more common scenario is to set up IPv6 at the frontend (eg. at the load balancers), and do all internal routing on IPv4. Only when you’re looking at an environment where IPv4 is completely unsupported does the scenario below make real sense.

Start by creating a new route table. We will use this route table for private routing. Navigate to the VPC Console and click the Route Tables tab. Then click the “Create route table” button.

create-route-table

create-route-table-details
Then create a new private subnet. Click the Subnets tab and fill in the following details:

create-private-subnet

As you can see, we’re still enabling the IPv6 CIDR block, and we’re assigning the 2a05:d018:810:5401::/64 range.

When the subnet has been created, select the subnet and click the “Route Table” tab in the bottom.

subnet-route-table

When we created the subnet, it automatically associated itself with the default route table. We want to change that into the private route table, so click the “Edit route table association” button and select the route table we created a few steps back.

edit-route-table-association

When you have saved the settings, navigate to the Egress Only Internet Gateways tab and click the “Create Egress Only Internet Gateway” button.

create-egress-only-internet-gateway

Select your VPC and click next. Then navigate back to the Route Tables tab, select your private route table and select “Edit routes” from the Actions menu.

edit-routes

Then add a route for ::/0 to the newly created egress only gateway.

add-egress-only-route

Save the route table. Then let’s add an IPv6 enabled instance to the new subnet. Navigate back to the EC2 console and create a new instance. In Step 3: Configure Instance Details, make sure to select the private subnet and enable IPv6.

ec2-private-subnet-ipv6

When you get to Step 6: Configure Security Group, make sure that you allow SSH access to the world (::/0). We will use this setting to prove that even though the SG is open, the instance is still private.

private-ipv6-security-group

Continue through the next steps and launch the instance. When the instance has booted, it should show IP addresses like these:

private-ipv6-addresses

The IPv6 address is in the 2a05:d018:810:5401::/64 range, the IPv4 address in the 10.0.1.0/24 range. Because we have no NAT gateway, this instance cannot make outbound connections to the IPv4 internet, but let’s see what we can do over IPv6.

First, let’s try to SSH to the machine directly. Running ssh ec2-user@2a05:d018:810:5401:7bc4:b425:f531:acc4 doesn’t seem to be doing anything, so even though the security group is open, we can’t connect. This is the Egress Only Internet Gateway blocking all inbound traffic.

Next, let’s connect to the (Bastion) instance we created in the previous chapter by running ssh ec2-user@2a05:d018:810:5400:3f50:acbc:e3da:6912, and when we’re logged in, connect to the second instance with ssh ec2-user@2a05:d018:810:5401:7bc4:b425:f531:acc4. You need to configure SSH Agent forwarding for this to work.

When the connection is successful, you’re on the private server!

[ec2-user@ip-10-0-0-80 ~]$ ssh ec2-user@2a05:d018:810:5401:7bc4:b425:f531:acc4


       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
5 package(s) needed for security, out of 13 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-10-0-1-146 ~]$ 

To verify that the server is not able to connect to IPv4 internet, run curl http://ipv4.google.com. This server is only reachable over IPv4 internet.

Next, run curl http://www.google.com -I -v. This server is accessible over both IPv4 and IPv6. The top of the response should look like this:

* Rebuilt URL to: http://www.google.com/
*   Trying 2a00:1450:400b:c01::6a...
* TCP_NODELAY set
* Connected to www.google.com (2a00:1450:400b:c01::6a) port 80 (#0)

And somewhere in the middle of the response you should see HTTP/1.1 200 OK, which means the connection was successful.

Conclusion

In this exercise we’ve built a network infrastructure with IPv4 and IPv6. We’ve seen that IPv4 uses a lot of NAT, both on the inbound and the outbound side. IPv6 doesn’t use NAT at all, but you need to use a special construct called an Egress Only Internet Gateway to secure your private servers.

Luc van Donkersgoed

Luc van Donkersgoed