AWS Basics- Bastian Hosts and NATS

Introduction

In previous post on AWS Elastic Compute Cloud (EC2) Basics, we launched two EC2 instances. One in public subnet and one is private subnet. With security groups configured, we were able to SSH to EC2 in public subnet.

In this post, we will continue and setup Bastian Host and NAT instance in our VPC. We will learn why we need those and some of the options available to us.

Here is how our architecture is currently setup, for the reference:

Now, if we want to SSH into EC2 instance on private subnet from our home/office (or using development machine), currently we can’t. Our instance has no public IP, it is in Private Subnet (no direct route from internet). This is where we can use a Bastian Server.

Bastian Host

Bastion servers are instances that sit within your public subnet and are typically accessed using SSH or RDP. The purpose of bastian host is to restrict the access to a private network from an external network.

Once remote connectivity has been established with the bastion host, it then acts as a ‘jumpserver’ , allowing you to use SSH or RDP to login to other instances (within private subnets) deeper within your network.

In this post, we will setup a Bastian server in our public subnet (so internet traffic is possible).

First we will connect to this Bastian server, from there, we can then connect to private EC2 instance (remember the rule in security group, we configured, in previous post, we allowed traffic from within our VPC to port 22 on the private EC2 instance).

Setting Up a Bastian Host

It’s agatha all along. We already have a bastian host.

Let me explain this, in the previous post, we launched an EC2 instance in public subnet and we already have configured an Internet Gateway for VPC and configured route in route table to allow incoming traffic. Later, we launched an ubuntu EC2 instance in private subnet and configured its security group to allow incoming traffic on port SSH from public subnet and that is all what was needed on connection level. I just did not mention that in previous post, to not get us distracted from those details. So our current EC2 instance in public subnet can be used as a Bastian Host.

The idea is that we SSH to the Bastian instance using its Elastic IP (or public IP). Then from the Bastian instance, SSH into private instance using its Private IP.

SSH into Bastian Host to Private EC2 Instance

Ok, let’s try this. First we need to SSH into EC2 public instance (we have seen previously, how to do that, please check that post if you need more info.).

Once we are in EC2 machine (bastian-host), from there, we can try to SSH into the EC2 instance on private subnet, run the following SSH command:

Notice, that we still can not SSH into the private instance. The reason for that, is that we also need key-pair file for connection, which is right now missing on the EC2 instance (Bastian Host) on our public subnet.

So, one thing we can do is to copy the keypair file on bastian host using the following command:

scp -i ./fm-keypair.pem fm-keypair.pem ubuntu@3.127.100.99:/home/mykeys

we can check that keypair file is now copied to EC2:

Now, if we try to SSH using this key file:

ssh -i ./fm-keypair.pem ubuntu@10.0.2.8

This may result in the following error message, ok it is trying, but still can’t SSH.

To fix this, we need to set permission on the key-pair file using chmod as shown below:

chmod 400 ./fm-keypair.pem

This command will change the permission of that file so it is only readable by us.

Ok, let’s try again to SSH to private ec2 instance and this time we are in (notice the private IP changes in the prompt):

Ok, so, Bastian Host, allowed us to connect to EC2 instance in private subnet. We can now use it to jump to EC2 instances in the private subnet.

Outbound Internet Access from Private EC2 instance

Ok, we are able to connect to private EC2 from internet via Bastian host. Incoming traffic is working as expected.

Now, I would like to install some software e.g. PostgreSQL on this private EC2 instance from internet (your may not have this requirement), meaning I want to access the internet from this instance. Can we do that? Let’s try this next:

I tried to curl google.com from private EC2 instance(you can try may be apt-get update command instead):

and as you can see that it just hangs there and nothing happens.

This is expected and is a good thing. If you remembered from previous posts, the whole point of private subnet is that it shall not be connected to internet directly (Inbound/Outbound) and this is what we are experiencing here.

Now, without internet, we can’t do much on this instance. For example, I can not download the package from internet. So we need a way to allow outbound internet traffic from this private EC2 instance.

Bastian Host job was to provide us a mean to SSH into private instance and we are able to do that. Now, if we want to go out to internet from private EC2 instance, we can use a NAT instance or NAT Gateway to give access to the internet without allowing inbound access to it.

What is a NAT Instance

A NAT (Network Address Translation) instance is, like a bastion host, an instance that lives in your public subnet.

A NAT instance, however, allows your private instances outgoing connectivity to the Internet, while at the same time blocking inbound traffic from the Internet.

Main reason to configure NAT instances is to allow private instances to access the Internet for important operating system updates, It is used for purposes like patching your OS etc.

Setting Up a NAT Instance

It is actually very simple to setup a NAT instance.

First, we will launch an EC2 instance in our public subnet. We also need to allocate an Elastic IP address and assign it to NAT instance as well. This will allow that our instance is be able to reach the internet. We will be using a special amazon image instead of ubuntu for our NAT instance.

Second, in order for our private EC2 instance to use the NAT instance to get to the internet, we need to configure a route in the Route table associated with our private subnet, pointing to the NAT instance as target.

Third, we need to disable IP source/destination check on the NAT instance.

So Traffic flow will be from: private EC2 –> Route Table –> NAT instance –>Internet Gateway –> Internet.

Question:

Can we use our Bastian-host as a NAT instance as well?

Answer:

Yes, that is totally fine. However, NAT instance also require some IP table configurations to allow this behavior and currently I am using ubuntu for Bastian Server and it will require me to run some bash commands on it to setup the IP table entries and I really don’t wanna do this at this time.

Another easy option is that there are some pre-configured amazon images you can use as a base for NAT instance and I am going to use this option for NAT instance (Later may be I will use this instance as a Bastian host and remove the ubuntu EC2 from the public subnet all together). For now, let’s keep things simple and use this pre-configured Amazon AMI based EC2 instance.

Launch the EC2 instance (NAT instance)

So, launch the EC2 instance wizard from the web console. Use the following image for NAT instance (you can use ami id in the search field to search for this instance):

This image has an empty alt attribute; its file name is image-105.png

select the right VPC and choose Public Subnet for our NAT instance (like our bastian-host) and launch the instance. If you like you can try to SSH into this instance:

This image has an empty alt attribute; its file name is image-107.png

So our instance is running and all the needed IP table stuff is already done for us.

Configure the Route for Private Subnet Route Table

Next, go to the route table for the Private subnet and add one more route (0.0.0.0/0) and for target select the NAT Instance eni we just created. This way we are configuring a catch-all route in our private subnet and sending traffic to NAT instance which is in public subnet:

So, any time private EC2 instance make an internet request, it will go to ENI of NAT instance and NAT instance being in public subnet and have a route configured to internet gateway will be able to reach out on the internet.

Now, if we launch more EC2 instances in the private subnet, they can all use this mechanism to reach out to internet. So NAT instance will be providing a single public IP for all these private IPs.

Disable Source/Destination Checks for NAT instance

Select the NAT instance we created above from the EC2 web-console dashboard and then from the Menu bar, choose the option to Change source/destination check as shown below:

This image has an empty alt attribute; its file name is image-102.png

Select the Stop checkbox if it is not selected and there is also some info about why this is needed.

Now, lets try again the curl command from private EC2 Instance and this time, it works:

Cool, our NAT instance is also setup now. Let’s look at the updated architecture diagram next.

Architecture Diagram

I also updated some of the names, Security groups, Route Table and Inbound/Outbound rules. I will cover Security Groups in details in later post. Here is how our architecture currently looks like:

Summary

Bastion host and NAT instance both help secure your AWS infrastructure by disallowing/limiting access to your instances over Cloud. Feel free to ask, If you have some questions or comment. Till next time, Happy Coding.

My Recent Books

8 thoughts on “AWS Basics- Bastian Hosts and NATS”

  1. This is a really good write up on all the moving parts to make a bastion work!
    Back when I started out on AWS, really well written articles like this saved my life.
    I was wondering what you think about doing bastions a little differently?
    For AWS specifically, we have security groups.
    All the resources in a self referencing SG can only communicate with themselves.
    You can spin up all the instances you want in a public subnet with a public IP and as long as they only have a self referencing SG, they’ll never hit or be reached by the publicly routeable internet.
    You need at least one instance in the same VPC (maybe it has to be in the same subnet, it’s been a while and I can’t honestly remember) with the self referencing SG plus one more SG that opens the ports you need for communication and bam, you have a bastion.
    In my personal opinion, using security groups to architect bastions is a lot less work and a lot less ‘gotchas’.

    • Thanks for your comment and yes SecurityGroups are the building blocks for managing access. Your idea of using self-referencing security groups is interesting and I have seen those being used very frequently with layered applications as well. I will try this approach with bastian host and will share my experience.

      In general, we definitely would like to lockdown access to bastian hosts and when we are dealing with just a couple of IP addresses then use of “MY IP” is also valid within security group and some times this itself is very simple to setup and manage.

  2. You should be forwarding ssh credentials with ssh instead of passing the key file to the bastion host. In a compromise, that key can be used in ways you didn’t intend it to be used.

    • Thanks for the comment. Yes, you are right. I will also prefer not to put the key on the bastian host. Can you share the command syntax for this (fwd ssh credentials)? and I will update the article with this info.

Comments are closed.