Accessing Public S3 Bucket from EC2 instance in a Private Subnet without NAT Gateway or IGW

Scenario : In a production AWS Cloud environment, we generally have our EC2 instances hosted in a private subnet which are not able to access internet or any public resource nor these EC2 instances are accessible through their public IP / DNS name. They are only accessible through a jump server placed in Public Subnet of that VPC and right Route Table configuration.  

Problem : Since S3 buckets are extensively used to push data like logs / archives from EC2 instances hosted in Private Subnet, they generally need NAT gatewate / IGW to be attached with that private subnet so that those S3 buckets could be accessed. But NAT gateway takes the traffic to internet and then connects to S3 which is a chargeable service. So how can we access S3 public buckets of the same regions from EC2 instances hosted in Private Subnet of an AWS VPC?

Solution: The solution to this problem is using VPC Endpoints. VPC Endpoints is a service provided by AWS which allows traffic from your private subnet to other AWS services like S3 without using any NAT Gateway, Elastic IP or IGW.

Assumptions: I am assuming that following things are already in place in your environment :

1. A public subnet having access to internet. (Assuming IP range as 10.0.0.0/24)
2. A Private Subnet within the same VPC. (Assuming IP range as 10.0.1.0/24)
3. A windows EC2 instance in Public Subnet having a public IP assigned to it so that you could access it from your desktop / laptop.
4. An EC2 instance (Take linux for better testing) in Private Subnet without any internet access by default.
5. Private subnet and public Subnet can access each other using their private IPs. 
6. You are able to access EC2 server of your private subnet through the jump server hosted in public subnet.
7. You have a S3 bucket which you can access with your configured access policies.
8. You have atleast one user with access key - pair which has full access on that S3 bucket.

Steps. 

We will now first test if the S3 bucket is accessible from the EC2 instance of private subnet.

1. Login to your Jump Server (Windows host) and connect to your EC2 instance (of Private Subnet) using its private IP. (in my case 10.0.1.70).

2. Once you login to the EC2 instance, configure AWS CLI so that you could connect to your AWS subscription from your private EC2 instance. Put following command on EC2 putty terminal:

$ aws configure

Enter the Access Key and Secret key of the user which has access to the S3 buckets you want to access. 

3. Now check if you are able to list the S3 bucket from this EC2 instance which your user already has access to. Put following command on EC2 putty window..

$ aws s3 ls

The window will not show anything because your user is not able to reach to S3 bucket from your EC2 instance which is hosted in Private Subnet and that subnet does not have any NAT Gateway or IGW associated with it.

Creating a VPC Endpoint 

4. To enable the EC2 instance of Priave Subnet to access S3 buckets without internet access, we will use VPC Endpoint.

5. Go to VPC Service on your AWS Console and select EndPoints from Left hand menu. and click on Create Endpoint

6. In the proceeding window. Select the AWS Service as " com.amazonaws.us-east-1.s3"  and VPC "your own VPC where your Private and Public Subnets are" . Also select the Route Table to which this Endpoint will be associated. The same route table must be associated with your private subnet. 

7. Now click on "Create Endpoint".

A message will be displayed saying "VPC Gateway Endpoint created successfully.

8. you can also see then newly created Endpoint when you click on Endpoints in left hand menu.

9. Cross check that your private subnet now has route entry which associates it with this Endpoint.

10..You have now created the VPC Endpoint, its time to test if your EC2 instance can now connect to S3 buckets or not?

11. Go to EC2 instance's putty terminal again and run the following command one more time: 

$ aws s3 ls 

This time , it should list the S3 buckets to which your user has access to.

12. This confirms that your EC2 instance which is hosted in public subnet which does not have any internet connection or is not associated with any NAT Gateway or IGW can now access the S3 buckets within that same region. 

13. you can perform all S3 operations on these buckets if your user has privileges to do so...

14. Lets copy a file name "S3toEC2usingVPCEndPoint.txt" from EC2 instance to S3 Bucket "lalit-privatesubnet-test-s3". Trigger following command :

$  aws s3 cp S3toEC2usingVPCEndPoint.txt s3://lalit-privatesubnet-test-s3

This command will upload a file named "S3toEC2usingVPCEndPoint.txt" from EC2 instance to S3 Bucket "lalit-privatesubnet-test-s3".

This is how we can configure VPC Endpoints in AWS to allow our EC2 instances hosted in private subnet to connect to S3 buckets without having any NAT Gateway or IGW or Internet Access.

Enjoy!!!!!