Know all the steps to host a website in AWS

This post is aimed at hosting a resilient and highly available website in AWS, starting from setting up VPC with layered architecture (2 private subnets and 1 public subnet), to to hosting a website utilizing all the necessary AWS tools. This will help to understand all the steps to host a website in AWS with hands on experience from designing the architecture to setting up the environment and hosting a highly available website. 

Design custom VPC with multi-tier web hosting architecture in AWS

This architecture consists of a Multi-AZ architecture with 3 availability zones with one more spare capacity for future expansion. When designing an Architecture it is always a good idea to think big and always have capacity allocated for the future. (Note: This diagram will be further improved moving froward)

Next finalize the CIDR range with subnets for all the layers required in the design. If you are not familiar with IP addresses or CIDR range check out the link below:

For this design, there are 3 AZs as the Sydney region and there is one space reserved for future AZ expansion,  so 16 subnets are needed. 

In this VPC architecture , 10.16.0.0/16 CIDR range is chosen, however it is essential to check all the CIDR ranges that are currently being used in your business environment to make sure there are no overlapping CIDR ranges. The best practice is to check the IP address Plan for your business to get a full understanding of what ranges are being used and select the range as per your company’s standards.

For 16 subnets, divide the 10.16.0.0/16 to 10.16.0.0/20 – 10.16.128.0/20 that makes up altogether 16 subnets. 

How to design VPC architecture in AWS Cloud?

Login to your AWS environment go to:

• Services > VPC
• Press Create VPC

Provide VPC name, and IPv4 CIDR block which will be 10.16.0.0/16 in this case. AWS allows a minimum /28 (16 IPs) and Max /16 (65,536 IPs) which are private CIDR and additionally we can use optional Public IPv4 secondary IPv4 Blocks. Only 1 IPv6  /56 CIDR Block is acceptable in AWS. Select, Amazon-provided IPv6 CIDR block unless required in your design. 

Select tenancy, dedicated tenancy can be used (dedicated hardware)  if you have a requirement or leave it to default for this demo. (Note: Default tenancy can change to dedicated  after launching in the VPC but dedicated tenancy cannot be changed to default)

Specify tags for the VPC as per your company standards, and press create VPC.

For this newly created VPC modify following:

• Right-Click the newly create VPC and go to => Edit DNS Hostnames and enable it, this will ensure instances launched in the VPC with public IP addresses get DNS hostnames.
• Right-click the newly created VPC and go to => Edit DNS resolution and enable it to ensure DNS resolution is supported.

Create Subnet in AWS VPC

AWS has reserved 3 IPs  altogether 5 IPs (2 for broadcast and network) cannot be used in any subnets in AWS. 

For Network address = 10.16.0.0/20 following IPs are reserved:

1. Network Address = 10.16.0.0 which is reserved and for the network 
2. Network + 1 = 10.16.0.1 = Reserved for AWS VPC Router
3. Network + 2 = 10.16.0.2 = Reserved for AWS DNS
4. Network + 3 = 10.16.0.3 = Reserved for AWS Future Use that has not been used yet
5. Last IP Address in subnet = 10.16.15.255 = Broadcast address

(Note: IPv4 Subnet Calculator is a handy tool to design subnets )

Following Subnet Ranges with names and corresponding AZ are selected for the demo:

NAME – CIDR –  AZ –  CustomIPv6Value

sn-reserved-A 10.16.0.0/20 AZA IPv6 00
sn-db-A 10.16.16.0/20 AZA IPv6 01
sn-app-A 10.16.32.0/20 AZA IPv6 02
sn-web-A 10.16.48.0/20 AZA IPv6 03

sn-reserved-B 10.16.64.0/20 AZB IPv6 04
sn-db-B 10.16.80.0/20 AZB IPv6 05
sn-app-B 10.16.96.0/20 AZB IPv6 06
sn-web-B 10.16.112.0/20 AZB IPv6 07

sn-reserved-C 10.16.128.0/20 AZC IPv6 08
sn-db-C 10.16.144.0/20 AZC IPv6 09
sn-app-C 10.16.160.0/20 AZC IPv6 0A
sn-web-C 10.16.176.0/20 AZC IPv6 0B

Note:- Remember to enable auto-assign ipv6 on every subnet 

Go to AWS console > VPC > Select Subnets.

Create Subnet > Select the VPC created earlier. 

Create all the subnets allocated earlier and finally press Create Subnets.

Note: For IPv6 CIDR block select Custom IPv6 and enter values between 00 to ff which means 256. Tag is automatically created that helps this subnet to be identified later.

Check subnets in the subnet list to ensure all are created correctly.

Next, make sure the IPv6 address is assigned to all the resources in all the subnets.

To do that select each subnet one by one and from actions select Modify auto-assign IP settings and check the box Enable auto-assign IPv6 address and save. Do this to every subnets, 16 subnets in this example.

Note: Cloudformation can be used to automate the task, which will be discussed in separate post

Setup VPC Router and Internet Gateway (IGW) in AWS

The next step of the architecture is the configuration of a VPC router.

Remember few things about the VPC router in AWS:

• Every VPC has a VPC router associate with it and it is highly available.
• Its main job is to route traffic between the subnets in the VPC.
• The router has a network interface in every subnet in the VPC and in every subnet “network + 1” address is used.
• VPC router is associated with route tables in each subnet. Route table’s role is to let the router know where and how to transfer traffic.
• A VPC has the Main route table which is a default. If the subnet has a route table then it is used else VPC route table is used as default.

Go to VPC console and press Route Tables, there are two route tables, one is for default VPC and other for custom VPC  created earlier.

In the Custom VPC route, the destination CIDR range is pointing to the Target = local – that means send traffic to VPC itself.  

Remember: The route table will pick the higher prefix as a priority as it will be more specific to the router. For eg., if there are two routes one with /16 and one with /32 then the higher prefix which is more specific to 1 Ip address (/32) will be picked by the router. 

Internet Gateway:

An internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows communication between the VPC and the internet. It is a Regional Resilient gateway attached to a VPC, which means it will cover all the AZ in the region. An internet gateway serves two purposes: to provide a target in your VPC route tables for internet-routable traffic, and to perform network address translation (NAT) for instances that have been assigned public IPv4 addresses.

Internet Gateway (IGW) runs within the AWS Public Zone and its main job is to provide Gateway traffic between the VPC and the Internet or AWS Public Zone resources such as S3, SQS, SNS, etc. The performance of IGW is managed by AWS.

To create Internet Gateway go to the VPC management console > Internet Gateways > Create Internet Gateway 

Name the Internet Gateway as per your company standards and Press Create Internet Gateway.

After the Internet Gateway is created it has to be attached to the VPC. To attach it to the VPC select actions on the right and Attach to VPC.

Next Select your custom VPC from the dropdown and attach it to Internet Gateway and press Attach internet gateway.

Now the VPC can communicate to the AWS public zone via the Internet Gateway.

The next step is to allow internet access to and from the Web subnets, for that configure the Route table in these subnets.

Configure route table 

In the VPC management console go to Route Tables > Create Route Table.

Name of the Route table as per your company naming standards. (for this demo “rprateek-vpc1-rt-web”). Select the VPC from the dropdown and press Create Route Table.

Next, associate the Subnet to the Route table > Subnet Association > Edit Subnet Association > Select WebA, WebB, and WebC subnets to associate with this route table and Save Associations. (Note: Web subnets are the public subnets)

Add 2 routes that will act as the default Route for IPv4 and IPv6. Edit Routes and Add Route and enter 0.0.0.0/0 (all IPs) and target Internet Gateway (that we created earlier).

Note: Since the prefix /16 is higher than /0 if the traffic comes from or to /16 it will be given priority and then only to /0, any traffic not destined for the VPC 10.16.0.0/16 will go through this Internet Gateway.

For IPv6 in use ::/0 and select the Internet Gateway and Save changes.

Route table is setup to route all traffic except that is meant for the VPC via the Internet Gateway for all three web subnets ie WebA, WebB, and WebC.

For it to work, any resources launched in the Web Subnets must be allocated with IPv4 public IP, so Enable auto-assign Public IPv4 addresses for web subnets. 

Go to Subnets => Select Web A => Actions => Modify auto assign IP Settings => Tick Enable auto-assign public IPv4 address => then Save. Do the same for Web B and Web C subnets.

Now all resources launched in these Subnets will be automatically allocated the Public IP addresses by AWS.

Time to test this setup if it is working or not, for that launch an EC2 instance in the Web Subnet. Call it a BASTION HOST. so that it can be used for other tasks as well. 

Check the below link if you are unaware of how to launch an instance.

The Bastion Host EC2 instance launched in Webs-A subnet  can access the internet. 

Next secure private subnets using NACL (Network Access Control List) and Security Group (SG).

 

What is NACL? (Network Access Control List) 

Network Access Control List (Network ACL, or NACL) is an optional layer of security for the VPC that acts as a firewall for a subnet. All traffic entering or exiting a subnet is checked against the NACL rules to determine whether the traffic is allowed in/out of the subnet. NACL rules are executed in a defined order. The first rule that matches the traffic will determine whether the traffic is allowed or denied. Some important points about NACLs are:

• They are stateless: NACLs are stateless and their initiation of the request and response are seen as different and thus for allowing the traffic and denying the traffic there are two separate rules.
• NACLs impacts data crossing subnet borders and can Explicitly allow or deny.
• NACLs cannot be assigned to any other AWS resources it is only used in the SUBNETS. One subnet can have one NACL at a time 
• By default NACLs allows all, both inbound and outbound traffic.
• The lowest rule number in the NACL will be processed first, if matched with the rule then it will stop processing higher numbers. For eg, a rule no 5 to denys the traffic and rule number 15 allows the traffic then the traffic will be denied and the rule ends. 

What is a Security Group? (SG)

A security groupacts as a virtual firewall for instance to control inbound and outbound traffic. Security groups act at the instance level, not the subnet level, threfore an instance can have up to five security groups assigned.

Things to remember about security groups are:

• Security groups have hidden Implicit DENY which means by default all the things are denied as default so you can only allow the traffic and cannot have a rule that denies it. Therefore sometimes NACLs are important to use in conjunction with Security Groups to deny some traffic.
• Security groups can filter based on AWS logical resources
• There no EXPLICIT DENY

What is NAT Gateway?

A NAT Gateway is Network Address Translation (NAT) service. A NAT gateway can be used so that instances in a private subnet can connect to services outside VPC but external services cannot initiate a connection with those instances.

NAT Gateway’s job is to allow multiple private IP addresses to masquerade behind the public IP address it holds. NAT Gateway communicates with Internet Gateway to communicate to the internet.

Things to remember about NAT Gateway:

• Must run from the public subnet
• Uses Elastic IPs (Static IPv4 Public)
• AZ resilient Service (HA in that AZ) – remember if AZ fails it also fails
• For region resilience – we must implement NATGW in each AZ with a Route table for each AZ with that NATGW as a target.
• It is managed service and can scale up to 45 Gbps of data and we can use multiple NATGW if there is heavy traffic to distribute the traffic. It costs 4c per hour and a data processing charge of 4c per Gib.
• NAT is not required for IPv6
• Internet Gateway works with all IPv6 directly

Revised architecture with NACLs, NAT Gateway looks like below:

Set up internet connection in Private Subnet using NAT Gateway

To test NAT Gateway a private instance is needed to check how it can be connected to internet via NAT. So for that, launch a private instance (rprateek-Private_Test) under private subnet sn-app-A. 

Since, the instance is Private, it is not accessible via internet (or normal methods). To overcome this use SSH Agent forwarding to securely connect to the Private Linux Instance via BASTION host instance that has Public IP.  

Use SSH-Agent Forwarding to connect to Private instances from Public Instance

Connect to public Bastion Host via SSH using the pem file. Then try to connect to private EC2 instance from the Bastion > Permission Denied !! why ? The Bastion Host doesn’t have the access keys .pem file to connect to private EC2 instance. 

To resolve this issue the private key can be copied to the Bastion Host public server to connect to private instance in the Private subnet. But this will be a serious security breach and is not a good practice to implement in a highly secure environment.

The second option is to use SSH – Agent Forwarding option that will allow passing private access key from the Public Instance to the Private instance to gain access.

To use SSH – Agent forwarding in Windows, download and use Putty software

Putty doesn’t recognize .pem file so convert the .pem file to Putty recognizable format which can be done by downloading PuttyGen software. (Note: Putty installation foder contains PuttyGen)

In Putty Gen, load the .pem file and press the Save Private Key button and give the name and path to save it. (Note: it will be .ppk file)

Next, use Pageant (found in Putty installation folder) for Putty Authentication Service. Open Pageant and Add the .ppk key. 

Configure PuTTY for SSH Connection

Enter the host address of the Bastion Host  

Host Name (or Ip address) : 54.252.251.25  (public IPv4 DNS or Public IP address) – (if public DNS is not showing any value then go to VPC that the instance is connected > Action > Edit DNS host Names > DNS Hostnames Enable)

Port: 22, SSH selected and Save the session by giving the name in Saved sessions and Press save so that it can be used easily later on.

Specify the username to connect to by going to Connection > Data > Auto-login username: ec2-user

Next important step is to allow Agent Forwarding in the Putty.

Open SSH > Auth and Tick Allow Agent Forwarding 

Click Open to connect to Bastion Host with Agent forwaring authority. 

Next, Connect to the Private Subnet Instance issuing private Instance ssh Command. Connection to private instance via Bastion host is successful but the private instance doesn’t have internet access. 

There is the need for NAT Gateway to allow internet in private Ec2 instance.

Create NAT Gateway

Go to VPC Console and click on NAT Gateways from the left menu and press Create NAT Gateway.

1. Name> rprateek-vpc1-natgw-A (or as per company standards)
2. Subnet> sn-web-A (NAT gateway resides on public subnet)
3. Connectivity type > Public.
4. Click Allocate Elastic IP (Elastic IP or reserved Public IP is required for Nat Gateway) 

(Note: NAT Gateway can incur cost, delete after testing and release Elastic IP to avoid any costs)

NAT Gateway for Web A Subnet is configured. NAT Gateway is AZ resilient so create 2 more NAT gateway for Web B Subnet and Web C Subnet, at the end there will be 3 NAT Gateways serving 3 AZs. 

Route Private IP to Internet Gateway via NAT Gateway

To route private IP to IG via NAT following steps can be performed:

 

Create Route Table

Go to VPC console > Route tables > Create Route Table.

Name:  rprateek-vpc1-rt-privateA (as per your company standard) 

VPC: rprateek-vpc-1 (current working VPC)

Create 3 route tables for all 3 availability zones: “rprateek-vpc1-rt-privateA”, “rprateek-vpc1-rt-privateB” and “rprateek-vpc1-rt-privateC” and add route to point to the NAT gateway. 

Go to Edit Routes > Add Route

Destination > 0.0.0.0/0 (allow all traffic)  and Target > NAT gateway A. Now all the traffic to and from the NAT gateway A will be processed.

Do the same for route tables for Private subnet B and Private subnet C.

Test the internet internet access through the private instance – it is still not working – what’s missing?

Route tables are associated with the NAT Gateways, but not associated with any subnets, so they won’t know where to send the traffic to.

Associate the private subnets to the route table.

Select the private route table A ie: rprateek-vpc1-rt-privateA > Subnet Associations  => Edit Subnet Associations.

Select all the private subnets in AZ A >  sn-app-A, sn-db-A, and sn-reserved-A.

Similarly, associate all the private subnets from AZ B to rprateek-vpc1-rt-privateB and AZ C to rprateek-vpc1-rt-privateC.

Go to private EC2 instance try to ping 1.1.1.1 to the public internet (it works).  Private instance is configured to access internet via the internet gateway successfully.

Note: Remember to delete the NAT Gateway and release Elastic IPs to avoid incurring charges while in demo or testing. 

NAT is configured next is preparing a Web server to host a website. 

Launch EC2 Instance (web-server-001-on-sn-web-A) and allow HTTP access to the server. 

Install Apache Server in EC2 instance

Connect to the instance and to install Apache server go to root > sudo su

Install Apache server > yum install httpd -y

This should install the Apache server in your instance.

Go to var/www/html directory and host a sample page to check the website is working. > cd /var/www/html

Download a sample website (Tried to used personal Github directory)

Install Git in the Instance> sudo yum install git -y

Clone the Git directory: git clone https://github.com/rprateek/testwebsite.git 

(Note: the Git directory is different in the images but the concept is simple just download html contents)

This will download the testwebsite folder in the HTML. To move all contents from testwebsite to HTML folder go to testwebsite folder and issue command > mv testwebsite/* .

Sometimes html folder needs read/write/execute permissions so issue > sudo chmod 7777 /var/www/html -R

Start the Http service > service httpd start

Sometimes you get error: Failed to start httpd.service: The name org.freedesktop.PolicyKit1 was not provided. See system logs and ‘systemctl status httpd.service’ for details.

This is just the warning ignore it, if you want to check the status of the server you can check by issuing the command > systemctl status httpd

Copy the Public IP or Public IPv4 DNS of the instance and check in the browser, website must be up and running

The next step is to configure AWS Route53 to host the DNS records and setup failover routing to S3 if the webserver goes down the static conten hosted in S3 will be presented. 

Host Static website in S3 for failover routing

Route53 is a highly available and scalable cloud Domain Name System (DNS) web service. An R53 hosted zone is a DNS database for a domain. for eg. www.example.com. These are globally resilient having multiple DNS Servers. It can be created with a domain registered via R53 or can be created separately. Hosted zones are what the DNS system references – Authoritative for a domain eg. example.com. It contains host DNS Records such as A, AAAA, MX, NS, TXT, etc…

DNS Database (Zone File) hosted by R53 (Public Name Servers) are accessible from the public internet & VPCs that are hosted on “4” R53 Name servers specific for the zone. Externally registered domains can point at R53 Public Zone.

R53 Private Hosted Zones

The hosted zones are not public and are associated with VPCs which are available only within those VPCs via Route53 resolver. These types of private hosted zones can be used for hosting internal intranet sites for an organisation. Thus, the same domain name can be used for internal (within the Private hosted zone) and external (public hosted zone) pointing to different servers.

Creat R53 Public Hosted Zone

Go to Route 53 management console > Create hosted zone

Domain Name > example.com (give the root domain)

Type > Public hosted zone

Tags > as per your company standards

Failover Routing using Route 53

The concept of failover routing is, if the main website hosted in EC2 web server goes down then point to the static website hosted in S3.

Assign Elastic IP to web sever

An Elastic IP address is a reserved public IP address that can be assigned to any EC2 instance in a particular region, until choosen to release it. EIP replaces the default public IP address and remains attached to EC2 instance until manually detached. 

It is important fo assing EIP to a webserver so that it has static public IPv4 address attached to it. 

Go to EC2 console > Elastic IPs  >Allocate Elastic IP address 

Add Tag > Name > EIP-for-Web1A (or as per the company standards) and press Allocate. 

Select the Elastic IP > Actions > Associate Elastic IP address

Resource Type > Instance 

Instance > Choose the web server 

Private IP address > Choose the private ip of the instance 

Tick > Allow this Elastic IP address to be reassociated and Press Associate. 

Create health check and configure failover routing in Route 53

Go to Route 53 > Health Checks > Create health check

Name > example.com-health (give name as per the company’s naming standard)

What to Monitor > Endpoint

Monitor an endpoint 

Specify Endpoint by > Ip address

Protocol > HTTP

IP Address > 13.236.131.128 (Elastic IP address of the webserver created before)

Host Name: example.com

Port: 80

Path / > index.html

Advanced Configuration

Request interval > Fast (10 seconds) (or as desired)

Then press Next. 

Get Notified when health check fails

Create alarm > No ( No alarm needed for now just testing failover routing)

Click > Create health check 

(Note: It will take few minutes for health check to change status from unkown to healthy, the progress can be checked in the health checkers tab)

Setup Failover Routing 

First delete the previously created simple routing A records from the hosted zone. 

Go to Route 53 > Hosted zones > Select the hosted zone for the website > Create Record > Failover > Next

Change the TTL > 60 ( to test just make it fail within a minute)

Define faileover Record for Primary – EC2

Value/Route traffic to > IP address or another value, depending on the record type > Enter the Elastic IP address of the web server. 

Failover record type > Primary (make this webserver the primary web server)

Health Check > Select the health check created earlier (example.com-health)

Record ID > EC2 (this has to be unique in this failover section)

Define faileover Record for Secondary – S3

Value/Route traffic to > Alias to S3 website endpoint

Choose Region > Asia pacific (sydney) (Select the region where the bucket is created)

S3 Endpoint > Select the S3 website endpoint. 

Failover record type > Secondary (this will be used when primary server fails its health check)

Health check > empty

Record ID > S3 

After primary and seconday failover records are created press Create records. 

Now the domain (example.com) should be pointing to the website hosted on EC2.