Five Scalable and Highly Available Deployment Architectures with AWS

In the enterprise software domain, a majority of organizations leverage cloud service providers for deploying their software products or solutions. While different types of architectures cater to various business needs, the effectiveness of a software solution extends beyond its initial design. Without adhering to an efficient, scalable, and highly available deployment strategy, the value of a well-architected software solution diminishes. Such oversight not only constitutes a financial loss for the organization but also impacts revenue adversely.

In this article, I will expound upon five deployment strategies for your software application, emphasizing three foundational pillars that undeniably contribute significant business value to an organization. Those three factors are,

Scalability
Ability to scale in or scale out the individual instances based on load hitting the application

High Availability
Ensuring the capability to remain resilient in the face of a catastrophe or failure is imperative. The objective is to sustain the functionality of the application, even in scenarios where instances may be inadvertently shut down.

Cost-effectiveness
Maximizing value return to the organization through the minimization of both infrastructure and deployment costs

Deployment 1 — Multi AZ EC2 instances combining with ASG and ALB as the load balancer.

In this approach, we structure our resource stack as follows

• Create a target group with multiple EC2 instances spanning across more than one availability zone. It is worth noting that more than one target group can be created based on the specific needs of the application.
• Develop an Auto Scaling Group within the target group, specifying the launch template and scaling metrics to ensure dynamic scaling based on demand.
• Generate an Application Load Balancer (ALB) to act as a load balancer. The ALB distributes traffic to the EC2 instances in target groups according to defined rules. This ensures effective load distribution and enhances the overall reliability and performance of the system.

pros

This architecture ensures high availability as the EC2 instances are deployed across two availability zones. Thus, if one AZ becomes non-functional, the application can still maintain its functionality.

This architecture ensures scalability by deploying EC2 instances within an auto-scaling group. The launch configuration template and scaling policy should be set up to enable scaling based on the incoming load. In case of high demand, additional EC2 instances are automatically spun up to manage the load, ensuring uninterrupted application functionality.

ALB routes requests to EC2 instances based on the routing policy, effectively balancing requests across each EC2 instance in the target group.

cons

Manage your infrastructure with either on-demand or savings plans.

Write raw launch commands, such as installing relevant packages and libraries, starting the server if applicable, installing Docker if in use, pulling the image, and running the containers.

During the scaling process, it will spin up an entire new instance of the selected EC2 family. Unlike AWS ECS, individual tasks cannot be deployed.

Deployment 2—Tasks deployment on EC2 with docker and ECS combining ALB as the load balancer.

In this approach, we structure our resource stack as follows

• Create an ECS cluster by specifying the EC2 launch type and allocating the EC2 instances for that specific cluster.
• Establish a service for each microservice component within this cluster and specify the target group to be utilized.
• Subsequently, formulate a task definition by delineating the memory and CPU units for the specific task. Include the Docker image path, environment variables, and credentials for Docker Hub, if chosen over ECR. Alternatively, ECR can be utilized.
• Conclude the process by generating an Application Load Balancer with the necessary rules. Proceed to configure the ECS service and target groups for integration with the created load balancer.

pros

This architecture ensures high availability by deploying ECS tasks across two availability zones. If one zone becomes non-functional, our application can continue functioning seamlessly.

With ECS, we eliminate the need to worry about application deployment and configuring launch commands. We only need to specify the Docker image, memory, and CPU in the task definition, and ECS will handle the deployment on the appropriate EC2 instances. It’s important to note that in ECS, the smallest and functional unit we consider is a task, not an EC2 instance.

Auto-scaling is seamlessly managed by ECS. In the event of high load, ECS will scale by spawning new tasks proportionally to the load.

The load balancer efficiently handles and routes the load to the appropriate target groups, consisting of ECS tasks.

During the scaling process, instead of adding an entire new EC2 instance, ECS adds a new task to the existing EC2 instances based on resource availability using Docker.

This architecture is particularly well-suited for containerized applications. Moreover, you pay for the underlying resources you use without any additional cost for the ECS service.

cons

Given that this strategy utilizes EC2 launch configuration, the responsibility falls on us to manage our own EC2 instances, including tasks such as installing patches and updates.

It’s important to note that this approach is most suitable for containerized applications.

However, it may be susceptible to challenges such as underutilization or overprovisioning of EC2 instances for unpredictable workloads. These factors can have financial implications for the organization.

Deployment 3 — Tasks deployment with ECS fargate combining ALB as the load balancer.

This strategy mirrors the previous one, with the key distinction being the adoption of a fully serverless approach. Here, we abstain from managing any resources independently, moving away from the use of self-managed EC2 instances.

• Create an ECS cluster by specifying the Fargate type.
• Establish a service for each microservice component within this cluster and specify the target group to be utilized.
• Subsequently, formulate a task definition by delineating the memory and CPU units for the specific task. Include the Docker image path, environment variables, and credentials for Docker Hub if chosen over ECR. Alternatively, ECR can be utilized.
• Conclude the process by generating an Application Load Balancer with the necessary rules. Proceed to configure the ECS service and target groups for integration with the created load balancer.

pros

Similar to the previous architecture, this design ensures high availability by deploying ECS tasks across two availability zones, maintaining functionality in case of zone failure.

Leveraging ECS eliminates concerns about application deployment and launch command configuration. Specify Docker image, memory, and CPU in the task definition, allowing ECS to deploy tasks on Fargate instances, with the task as the smallest functional unit.

Auto-scaling is managed by ECS, dynamically adding tasks in response to high loads.

The load balancer efficiently directs traffic to target groups of ECS tasks.

During scaling, a new task is added to the target group, avoiding the need for an entire new EC2 instance.

This architecture is well-suited for containerized applications, with payment based on underlying resources, excluding extra ECS service costs.

Unlike the previous setup, AWS manages EC2 instances in Fargate. Our primary responsible element is a task, making it effective for applications with unpredictable loads and mitigating concerns about underutilization or overprovisioning.

cons

Not a good option to use if your application has a predictable load.

best suited for containerized applications.

Deployment 4 — AWS lambda functions integration with API gateway.

For a fully serverless approach with minimal cloud operations, ideal for microservices with execution times under 15 minutes, use this strategy.

No need to build a REST API from scratch, simply implement the required functionality in concise code. The API gateway manages all endpoint routing.

• Create an API gateway resource.
• Establish lambda resources for each intended endpoint and deploy the code.
• Configure the API gateway endpoint by linking each lambda function to its corresponding endpoint.

Pros

Create only the necessary functionality. For Node.js, it involves a function without a server; for Java, a function without a main class or method, and similarly for other languages.

This results in a lightweight and small codebase, leading to faster development and easy management.

Adopt a fully serverless approach, eliminating the need to configure servers, load balancers, target groups, or auto-scaling. AWS handles these aspects; you deploy, and AWS manages the rest.

Utilizing small executable functions reduces code complexity and development effort, resulting in cost savings.

Due to scalability based on load, this strategy is well-suited for applications with unpredictable loads.

Cons

Ideal for small to moderately complex projects, but not suitable for REST APIs with over 20 endpoints.

Each task must be completed within 15 minutes, otherwise the lambda function will be terminated automatically.

Deployment 5— AWS EKS integration combined with ALB.

This Kubernetes deployment on AWS caters to organizations that prioritize Kubernetes over other container-managed services. Some entities adhere to internal policies, while for others, Kubernetes is deeply ingrained in their technical architecture. In such contexts, this strategy proves highly suitable.

• To initiate the process, a Kubernetes cluster needs to be established and deployed within the AWS environment.
• Worker nodes can be deployed using three strategies —
  self-managed nodes, managed node groups, and Fargate.
• Worker node target groups should be configured to span across multiple Availability Zones (AZs).
• An Application Load Balancer (ALB) can be configured by integrating an ingress controller with ALB.

Pros

You can leverage built-in Kubernetes functionalities if you’re already familiar with the Kubernetes ecosystem.

Additionally, you can integrate other AWS services with Kubernetes to enhance business value, such as CloudWatch, IAM, KMS, and S3.

The control plane of the Kubernetes cluster will be managed by AWS, leaving you responsible only for node and node group management.

AWS will deploy the most efficient EC2 instances as pods, aligning with your memory and CPU configurations. This eliminates concerns about overprovisioning or underutilization.

In managed node groups, auto-scaling will be handled by AWS EKS; we simply need to configure the maximum and minimum instances.

Due to scalability based on load, this strategy is well-suited for applications with unpredictable loads.

Cons

The learning curve is somewhat steep as it requires knowledge of both Kubernetes and AWS infrastructure.

Costs tend to be higher compared to ECS. Since the control plane is managed by AWS, there are expenses associated with that in addition to paying for the underlying resources.

Ending

In this article, I have presented five well-recognized and organized deployment architectures that can significantly enhance the business needs of organizations. However, before implementing any of the above deployments in the real world, it’s crucial to consider the following trade-offs. Neglecting these trade-offs could result in the implemented architecture being detrimental to the organization.

• Always be mindful of your organization’s needs and implement the architecture in a way that minimizes costs while achieving maximum business requirements.
• Consider scenarios that may lead to overprovisioning and underutilization, and mitigate them as much as possible.
• Review your architecture with peers, engineering leads, and architects to gain valuable feedback.
• Identify and address bottlenecks that could impact resource allocation and optimize accordingly.
• Utilize monitoring tools such as Datadog to track logs, CPU, memory metrics, and costs.
• Plan for future requirements rather than focusing solely on short-term goals, and configure the architecture accordingly.
• Document the architecture, any modifications made over time, and critical points to ensure clarity and facilitate future maintenance.