AWS provides several choices in services when it comes to provisioning resources and deploying applications. If you are not sure which service is best for a particular situation, you’re not alone. I’ve been studying for the AWS DevOps Engineer Professional Exam and, at least for me, there is a lot of confusion surrounding when to use which service. I am certain there are many others who are also looking for answers as well. Just open your browser and type something like “OpsWorks vs “. You will see what I mean. Unfortunately, many of those articles are a bit vague and provide advice such as “Elastic Beanstalk is for developers. OpsWorks is for operations and CloudFormation is for fine grained control”. Unfortunately, the problems on the exam don’t quite slice-and-dice it that easily. This post is going to explore the question from a standpoint of the following three domains: Application Deployment, AWS Resource Allocation and Configuration Management. All three tools provide some level of support for these domains, but in slightly different ways. Let’s dig in.
First some definitions:
Application Deployment – How application software or source code for your application gets to your compute resources. How versioning is handled. How updates are applied and/or rolled back.
AWS Resource Allocation – How do you define required compute resources. How are they allocated and organized. What resources are available by default.
Configuration Management – How your compute resources get configured, with dependencies installed, in order to support your application software.
According to Amazon:
“AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker on familiar servers such as Apache, Nginx, Passenger, and IIS.
You can simply upload your code and Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, auto-scaling to application health monitoring. At the same time, you retain full control over the AWS resources powering your application and can access the underlying resources at any time.”
In other words, EB provides a combination of Application Deployment, AWS Resource Allocation and Configuration Management all wrapped up in one service.
Application Deployment – When you use the AWS Elastic Beanstalk console to deploy a new application or an application version, you’ll need to upload a source bundle or reference an existing S3 bucket containing . Your source bundle must consist of a single ZIP file or WAR file and be no larger than 512MB. Alternatively, using the EB CLI, you can retrieve your source code directly from GIT or AWS CodeCommit. Finally, you can also use AWS CodeBuild to build your source code, test and deploy.
Application Deployment strategy can be set to use a All-at-Once, Rolling, Rolling with Extra Batch or an Immutable deployment. Immutable deployment launches a full set of new instances running the new version of the application in a separate Auto Scaling group, alongside the instances running the old version. When all of the new instances pass health checks, Elastic Beanstalk transfers them to the original Auto Scaling group, and terminates the temporary Auto Scaling group and old instances.
Elastic Beanstalk uses a standardized directory structure for hooks. These are scripts that are run during lifecycle events and in response to management operations: when instances in your environment are launched, or when a user initiates a deployment or uses the restart application server feature.
AWS Resource Allocation – EB is primarily for web applications and can be setup to run your application for low cost (single instance) or as highly available (load balancing, auto scaling). Beneath the covers, EB is using CloudFormation to take care of allocating AWS resources such as Load Balancers and AutoScaling Groups, etc. EB integrates with AWS RDS to provide a database instance to your environment. Other resources that can be configured with EB are Security (service role and instance role/profile), Monitoring (Health Check and Event Log Streaming), Notifications, Network (VPC) and Tags.
Configuration Management – EB supports a number of different platforms including Go, .Net, Java, Node.js, Ruby, PHP, Python and Tomcat, as well as Docker and AWS ECS (Multi-Container Docker). Based on your selection for the platform AWS provides a standard AMI that will be used to the instances. This removes the majority of the configuration management tasks because the AMIs are configured to launch with the correct software and configuration. You can, however, create your own AMI from one of these base AMI’s. The nice thing about the base AMI’s is that AWS takes care or updating the instances in your environment when platform updates are required.
According to Amazon:
“AWS OpsWorks is a configuration management service that provides managed instances of Chef and Puppet. Chef and Puppet are automation platforms that allow you to use code to automate the configurations of your servers. OpsWorks lets you use Chef and Puppet to automate how servers are configured, deployed, and managed across your Amazon EC2 instances or on-premises compute environments.”
So right off the bat, you can see that OpsWorks is more heavy on Configuration Management. Chef is used to streamline the task of configuring and maintaining a company’s servers, and can integrate with cloud-based platforms. Chef uses a domain-specific language (DSL) for writing system configuration called “recipes”. OpsWorks is not just for Configuration Management, it can still of deploy applications and allocate AWS resources.
OpsWorks organizes resources in an top-level entity called a Stack. Stacks represent a set of instances that you want to manage collectively. Stacks contain one or more Layers which contain stack components such as a load balancer or a set of application servers.
Application Deployment – An OpsWorks Stacks app is the code that is run on an application server. The app itself resides in S3 or other code repository. When you deploy an application, OpsWorks triggers a deploy event on the application server which then runs the layer’s Deploy recipes. The Deploy recipes take care of installing the application on the server. These OpsWorks life cycle events (Setup, Configure, Deploy, UnDeploy and Shutdown) fire at specific phases of the EC2 instances life cycle as well as in response to actions on the OpsWorks Console.
AWS Resource Allocation – When you set up an OpsWorks stack, resulting instances will have an OpsWorks agent installed which contains the Chef client. The Chef client is responsible for download and interpreting the Chef Recipes and Cookbooks. The OpsWorks agent orchestrates communication with the OpsWorks Service to handle the allocation and provisioning of AWS Resources as well as firing OpsWorks Life Cycle Events.
Configuration Management – As mentioned above, OpsWorks using Chef recipes for configuration management. You can also create your own custom recipes an put them in an online repository, Git, or a zip file. You specify which recipes you want to execute during the various life cycle events.
According to Amazon: “AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment. CloudFormation allows you to use a simple text file to model and provision, in an automated and secure manner, all the resources needed for your applications across all regions and accounts.”
CloudFormation uses JSON templates to model all the resources in your environment from VPC to EC2 to Load Balancers etc. Essentially anything you can create with the AWS SDK or CLI, you can create with a CloudFormation template.
Application Deployment – While CloudFormation automates infrastructure deployments, when using CloudFormation, the recommended service for deploying applications to EC2 instances is AWS CodeDeploy. According to AWS: “AWS CodeDeploy is a fully managed deployment service that automates software deployments to a variety of compute services such as Amazon EC2, AWS Lambda, and your on-premises servers. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. ”
The CodeDeploy application specification files is called AppSpec.yaml. This file tells CodeDeploy how to:
AWS Resource Allocation – As mentioned above, CloudFormation is the tool for automating the creation and management of AWS resources. The templates can be parameterized for reuse as well as nested. Most any property of any resource can be specified via JSON in these templates, making them very powerful and likely very hard to read. CloudFormation templates can be executed via the AWS Console or from the CLI. The CloudFormation Console provides a visual design tool to assist in grpahically configuring templates. The are libraries of existing standard templates as well as third party templates that can be used directly or modified.
Configuration Management – CloudFormation excels at resource allocation but leaves EC2 Instance configuration management to either using pre-backed AMIs or EC2 User Data scripts. Use the AWS::CloudFormation::Init type to include metadata on an Amazon EC2 instance for the cfn-init helper script. If your template calls the cfn-init script, the script looks for resource metadata rooted in the AWS::CloudFormation::Init metadata key. Associate the CreationPolicy attribute with a resource to prevent its status from reaching create complete until AWS CloudFormation receives a specified number of success signals or the timeout period is exceeded. To signal a resource, you can use the cfn-signal helper script or SignalResource API. With the DependsOn attribute you can specify that the creation of a specific resource follows another. When you add a DependsOn attribute to a resource, that resource is created only after the creation of the resource specified in the DependsOn attribute.
While all three tools provide for some level of Code Deployment, AWS Resource Allocation and Configuration Management, which tool to use depends on the situation. Here is my take:
If you are using or planing on using Configuration Management tools such as Chef or Puppet, then obviously OpsWorks is the way to go. Neither Elastic BeanStalk nor CloudFormation provide for the integration a rich configuration management tool.
If you are building Web applications and are ok with combining standard AWS Resources with pre-configured platforms, then obviously Elastic Beanstalk is your choice. EB deployment is very streamlined from the desktop or from a source code repository such as GIT.
If you are deploying an infrastructure of resources that do not fit the traditional load balanced web application, you will want to use CloudFormation. At this point I don’t see any advantage of using CloudFormation over EB for Web applications. As stated before, EB is using CloudFormation under the covers. But not every architecture is a traditional load balanced web architecture.
I hope this article helped clear up some questions for you. Writing it actually helped me organize my thoughts around the topic. I’m sure I’ll be posting updates as my level of awareness increases. Please comment if you would like to add your 2 cents. Cheers!
CEO of McRaeSoft.com and Independent Software Development Consultant. Specializing in AWS Cloud Architecture, Development and DevOps and Cloud Security.