Virtualization is a technology widely used by cloud computing providers to assemble virtual computer systems from a physical device. Despite the various benefits of virtualization, engineers worldwide are still investigating techniques to increase its performance and efficiency. For this reason, the AWS research team created Firecracker, a virtualizer capable of creating lightweight Virtual Machines (VMs).
Background: an Inefficient Virtualization Approach
Nowadays, the serverless model is one of the most widely adopted options worldwide for implementing applications. Serverless allows companies to focus primarily on business strategies, abstracting the software and hardware management tasks and optimizing Dev teams’ work times.
For this reason, Amazon created AWS Lambda and AWS Fargate. These serverless computing platforms enable developers to run code without thinking about the configuration and maintenance of the back-end servers. Serverless solutions from AWS and other providers must be able to scale computational resources, run code as close to the end user as possible (reducing latency), perform deployments and updates rapidly, and ensure the security of the information of its users. And all this automatically!
AWS uses virtualization in Lambda and Fargate to isolate accounts from different customers and uses Linux containers to isolate functions. In other words, AWS uses a VM to isolate and run multiple functions of a single client. The problem with the actual AWS approach is that containers do not provide the necessary security. In addition, its traditional virtualization presents performance difficulties such as inefficient packing of functions in fixed-size VMs, long initialization times (commonly seconds), high memory and CPU consumption (limiting VM density), and intrinsic hypervisor issues.
With this in mind, AWS engineers looked at ways to optimize their current virtualization model by creating a new VM deployment model, but one that included the following features:
- Isolate and run multiple functions on the same hardware and provide strong security against a wide range of attacks.
- Run high workload densities with low overhead.
- Run functions in the same way as those that are executed natively.
- Have compatibility with a wide range of existing software on the AWS platform.
- Create and delete functions quickly.
- Allocate computational resources (e.g., CPU, memory) efficiently and intelligently.
AWS Firecracker
To create a more efficient virtualization technology, AWS analyzed both the traditional solutions and the versions still in development. The engineers considered that container-based technologies provide efficient operational isolation but not a reliable security level. While the use of language features, such as Java Virtual Machine (JVM) or V8, does not fit the isolation techniques used in Lambda and Fargate.
The AWS team kept the idea of using virtualization and created Firecracker, a more efficient open-source solution for managing multi-tenant container and function-based services. Firecracker uses the Linux Kernel-based Virtual Machine (KVM) virtualization infrastructure to replace the traditional Virtual Machine Monitor (VMM) and provide lightweight VMs called microVMs.
The strategy was to create a VMM with built-in Linux components and exclude non-essential functionalities to provide OS guests with the right features, performance, and design, while also taking advantage of containers’ speed and efficiency.
Firecracker is written in Rust and has a REST API for configuring, managing, starting, and stopping microVMs and for providing a metadata service for data exchange between the host and guest services.
According to the article published by the AWS research team, the “REST APIs exist for specifying the guest kernel and boot arguments, network configuration, block device configuration, guest machine configuration and cpuid, logging, metrics, rate limiters, and the metadata service.”
Firecracker Performance
To evaluate the efficiency of the new virtualizer, AWS engineers performed several tests by using metrics and comparing Firecracker with other VMMs, such as Cloud Hypervisor and QEMU.
For example, for the boot time metric, defined as “the time between when VMM process is forked, and the guest kernel forks its init process,” Firecracker reached boot times up to 125 ms and 150 ms for starting microVMs in serial and parallel, respectively. When compared with the other VMMs, Firecracker exhibited similar boot times to the Cloud Hypervisor and lower boot times than QEMU (up to 300 ms).
For the memory overhead, the metric was “the difference between memory used by the VMM process and the configured MicroVM size.” Under this metric, Firecracker showed an overhead of 3 MB per microVM, while the Cloud Hypervisor and the QEMU had an overhead of 13 MB and 131 MB per VM, respectively.
Finally, the engineers evaluated the IO performance with Fio, an open-source benchmarking tool that enables the testing of networks and storage devices. For this test, “Fio was configured to perform random IO directly against the block device, using direct IO through libaio, while the local NVMe SSDs backed all tests on the m5d.metal server.” The results showed some limitations regarding the bandwidth. Still, for latency, the figure below shows that Firecracker has a significantly higher overhead for large blocks but performs well for small ones.
Summary
Firecracker is a VMM modified and optimized to deploy lightweight Virtual Machines (microVMs) quickly and securely. Its primary goal is to improve the developer experience on AWS products like Lambda and Fargate, which currently support millions of production workloads and billions of requests per month.
Firecracker is still a project in development and encourages developers to make contributions. Firecracker is licensed under the Apache license, allowing you to freely manage your contributions on the terms you choose.