Review: AWS Bottlerocket vs. Google Container-Optimized OS
Running large numbers of containers to deploy an application requires a rethink of the role of the operating system. Google’s Container-Optimized OS and AWS’s Bottlerocket take the traditional virtualization paradigm and apply it to the operating system, with containers the virtual OS and a minimal Linux fulfilling the role of the hypervisor.
Various flavors of Linux optimized for containers have been around for a few years and have evolved ever smaller footprints as the management and user-land utilities moved to the cluster management layer or to containers. These container-optimized operating systems are ideal when you need to run applications in Kubernetes with minimal setup and do not want to worry about security or updates, or want OS support from your cloud provider.
Container OSs solve several issues commonly encountered when running large container clusters, such as keeping up with OS vulnerabilities and patching potentially hundreds of instances, updating packages while dealing with potentially conflicting dependencies, degraded performance from a large dependency tree, and other OS headaches. The job is challenging enough with a few racks of servers and nearly impossible without infrastructure support when managing thousands.
Bottlerocket is purpose-built for hosting containers in Amazon infrastructure. It runs natively in Amazon Elastic Kubernetes Service (EKS), AWS Fargate, and Amazon Elastic Container Service (ECS).
Bottlerocket is essentially a Linux 5.4 kernel with just enough added from the user-land utilities to run containerd. Written primarily in Rust, Bottlerocket is optimized for running both Docker and Open Container Initiative (OCI) images. There’s nothing that limits Bottlerocket to EKS, Fargate, ECS, or even AWS. Bottlerocket is a self-contained container OS and will be familiar to anyone using Red Hat flavors of Linux.
Bottlerocket integrates with container orchestrators such as Amazon EKS to manage and orchestrate updates, and support for other orchestrators can be adding by building variants of the operating system to add the necessary orchestration agents or custom components to the build.
Bottlerocket’s approach to security is to minimize the attack surface to protect against outside attackers, minimize the impact that a vulnerability would have on the system, and provide inter-container isolation. To isolate containers, Bottlerocket uses container control groups (cgroups) and kernel namespaces for isolation between containers running on the system. eBPF (enhanced Berkeley Packet Filter) is used to further isolate containers and to verify container code that requires low-level system access. The eBPF secure mode prohibits pointer arithmetic, traces I/O, and restricts the kernel functions the container has access to.
The attack surface is reduced by running all services in containers. While a container might be compromised, it’s less likely the entire system will be breached, due to container isolation. Updates are automatically applied when running the Amazon-supplied edition of Bottlerocket via a Kubernetes operator that comes installed with the OS.