Developers love containers. They’re easy to use and fast to start. You can run a lot of them on even simple hardware. Startup overhead has always been a bane of development and testing, and this overhead only increases with microservices architectures. If a developer needs a half dozen services, he might easily waste a day or two with setup -- configuring hardware, running installers, fighting incompatibilities.
With containers, that collapses to minutes or seconds and can be run on one development workstation. The readily available repositories of useful container images multiply developer productivity, much like open source does, but without the trouble of doing a build. Operations teams have been slower to adopt containers. One reason is that many applications they must support aren’t yet containerized. Another reason is a reluctance to move away from VMs.
For ops, the shift from bare metal to VMs was natural. Individual VMs look and can be managed like individual systems, using the same tools and processes. Early concerns about VM security were allayed by the long production history of VMs in the mainframe world, by the ability to apply the same controls as used for bare-metal systems, by hardware virtualization support, and by the evolving maturity of VM management tools.
Many early security worries came down to one question: Were VMs as secure as bare metal? Now similar questions are being raised about containers. How secure are containers, and how do they compare to VMs? Certainly if we compare services running in containers against the same services running as separate processes on the same system, the container version is more secure. The separation provided by Linux namespaces and cgroups provides barriers that do not exist between plain processes. The comparison with VMs is less clear cut. Let’s take a look at VMs and containers from a security perspective.
In this article, I’ll take two different approaches to comparing VM and container security. The first approach will be more structural, or theoretical, looking at the characteristics of each from a security perspective. Then I’ll apply a more practical analysis by looking at what happens in a typical breach and how it might be affected by container and VM architectures.
Structural view
For the structural approach I’ll compare the attack surface of both systems. An attack surface represents the number of points at which a system can be attacked. It isn’t precisely defined (as a number, for example) but is useful for comparisons. To a burglar, a house with 10 doors has a greater attack surface than a house with one door, even if the doors are identical. One door might be left unlocked; one lock might be defective; doors in different locations might offer an intruder more privacy, and so on.
In computer systems, the attack surface includes anything where the attacker (or software acting on his behalf) can “touch” the target system. Network interfaces, hardware connections, and shared resources are all possible attack points. Note that the attack surface doesn’t imply that an actual vulnerability exists. All 10 doors might be perfectly secure. But a larger attack surface means more places to protect and the greater likelihood the attacker will find a weakness in at least one.
The total attack surface depends on the number of different touch points and the complexity of each. Let’s look at a simple example. Imagine an old-fashioned system that serves up stock market quotes. It has a single interface, a simple serial line. The protocol on that line is simple too: A fixed length stock symbol, say five characters, is sent to the server, which responds with a fixed-length price quotation -- say, 10 characters. There's no Ethernet, TCP/IP, HTTP, and so on. (I actually worked on such systems long ago in a galaxy far, far away.)
The attack surface of this system is very small. The attacker can manipulate the electrical characteristics of the serial line, send incorrect symbols, send too much data, or otherwise vary the protocol. Protecting the system would involve implementing appropriate controls against those attacks.
Now imagine the same service, but in a modern architecture. The service is available on the Internet and exposes a RESTful API. The electrical side of the attack is gone -- all that will do is fry the attacker’s own router or switch. But the protocol is enormously more complex. It has layers for IP, TCP, possibly TLS, and HTTP, each offering the possibility of an exploitable vulnerability. The modern system has a much larger attack surface, though it still looks to the attacker like a single interface point.
Bare-metal attack surface
For an attacker not physically present in the data center, the initial attack surface is the network into the server. This led to the “perimeter view” of security: Protect the entry points into the data center and nothing gets in. If the attacker cannot get in, it doesn’t matter what happens between systems on the inside. It worked well when the perimeter interfaces were simple (think dial-up), but fostered weaknesses on internal interfaces. Attackers who found a hole in the perimeter would often discover that the internal attack surface of the server farm was much larger than the external one, and they could do considerable damage once inside.
This internal attack surface included network connections between servers but also process-to-process interactions within a single server. Worse, since many services run with elevated privileges (“root” user), successfully breaking into one would effectively mean unfettered access to anything else on that system, without having to look for additional vulnerabilities. A whole industry grew up around protecting servers -- firewalls, antimalware, intrusion detection, and on and on -- with less than perfect results.
There are also interesting “side channel” attacks against servers. Researchers have shown examples of using power consumption, noise, or electromagnetic radiation from computers to extract information, sometimes very sensitive data such as cryptographic keys. Other attacks have leveraged exposed interfaces like wireless keyboard protocols. In general, however, these attacks are more difficult -- they might require proximity to the server, for example -- so the main path of coming “down the wire” is more common.
VM attack surface
When VMs are used in the same way as bare metal, without any difference in the architecture of the application (as they often are), they share most of the same attack points. One additional attack surface is potential failure in the hypervisor, OS, or hardware to properly isolate resources between VMs, allowing a VM to somehow read the memory of another VM. The interface between the VM and the hypervisor also represents an attack point. If a VM can break through and get arbitrary code running in the hypervisor, then it can access other VMs on the same system. The hypervisor itself represents a point of attack since it exposes management interfaces.
There are additional attack points depending on the type of VM system. Type 2 VM systems use a hypervisor running as a process on an underlying host OS. These systems can be attacked by attacking the host OS. If the attacker can get code running on the host system, he can potentially affect the hypervisor and VMs, especially if he can get access as a privileged user. The presence of an entire OS, including utilities, management tools, and possibly other services and entry points (such as SSH) provides a number of possible attack points. Type 1 VM systems, where the hypervisor runs directly on the underlying hardware, eliminate these entry points and therefore have a smaller attack surface.
Container attack surface
As with VMs, containers share the fundamental network entry attack points of bare-metal systems. In addition, like Type 2 virtual machines, container systems that use a “fully loaded” host OS are subject to all of the same attacks available against the utilities and services of that host OS. If the attacker can gain access to that host, he can try to access or otherwise affect the running containers. If he gets privileged (“root”) access, the attacker will be able to access or control any container. A “minimalist” OS (such as Apcera’s KurmaOS) can help reduce this attack surface but cannot eliminate it entirely, since some access to the host OS is required for container management.
The basic container separation mechanisms (namespaces) also offer potential attack points. In addition, not all aspects of processes on Linux systems are namespaced, so some items are shared across containers. These are natural areas for attackers to probe. Finally, the process to kernel interface (for system calls) is large and exposed in every container, as opposed to the much smaller interface between a VM and the hypervisor. Vulnerabilities in system calls can offer potential access to the kernel. One example of this is the recently reported vulnerability in the Linux key ring.
Architectural considerations
For both VMs and containers, the size of the attack surface can be affected by the application architecture and how the technology is used.
Many legacy VM applications treat VMs like bare metal. In other words, they have not adapted their architectures specifically for VMs or for security models not based on perimeter security. They might install many services on the same VM, run the services with root privileges, and have few or no security controls between services. Rearchitecting these applications (or more likely replacing them with newer ones) might use VMs to provide security separation between functional units, rather than simply as a means of managing larger numbers of machines.
Containers are well suited for microservices architectures that “string together” large numbers of (typically) small services using standardized APIs. Such services often have a very short lifetime, where a containerized service is started on demand, responds to a request, and is destroyed, or where services are rapidly ramped up and down based on demand. That usage pattern is dependent on the fast instantiation that containers support. From a security perspective it has both benefits and drawbacks.
The larger number of services means a larger number of network interfaces and hence a larger attack surface. However, it also allows for more controls at the network layer. For example, in the Apcera Platform, all container-to-container traffic must be explicitly permitted. A rogue container cannot arbitrarily reach out to any network endpoint.
Short container lifetime means that if an attacker does get in, the time he has to do something is limited, as opposed to the window of opportunity presented by a long-running service. The downside is that forensics are harder. Once the container is gone, it cannot be probed and examined to find the malware. These architectures also make it more difficult for an attacker to install malware that survives after container destruction, as he might on bare metal by installing a driver that loads on boot. Containers are usually loaded from a trusted, read-only repository, and they can be further secured by cryptographic checks.
Now let’s consider what goes on during a breach.
Protection against breaches
Attackers typically have one or two goals in cracking into a server system. They want to get data or to do damage.
If they’re after data, they want to infiltrate as many systems as possible, with the highest privileges possible, and maintain that access for as long as possible. Achieving this gives them time to find the data, which might be already there -- a poorly secured database, for example -- or might require slow collection over time as it trickles in, such as collecting transactions as they come in from users. Maintaining access for a long time requires stealth. The attack also requires a way to get the data out.
If the attacker is trying simply to do damage, the goal again is to access as many systems and privileges as possible. But there is a balancing act: Once the damage starts it will presumably be noticed, but the longer the attacker waits to start (while the malware filters from system to system), the greater the chance of being detected. Getting data out is less important than coordinated control of the malware. The idea is to infect as many systems as possible, then damage them at a synchronized point, either prearranged or on command.
Breaches involve a number of elements. Let’s look at each and see if VMs and containerized architectures can affect the attack surface for each.