Container Security
- Summary
-
Discussion
- Which are the basic security mechanisms that come with containers?
- Is it okay to run my process with root access within the container?
- What are the various security threats facing containers?
- What are some best practices to secure containers?
- Isn't it better to use VMs instead of containers for better security?
- Which tools can help me secure my containers?
- Milestones
- References
- Further Reading
- Article Stats
- Cite As
With the wide adoption of container-based applications, systems became more complex and security risks increased. This laid the groundwork for container security. Vulnerabilities like dirty copy-on-write only furthered this thinking. This led to a shift left in security along the software development lifecycle, making it a key part of each stage in container app development, also known as DevSecOps. The goal is to build secure containers from the ground up without reducing time to market.
DevOps practice has enabled faster release cycles. At runtime, an application may dynamically scale to hundreds or even thousands of containers. It's impossible to have manual policies to secure these containers. Moreover, a production environment involves many layers such as images, registries, orchestrators and container runtimes. Hence, traditional security practices can't be directly applied to securing containers.
Discussion
-
Which are the basic security mechanisms that come with containers? In terms of filesystem, networking and processes, containers are isolated from one another using namespaces. To control access to resources such as CPU and RAM, there's cgroups.
To control kernel calls from a container, we use Linux kernel features called capabilities and seccomp. Capabilities provides fine-grained access control such as denying "mount" operations or access to raw sockets. Even if someone gains root access within a container, the damage they can do is limited. Secure computing mode (seccomp) disables access to system calls except those allowed. Both these adopt a whitelist approach.
Red Hat distributions comes with SELinux while Debian distributions come with AppArmor. Both these are security modules that provide access control by implementing Mandatory Access Control (MAC). Only users and processes with sufficient permissions can perform various actions. The access policies can't be changed by users.
We can also run the kernel with GRSEC and PAX to provide extra compile-time and runtime safety checks.
-
Is it okay to run my process with root access within the container? Some processes running on typical servers run with root privileges. Examples are SSH daemon, cron daemon, logging daemons, kernel modules, and network configuration tools. It may therefore appear that these processes need to run with root privileges inside a container. In reality, such tasks are handled by the infrastructure surrounding the container such as the host machine or the container platform. Even when a specific process needs root privilege, limit access using capabilities.
In general, stick to the principle of least privilege. Don't give more permissions than needed. One way to implement this is to use known user ID and group ID that have the minimum required privileges. An alternative available in Docker is to use a command-line option; example,
docker run --user 1001 ...
, which avoids root access.Most processes inside containers are app services and should not require root access. Docker itself requires root access but not the containers.
-
What are the various security threats facing containers? NIST has identified key threats to containers:
- Image: An image may lack security updates or become outdated, particularly when security bugs have been discovered after the image was created. Image may contain a misconfiguration, such as running a command with a greater privilege than needed. Image may include malware, particularly when a base layer comes from a third party or untrusted source. Secrets such as SSH private keys may be in the image as clear texts.
- Registry: Images stored at registries may be stale. Registries themselves can be compromised. Pulling images over insecure connections is also an issue.
- Orchestrator: If access privileges are not properly scoped, users can access and affect containers that don't belong to them. Wrong configurations may allow unauthorized nodes to join a cluster.
- Container: At runtime, a malicious software can affect other containers or the host OS. App vulnerabilities can affect the container. Unplanned or unsanctioned containers, especially in a development environment, are a risk.
- Host: The host itself has an attack surface. Because the kernel is shared, runtime isolation is not as high as that of hypervisors.
-
What are some best practices to secure containers? We list some industry best practices to secure containers:
- Called image provenance, have a system such as Docker Content Trust to identify the source of an image that's running in production.
- Containers must run with minimum resources and privileges. Use AppArmor or SELinux to enforce access control. Perhaps the most important advice is to not use root privilege to run the container's main process.
- At runtime, detect threats and respond to the same automatically.
- Vulnerability scanning, regular auditing, and software updates must be automated.
- Since containers are not permanent, traditional operational models may not work. Organizations may have to formalize new models.
- Use Docker Bench for Security to know if you're following best practices around container deployment.
- Minimize the use of third-party images and use only signed images.
- Mount the host filesystem as read-only or disable UNIX sockets, thus limiting access to an attacker.
-
Isn't it better to use VMs instead of containers for better security? Containers offer only software-based isolation using namespaces and cgroups. Multiple containers share the same kernel. If one container is compromised, the attacker can potentially gain access to other containers on the host. This is where the hypervisor layer of VMs provides hardware-enforced isolation.
But it's possible to mix both VMs and containers. By running multiple containers inside a VM, we get the performance of containers without sacrificing the security of hypervisor. It's common among cloud providers to isolate tenants using VMs. For greater security, we can use VMs to separate containers that need to run at different security levels. For example, a billing service may run at a higher security level compared to a user-facing web service.
Joyent and IBM isolate tenants by running containers on bare metal without using VMs. Hyper, Intel and VMware are building fast VM-based frameworks with the aim of getting the best of both worlds. The idea is to secure a container without the bloat of a VM. This is being called Kata Containers. It uses runV container runtime.
-
Which tools can help me secure my containers? While container platforms and containers themselves offer a basic level of security, this isn't enough. There's a need to scan container images or monitor a running container. There are many vendors offering products that do these. Some of them are Alert Logic, Anchore, Aporeto, Aqua Security, Capsule8, NeuVector, Qualys, StackRox, Sysdig, and Twistlock.
Any good tool should have visibility of containers, images and host processes. It should validate container signatures. It should alert when there are image drifts. It should monitor slaves and masters of Swarm or Kubernetes clusters. Tool should have an easy-to-use dashboard. It should make use of image metadata to enable filtering by labels and tags. It should classify threats by severity and priority. Select a tool that integrates well with your CI/CD pipeline.
Google's Container Registry vulnerability scanning, Red Hat's Atomic Scan, IBM's Bluemix Vulnerability Advisor, CoreOS Clair, Docker Security Scanning, Aqua Security's Peekr, and Twistlock Trust can do security scanning. Runtime threat detection and response are offered by Aqua Security, Joyent Triton SmartOS, Twistlock and Red Hat OpenShift.
Milestones
2015
Docker Content Trust (DCT) is released as a new feature in Docker Engine 1.8. This adds a digital signature to an image before it's pushed to the registry. When images are pulled, the signatures are verified. Through the implementation of The Update Framework (TUF), rollback attacks and freeze attacks are also prevented.
2016
2018
2019
Two researchers discover a way to escape from a container into the host system and obtain root privileges. This is possible with a compromised container even if it's using the default or hardened configuration. The exploit overwrites docker-runc
binary on the host with malicious software. This bug in runc, called CVE-2019-5736, affects Docker and other container platforms. A patch is released in February.
References
- Ali Babar, Muhammad and Ben Ramsey. 2017. "Understanding Container Isolation Mechanisms for Building Security-Sensitive Private Cloud." Centre for Research on Engineering Software Technologies, University of Adelaide, via ResearchGate, January. Accessed 2019-08-09.
- Campbell, Marc. 2017. "Processes In Containers Should Not Run As Root." Medium, September 28. Accessed 2019-08-09.
- Docker Docs. 2019a. "Docker security." Docker v19.03, August 02. Accessed 2019-08-09.
- Docker Docs. 2019b. "Seccomp security profiles for Docker." Docker v19.03, August 02. Accessed 2019-08-09.
- Firshman, Ben. 2015. "Announcing Docker 1.8: Content Trust, Toolbox, and Updates to Registry and Orchestration." Blog, Docker, August 12. Accessed 2019-08-09.
- Fulton III, Scott M. 2017. "Kata Containers: Secure, Lightweight Virtual Machines for Container Environments." The New Stack, December 18. Accessed 2019-08-09.
- Grattafiori, Aaron. 2016. "Understanding and Hardening Linux Containers." NCC Group Whitepaper, Version 1.1, June 29. Accessed 2019-08-09.
- HyTrust. 2019. "HyTrust CloudControl 6.0 Container Edition." Accessed 2019-08-09.
- Iwaniuk, Adam, and Borys Popławski. 2019. "CVE-2019-5736: Escape from Docker and Kubernetes containers to root on host." Blog, Dragon Sector, February 13. Accessed 2019-08-09.
- Jennings, Richi. 2019. "Hackers love Docker: Container catastrophe in 3, 2, 1..." TechBeacon, February 14. Accessed 2019-08-09.
- Kaul, Nikhil and Juan Sebastian Oviedo. 2018. "Guard against security vulnerabilities in your software supply chain with Container Registry vulnerability scanning." Blog, Google Cloud, September 20. Accessed 2019-08-09.
- Kerner, Sean Michael. 2018. "10 Top Container and Kubernetes Security Vendors." eSecurity Planet, December 26. Accessed 2019-08-09.
- Lawrence, Gabriel. 2016. "Dirty COW - (CVE-2016-5195) - Docker Container Escape." Blog, Paranoid Software, October 26. Accessed 2019-08-09.
- McAllister, Neil. 2018. "Container security: What you need to know about the NIST standards." TechBeacon, May 04. Accessed 2019-08-09.
- Mouat, Adrian. 2016. "Assessing the Current State of Container Security." The New Stack, September 12. Accessed 2019-08-09.
- Prevent Breach. 2018. "Container Security: What to Look For." Prevent Breach, via YouTube, June 27. Accessed 2019-08-09.
- Shoenberger, John. 2019. "Docker Security Tips & Best Practices." Threat Stack, January 03. Accessed 2019-08-09.
- Souppaya, Murugiah, John Morello, and Karen Scarfone. 2017. "Application Container Security Guide." NIST Special Publication 800-190, September. Accessed 2019-08-09.
- Tozzi, Chris. 2017. "The Ultimate Guide to Container Security." Twistlock, July 06. Accessed 2019-08-09.
- Trend Micro. 2019. "Container Security: Examining Potential Threats to the Container Environment." Trend Micro, May 14. Accessed 2019-08-09.
Further Reading
- Huang, Fei. 2016. "15 Tips for a Run-time Container Security Strategy." NeuVector, October 11. Updated 2018-03-22. Accessed 2019-08-09.
- Trend Micro. 2019. "Container Security: Examining Potential Threats to the Container Environment." Trend Micro, May 14. Accessed 2019-08-09.
- RedHat. 2019. "Ten layers of container security." Resources, RedHat. Accessed 2019-08-09.
- Docker. 2016. "Introduction to Container Security." White paper, Docker, August. Accessed 2019-08-09.
- Grattafiori, Aaron. 2016. "Understanding and Hardening Linux Containers." NCC Group Whitepaper, Version 1.1, June 29. Accessed 2019-08-09.
Article Stats
Cite As
See Also
- Containerization
- Container Orchestration
- DevSecOps
- Three Rs of Security
- Cloud Security
- Cloud Monitoring