Linux 容器技术史话：从 chroot  到未来
容器 docker
![](/data/attachment/album/201602/04/001041qaejsssu4lfk49f5.jpg)


Linux 容器是一个在单一 Linux 主机上提供多个隔离的 Linux 环境的[操作系统级虚拟](https://en.wikipedia.org/wiki/Operating-system-level_virtualization#IMPLEMENTATIONS)技术。不像虚拟机（VM），容器并不需要运行专用的<ruby> 访客 <rp>  （ </rp> <rt>  guest </rt> <rp>  ） </rp></ruby>操作系统。容器们共享<ruby> 宿主机的 <rp>  （ </rp> <rt>  host </rt> <rp>  ） </rp></ruby>操作系统内核，并使用访客操作系统的系统库来提供所需的功能。由于不需要专用的操作系统，因此容器要比虚拟器启动快得多。


![](/data/attachment/album/201602/04/182525u1sn21xzr8lln78r.jpg)


（图片来自: Docker Inc.）


容器借助 Linux 内核的 Namespaces、Apparmor、SELinux <ruby> 情景模式 <rp>  （ </rp> <rt>  profile </rt> <rp>  ） </rp></ruby>、chroot 和 CGroup 等功能来提供类似于虚拟机的隔离环境。Linux 的安全模块可以确保正确地控制容器对宿主机和内核的访问，从而避免各种入侵活动。此外，在宿主机上可以运行不同的 Linux 发行版，只要它们运行在同样的 CPU 架构下。


简单来说，容器提供的是一种基于各种 Linux 发行版创建容器镜像的方法、一套管理容器生命周期的 API、与该 API 交互的客户端工具、保存快照的功能、在宿主机之间迁移容器实例的能力，等等。


### 容器历史


以下是容器简史，内容来自维基百科及其它来源：


#### 1979 — chroot


容器的概念始于 1979 年的 UNIX [chroot](https://en.wikipedia.org/wiki/Chroot)，它是一个 UNIX 操作系统上的系统调用，用于将一个进程及其子进程的根目录改变到文件系统中的一个新位置，让这些进程只能访问到该目录。这个功能的想法是为每个进程提供独立的磁盘空间。其后在 1982年，它被加入到了 BSD 系统中。


#### 2000 — FreeBSD Jails


[FreeBSD Jails](https://en.wikipedia.org/wiki/FreeBSD_jail) 是最早的容器技术之一，它由 R&D Associates 公司的 Derrick T. Woolworth 在 2000 年为 FreeBSD 引入。这是一个类似 chroot 的操作系统级的系统调用，但是为文件系统、用户、网络等的隔离增加了进程沙盒功能。因此，它可以为每个 jail 指定 IP 地址、可以对软件的安装和配置进行定制，等等。


#### 2001 — Linux VServer


[Linux VServer](https://en.wikipedia.org/wiki/Linux-VServer) 是另外一种 jail 机制，它用于对计算机系统上的资源（如文件系统、CPU 处理时间、网络地址和内存等）进行安全地划分。每个所划分的分区叫做一个<ruby> 安全上下文 <rp>  （ </rp> <rt>  security context </rt> <rp>  ） </rp></ruby>，在其中的虚拟系统叫做<ruby> 虚拟私有服务器 <rp>  （ </rp> <rt>  virtual private server，VPS </rt> <rp>  ） </rp></ruby>。


#### 2004 — Solaris Containers


[Solaris Containers](https://en.wikipedia.org/wiki/Solaris_Containers) 支持在 x86 和 SPARC 系统，首次出现在 2004 年 2 月发布的 Solaris 10 的 build 51 beta 上，其后完整发布在 2005 年的 Solaris 10 上。 Solaris Container 是由系统资源控制和通过 zones 提供的<ruby> 边界分离 <rp>  （ </rp> <rt>  boundary separation </rt> <rp>  ） </rp></ruby>所组合而成的。zones 是一个单一操作系统实例中的完全隔离的虚拟服务器。


#### 2005 — OpenVZ


[OpenVZ](https://en.wikipedia.org/wiki/OpenVZ) 类似于 Solaris Containers，它通过对 Linux 内核进行补丁来提供虚拟化、隔离、资源管理和<ruby> 状态检查 <rp>  （ </rp> <rt>  checkpointing </rt> <rp>  ） </rp></ruby>。每个 OpenVZ 容器都有一套隔离的文件系统、用户及用户组、进程树、网络、设备和 IPC 对象。


#### 2006 — Process Containers


[Process Containers](https://en.wikipedia.org/wiki/Cgroups) 是由 Google 在 2006 年实现的，用于对一组进程进行限制、记账、隔离资源使用（CPU、内存、磁盘 I/O、网络等）。后来为了避免和 Linux 内核上下文中的“容器”一词混淆而改名为 Control Groups。它被合并到了 2.6.24 内核中。这表明 Google 很早就参与了容器技术的开发，以及它们是如何回馈到社区的。


#### 2007 — Control Groups


如上面所述，Control Groups （即 cgroups）是由 Google 实现的，并于 2007 年加到了 Linux 内核中。


#### 2008 — LXC


[LXC](https://en.wikipedia.org/wiki/LXC) 的意思是 LinuX Containers，它是第一个最完善的 Linux 容器管理器的实现方案，是通过 cgroups 和 Linux <ruby> 名字空间 <rp>  （ </rp> <rt>  namespace </rt> <rp>  ） </rp></ruby>实现的。LXC 存在于 liblxc 库中，提供了各种编程语言的 API 实现，包括 Python3、Python2、Lua、Go、Ruby 和 Haskell。与其它容器技术不同的是， LXC 可以工作在普通的 Linux 内核上，而不需要增加补丁。现在 [LXC project](https://linuxcontainers.org/lxc/introduction/) 是由 Canonical 公司赞助并托管的。


#### 2011 — Warden


[Warden](https://docs.cloudfoundry.org/concepts/architecture/warden.html) 是由 CloudFoundry 在 2011 年开发的，开始阶段是使用的 LXC，之后替换为他们自己的实现方案。不像 LXC，Warden 并不紧密耦合到 Linux 上，而是可以工作在任何可以提供隔离环境的操作系统上。它以后台守护进程的方式运行，为容器管理提供了 API。请参考 [Warden 文档](https://docs.cloudfoundry.org/concepts/architecture/warden.html)和[这个博客文章](http://blog.altoros.com/cloud-foundry-containers-warden-docker-garden.html)了解更多。


#### 2013 — LMCTFY


[lmctfy](https://github.com/google/lmctfy) 的意思是“<ruby> 让我为你包含 <rp>  （ </rp> <rt>  Let Me Contain That For You </rt> <rp>  ） </rp></ruby>”。这是一个 Google 容器技术的开源版本，提供 Linux 应用容器。Google 启动这个项目旨在提供性能可保证的、高资源利用率的、资源共享的、可超售的、接近零消耗的容器（参考自：[lmctfy 演讲稿](http://www.linuxplumbersconf.org/2013/ocw//system/presentations/1239/original/lmctfy%20(1).pdf)）。现在为 Kubernetes 所用的 cAdvisor 工具就是从 lmctfy 项目的成果开始的。lmctfy 首次发布于 2013 年10月，在 2015 年 Google 决定贡献核心的 lmctfy 概念，并抽象成 libcontainer，因此，lmctfy 现在已经没有活跃的开发了。


lincontainer 项目最初由 [Docker](https://github.com/docker/libcontainer) 发起，现在已经被移交给了<ruby> <a href="https://github.com/opencontainers/runc/tree/master/libcontainer">  开放容器基金会 </a> <rp>  （ </rp> <rt>  Open Container Foundation </rt> <rp>  ） </rp></ruby>。


#### 2013 — Docker


[Docker](https://en.wikipedia.org/wiki/Docker_(software)) 是到现在为止最流行和使用广泛的容器管理系统。它最初是一个叫做 dotCloud 的 PaaS 服务公司的内部项目，后来该公司改名为 Docker。类似 Warden，Docker 开始阶段使用的也是 LXC ，之后采用自己开发的 libcontainer 替代了它。不像其它的容器平台，Docker 引入了一整个管理容器的生态系统，这包括高效、分层的容器镜像模型、全局和本地的容器注册库、清晰的 REST API、命令行等等。稍后的阶段， Docker 推动实现了一个叫做 Docker Swarm 的容器集群管理方案。


#### 2014 — Rocket


[Rocket](https://coreos.com/blog/rocket/) 是由 CoreOS 所启动的项目，非常类似于 Docker，但是修复了一些 Docker 中发现的问题。CoreOS 说他们的目的是提供一个比 Docker 更严格的安全性和产品需求。更重要的是，它是在一个更加开放的标准 App Container 规范上实现的。在 Rocket 之外，CoreOS 也开发了其它几个可以用于 Docker 和 Kubernetes的容器相关的产品，如：[CoreOS 操作系统](https://en.wikipedia.org/wiki/CoreOS)、[etcd](https://coreos.com/etcd/) 和 [flannel](https://github.com/coreos/flannel)。


#### 2016 — Windows Containers


微软 2015 年也在 Windows Server 上为基于 Windows 的应用添加了容器支持，它称之为 [Windows Containers](https://msdn.microsoft.com/en-us/virtualization/windowscontainers/about/about_overview)。它与 Windows Server 2016 一同发布。通过该实现， Docker 可以原生地在 Windows 上运行 Docker 容器，而不需要启动一个虚拟机来运行 Docker（ Windows 上早期运行 Docker 需要使用 Linux 虚拟机）。


### 容器的未来


截止到今天（2016年1月），在行业内有一个显著趋势，部署软件应用从虚拟机逐渐移到了容器。其主要的原因是容器相比于虚拟机而言更加灵活和低消耗。Google 已经使用容器技术好多年了，它在 [Borg 和 Omega](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf) 容器集群管理平台上可以成规模地运行 Google 应用。更重要的是，Google 为容器领域贡献了 cgroups 的实现和参与了 libcontainer 项目。Google 也在过去这些年借助容器在性能、资源利用和整体效率方面取得了巨大收益。最近，一直没有操作系统级的虚拟化技术的微软，也在 Window Server 上迅速采取动作实现了对容器的原生支持。


Docker 、Rocket 以及其它的容器平台并不能以一个单一主机运行在产品环境中，原因是这样面临着单点故障。当一组容器运行在一个单一宿主机时，如果宿主机失效，所有运行在该宿主机上的容器也会失效。要避免这个问题，应该使用容器宿主机集群。Google 借助其在 Borg 中取得的经验，开发了一个叫做 [Kubernetes](http://kubernetes.io/) 的开源容器集群管理系统。Docker 也启动了一个叫做 [Docker Swarm](https://docs.docker.com/swarm/) 的解决方案。目前这些解决方案还都处于相当早期的阶段，也许需要几个月或来年才实现完整的功能集合，才能变得稳定，从而广泛应用于行业内的产品环境中。


<ruby> <a href="http://martinfowler.com/articles/microservices.html">  微服务 </a> <rp>  （ </rp> <rt>  Microservices </rt> <rp>  ） </rp></ruby>是另一个突破性技术，在软件架构上可以将容器用于部署。微服务并不是一个新东西，只是一个相比标准的 Web 服务超快的轻量级 Web 服务。这是通过将功能单元（也许是一个单一服务或 API 方法）打包到一个服务中，并内嵌其到一个轻量级 Web 服务器软件中实现的。


通过对上面的介绍，我们可以预测接下来几年，容器也许会替代虚拟机，某些情况下也许会全部取代。去年我为一些企业在 POC 层面上实现了基于容器的解决方案。也有人想要在产品环境中尝试它们，这一天也许在容器集群管理系统进一步成熟后很快到来。


OK
# The Evolution of Linux Containers and Their Future

### A history of containerization technology starting in 1979, and what the future holds for Docker and similar technologies.

Join the DZone community and get the full member experience.

[Join For Free](/static/registration.html)

Linux containers are an [operating system level virtualization](https://en.wikipedia.org/wiki/Operating-system-level_virtualization#IMPLEMENTATIONS) technology for providing multiple isolated Linux environments on a single Linux host. Unlike virtual machines (VMs), containers do not run dedicated guest operating systems. Rather, they share the host operating system kernel and make use of the guest operating system system libraries for providing the required OS capabilities. Since there is no dedicated operating system, containers start much faster than VMs.

*Image credit: Docker Inc.*

Containers make use of Linux kernel features such as Namespaces, Apparmor, SELinux profiles, chroot, and CGroups for providing an isolated environment similar to VMs. Linux security modules guarantee that access to the host machine and the kernel from the containers is properly managed to avoid any intrusion activities. In addition containers can run different Linux distributions from its host operating system if both operating systems can run on the same CPU architecture.

In general, containers provide a means of creating container images based on various Linux distributions, an API for managing the lifecycle of the containers, client tools for interacting with the API, features to take snapshots, migrating container instances from one container host to another, etc.

## Container History

Below is a short summary of container history extracted from Wikipedia and [other sources](http://pivotal.io/platform/infographic/moments-in-container-history):

### 1979 — chroot

The concept of containers was started way back in 1979 with UNIX [chroot](https://en.wikipedia.org/wiki/Chroot). It’s an UNIX operating-system system call for changing the root directory of a process and it's children to a new location in the filesystem which is only visible to a given process. The idea of this feature is to provide an isolated disk space for each process. Later in 1982 this was added to BSD.

### 2000 — FreeBSD Jails

[FreeBSD Jails](https://en.wikipedia.org/wiki/FreeBSD_jail) is one of the early container technologies introduced by Derrick T. Woolworth at R&D Associates for FreeBSD in year 2000. It is an operating-system system call similar to chroot, but included additional process sandboxing features for isolating the filesystem, users, networking, etc. As a result it could provide means of assigning an IP address for each jail, custom software installations and configurations, etc.

### 2001 — Linux VServer

[Linux VServer](https://en.wikipedia.org/wiki/Linux-VServer) is a another jail mechanism that can be used to securely partition resources on a computer system (file system, CPU time, network addresses and memory). Each partition is called a security context, and the virtualized system within it is called a virtual private server.

### 2004 — Solaris Containers

[Solaris Containers](https://en.wikipedia.org/wiki/Solaris_Containers) were introduced for x86 and SPARC systems, first released publicly in February 2004 in build 51 beta of Solaris 10, and subsequently in the first full release of Solaris 10, 2005. A Solaris Container is a combination of system resource controls and the boundary separation provided by zones. Zones act as completely isolated virtual servers within a single operating system instance.

### 2005 — OpenVZ

[OpenVZ](https://en.wikipedia.org/wiki/OpenVZ) is similar to Solaris Containers and makes use of a patched Linux kernel for providing virtualization, isolation, resource management, and checkpointing. Each OpenVZ container would have an isolated file system, users and user groups, a process tree, network, devices, and IPC objects.

### 2006 — Process Containers

[Process Containers](https://en.wikipedia.org/wiki/Cgroups) was implemented at Google in year 2006 for limiting, accounting, and isolating resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. Later on it was renamed to Control Groups to avoid the confusion multiple meanings of the term “container” in the Linux kernel context and merged to the Linux kernel 2.6.24. This shows how early Google was involved in container technology and how they have contributed back.

### 2007 — Control Groups

As explained above, Control Groups AKA cgroups was implemented by Google and added to the Linux Kernel in 2007.

### 2008 — LXC

[LXC](https://en.wikipedia.org/wiki/LXC) stands for LinuX Containers and it is the first, most complete implementation of Linux container manager. It was implemented using cgroups and Linux namespaces. LXC was delivered in liblxc library and provided language bindings for the API in Python3, Python2, Lua, Go, Ruby, and Haskell. Contrast to other container technologies LXC works on vanila Linux kernel without requiring any patches. Today [LXC project](https://linuxcontainers.org/lxc/introduction/) is sponsored by Canonical Ltd. and hosted here.

### 2011 — Warden

[Warden](https://docs.cloudfoundry.org/concepts/architecture/warden.html) was implemented by CloudFoundry in year 2011 by using LXC at the initial stage and later on replaced with their own implementation. Unlike LXC, Warden is not tightly coupled to Linux. Rather, it can work on any operating system that can provide ways of isolating environments. It runs as a daemon and provides an API for managing the containers. Refer to [Warden documentation](https://docs.cloudfoundry.org/concepts/architecture/warden.html) and [this blog post](http://blog.altoros.com/cloud-foundry-containers-warden-docker-garden.html) for more detailed information on Warden.

### 2013 — LMCTFY

[lmctfy](https://github.com/google/lmctfy) stands for “Let Me Contain That For You”. It is the open source version of Google’s container stack, which provides Linux application containers. Google started this project with the intention of providing guaranteed performance, high resource utilization, shared resources, over-commitment, and near zero overhead with containers (Ref: [lmctfy presentation](http://www.linuxplumbersconf.org/2013/ocw//system/presentations/1239/original/lmctfy%20(1).pdf)). The cAdvisor tool used by Kubernetes today was started as a result of lmctfy project. The initial release of lmctfy was made in Oct 2013 and in year 2015 Google has decided to contribute core lmctfy concepts and abstractions to libcontainer. As a result now no active development is done in LMCTFY.

The libcontainer project was initially started by [Docker](https://github.com/docker/libcontainer) and now it has been moved to [Open Container Foundation](https://github.com/opencontainers/runc/tree/master/libcontainer).

### 2013 — Docker

[Docker](https://en.wikipedia.org/wiki/Docker_(software)) is the most popular and widely used container management system as of January 2016. It was developed as an internal project at a platform-as-a-service company called dotCloud and later renamed to Docker. Similar to Warden, Docker also used LXC at the initial stages and later replaced LXC with it’s own library called libcontainer. Unlike any other container platform, Docker introduced an entire ecosystem for managing containers. This includes a highly efficient, layered container image model, a global and local container registries, a clean REST API, a CLI, etc. At a later stage, Docker also took an initiative to implement a container cluster management solution called Docker Swarm.

### 2014 — Rocket

[Rocket](https://coreos.com/blog/rocket/) is a much similar initiative to Docker started by CoreOS for fixing some of the drawbacks they found in Docker. CoreOS has mentioned that their aim is to provide more rigorous security and production requirements than Docker. More importantly, it is implemented on App Container specifications to be a more open standard. In addition to Rocket, CoreOS also develops several other container related products used by Docker and Kubernetes: [CoreOS Operating System](https://en.wikipedia.org/wiki/CoreOS), [etcd](https://coreos.com/etcd/), and [flannel](https://github.com/coreos/flannel).

### 2016 — Windows Containers

Microsoft also took an initiative to add container support to the Microsoft Windows Server operating system in 2015 for Windows based applications, called [Windows Containers](https://msdn.microsoft.com/en-us/virtualization/windowscontainers/about/about_overview). This is to be released with Microsoft Windows Server 2016. With this implementation Docker would be able to run Docker containers on Windows natively without having to run a virtual machine to run Docker (earlier Docker ran on Windows using a Linux VM).

## The Future of Containers

As of today (Jan 2016) there is a significant trend in the industry to move towards containers from VMs for deploying software applications. The main reasons for this are the flexibility and low cost that containers provide compared to VMs. Google has used container technology for many years with [Borg and Omega](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf) container cluster management platforms for running Google applications at scale. More importantly, Google has contributed to container space by implementing cgroups and participating in libcontainer projects. Google may have gained a huge gain in performance, resource utilization, and overall efficiency using containers during past years. Very recently Microsoft, who did not had an operating system level virtualization on the Windows platform took immediate action to implement native support for containers on Windows Server.

Docker, Rocket, and other container platforms cannot run on a single host in a production environment, the reason is that they are exposed to single point of failure. While a collection of containers are run on a single host, if the host fails, all the containers that run on that host will also fail. To avoid this, a container host cluster needs to be used. One of the first most open source container cluster management platforms to solve this problem was [Apache Mesos](https://en.wikipedia.org/wiki/Apache_Mesos). It was initially developed at University of California, Berkeley as a research project and later moved to Apache in around year 2012. Google took a similar step to implement a cutting edge, open source container cluster management system called [Kubernetes](http://kubernetes.io/) in year 2014 with the experience they got from Borg. Docker also started a solution called [Docker Swarm](https://docs.docker.com/swarm/) in year 2015. Today these solutions are at their very early stages and it may take several months and may be another year to complete their full feature set, become stable and widely used in the industry in production environments.

[Microservices](http://martinfowler.com/articles/microservices.html) are another groundbreaking technology rather a software architecture which uses containers for their deployment. A microservice is nothing new but a lightweight implementation of a web service which can start extremely fast compared to a standard web service. This is done by packaging a unit of functionality (may be a single service/API method) in one service and embedding it into a lightweight web server binary.

By considering the above facts we can predict that in next few years, containers may take over virtual machines, and sometimes might replace them completely. Last year I worked with a handful of enterprises on implementing container-based solutions on a POC level. There were few who wanted to take the challenge and put them in production. This may change very quickly as the container cluster management systems get more mature.

Opinions expressed by DZone contributors are their own.

Comments