LXC (Linux Containers) is the new flagship of container-based virtualization on Linux. Although being around for quite some time, LXC is still not feature complete which leads to many people still using OpenVZ. Mostly based on Michael Renner’s work, here are the features that are missing and those already complete in a nice overview. The data represented here takes Ubuntu 12.04 as a basis, as it is the distribution that supports LXC best.
Fresh up! Read my LXC tutorial!
|Comes with Linux kernel||No||Yes|
|Limiting memory usage||Yes||Yes|
|Limiting kernel memory usage||Yes||No|
|Limiting CPU usage||Yes||Yes|
|Limiting disk usage||Yes||Partial/Workaround|
|Limiting disk IO||No||Yes|
|Container lockdown (security)||Yes||Partial|
Comes with Linux kernel
LXC comes with the Linux kernel whereas OpenVZ does not, although OpenVZ uses many facilities that are present in the kernel no small part to their contribution. Installing LXC is much easier, because you don’t need to build a custom kernel. If you are however using Redhat, SuSE or the likes (CentOS, etc), you can use the prebuilt OpenVZ kernel images for your project. Be aware that OpenVZ patches only apply to quite old kernels,
so there definitely is a security risk so you may not be able to get them to work. Newer kernels are not supported.
On a side note, OpenVZ is now in the process of porting their much userspace utilities to the LXC base in the Linux kernel, so we will definitely see some improvement there in the future. As Kir Kolyshkin points out in the comments, vzctl 4.3 supports non-OpenVZ containers and even live migration through CRIU.
Limiting memory usage
Limiting user space memory usage works both in LXC and OpenVZ. In LXC you can use two cgroup settings in the VM-s configuration file to adjust the guest’s allowed memory. The most important settings are lxc.cgroup.memory.limit_in_bytes for memory and lxc.cgroup.memory.memsw.limit_in_bytes for swap size. Depending on your exact kernel version there are also some other settings worth exploring. You can list them by looking at /sys/fs/cgroup/memory/lxc/your-vm-name.
Limiting kernel memory
Kernel memory can be used by applications while they interact with the kernel. When using OpenVZ, that memory can be limited per VE. As Kir Kolyshkin pointed out in the comments, with LXC you are a bit out of luck. That feature is something we are to expect in the future.
Limiting CPU usage
Limiting CPU usage again works in LXC and OpenVZ. CPU bandwidth is limited by assigning shares. The more shares a VM has the more likely it is to get CPU time. You can set a VM’s CPU shares by adjusting the lxc.cgroup.cpu.shares setting. You can also limit the CPU cores of a VM using lxc.cgroup.cpuset.cpus
Limiting disk usage
Unfortunately there is no central place for disk quotas in Linux, therefore LXC doesn’t support disk quotas. Limits can be placed on VM’s by putting them on LVM volumes but this means less IO performance. Depending on your use case this may not be ideal.
Limiting disk IO
Limiting disk IO is important in scenarios, where you have IO hungry applications next to each other. Quite surprisingly this feature hasn’t made it into OpenVZ and is only available in the commercial Virtuzzo application. LXC however has this feature via the aforementioned cgroups. The settings for IO limiting are located under lxc.cgroup.blkio. For details please look into /sys/fs/cgroup/blkio/lxc.
Checkpointing is a feature very close to the traditional hibernation. The state of the VM is saved into a file and can be reloaded. This feature will be available using the lxc-checkpoint command, but is not implemented at the time of writing.
Live migration means that a VM can be migrated to another physical host without actually shutting down the VM itself. In other words, the memory contents are also moved to the new host. Where OpenVZ has had this feature for quite a long time, LXC still has parts missing as Michael indicates. Personally I hope that we will see this feature in the near future.
As a workaround you can use CRIU, which is a project that implements user-space live migration.
Container lockdown (security)
Isolation is an important part of virtualization. OpenVZ has done quite a good job at this, but LXC still has issues here. Even with AppArmor enabled, in Ubuntu you still have access to dmesg from the guests and /proc/kcore and /proc/sysrq-trigger are still accessible, so a root user in a guest VM could easily restart the host machine. Improvements are planned for Ubuntu version 13.04.
LXC is well on it’s way, but still not there. The most painful lack of features are the ones locking down the guest properly. Ubuntu has these features planned for 13.04 which is here in a few months, but as it’s not an LTS version, it isn’t going to make sysadmins happier.
To sum it all up, LXC is good if you want to use if for flexibility, but not quite adequate for hosting foreign VM’s yet. If you are in that business, you should really wait another year or so.