| |||
k001 | wiki.openvz.org is down today, joining the SOPA and PIPA protest. Read more: * http://americancensorship.org/ * https://www.eff.org/deeplinks/2012/01/h * http://google.com/takeaction The site will come alive tomorrow. Now, if you need access to the site, just reload the page. | ||
| |||
| |||
k001 | VSwap is an excellent feature, simplifying container resource management a lot. Now it's time to also simplify the command line interface, i.e. vzctl. Here is what we did recently (take a look at vzctl git repo if you want to review the actual changes): 1. We no longer require to set kmemsize, dcachesize and lockedpages parameters to non-unlimited values (this is one of the enhancements in the recent kernels we have talked about recently). Therefore, setting these parameters to fractions of CT RAM (physpages) are now removed from configuration files and vzsplit output. 2. There is no longer a need to specify all the UBC parameters in per-container configuration file. If you leave some parameters unset, the kernel will use default values (usually unlimited). So, VSwap example configs are now much smaller, with as much as 19 parameters removed from those. 3. vzctl set now supports two new options: --ram and --swap. These are just convenient short aliases for --physpages and --swappages, the differences being that you only need to specify one value (the limit) and the argument is in bytes rather than pages. So, to configure a container named MyCT to have 1.5GB of RAM and 3GB of swap space available, all you need to do is just run this command: vzctl set MyCT --ram 1.5G --swap 3G --save4. This is not VSwap-related, but nevertheless worth sharing. Let's illustrate it by a copy-paste from a terminal: # vzctl create 123 --ostemplate centos-4-x86_64All this will be available in vzctl-3.0.30, which is to be released soon (next week? who knows). If you can't wait and want to test this stuff, here's a nightly build of vzctl available (version 3.0.29.3-27.git.0535fe1) from http://wiki.openvz.org/Download/vzctl/n | ||
| |||
| |||
k001 | vSwapThe best feature of the new (RHEL6-based) 042 series of the OpenVZ kernels is definitely vSwap. The short story is, we used to have 22 user beancounter parameters which every seasoned OpenVZ user knows by heart. Each of these parameters is there for a reason, but 22 knobs are a bit too complex to manage for a mere mortal, especially bearing in mind that
We worked on that for a few years, and the end result is called vSwap (where V is for Vendetta, oh, pardon me, Virtual). vSwap concept is as simple as a rectangular. For each container, there are only two required parameters: the memory size (known as physpages) and the swap size (swappages). Almost everyone (not only an admin, but even an advanced end user) knows what is RAM and what is swap. On a physical server, if there is not enough memory, the system starts to swap out memory pages to disk, then swap in some other pages, which results in severe performance degradation but it keeps the system from failing miserably.It's about the same with vSwap, except that
Some VSwap internalsNow, there are only two knobs per container on a dashboard, namely RAM and swap, and all the complexity is hidden under the hood. I am going to describe just a bit of that undercover mechanics and explain what does the "Reworked VSwap kernel memory accounting" line from the 042stab040.1 kernel changelog stands for.The biggest problem is, RAM for containers is not just RAM. First of all, there is a need to distinguish between
The kernel memory is really complex thingie. Right, it is the memory that kernel allocates for itself in order for programs in a particular container to run. This includes a lot of stuff I'd rather not dive into, if I want to keep this piece as an article not a tome. Having said that, two particular kernel memory types are worth explaining. First is the page cache, the kernel mechanism that caches disk contents in memory (that would be unused otherwise) to minimize the I/O. When a program reads some data from a disk, that data are read into the page cache first, and when a program writes to a disk, data goes to the page cache (and then eventually are written (flushed) to disk). In case of repeated disk access (which happens quite often) data is taken from a page cache, not from the real disk, which greatly improves the overall system performance, since a disk is much slower than RAM. Now, some of the page cache is used on behalf of a container, and this amount must be charged into "RAM used by this container" (i.e. physpages). Second is the directory entry cache (dcache for short) is yet another sort of cache, and another sort of the kernel memory. Disk contents is a tree of files and directories, and such a tree is quite tall and wide. In order to read the contents of, say, /bin/sh file, kernel have to read the root (/) directory, find 'bin' entry in it, read /bin directory, find 'sh' entry in it and finally read it. Although these operations are not very complex, there is a multitude of those, they take time and are repeated often for most of the "popular" files. In order to improve performance, kernel keeps directory entries in memory — this is what dcache is for. The memory used by dcache should also be accounted and limited, since otherwise it's easily exploitable (not only by root, but also by an ordinary user, since any user is free to change into directories and read files). Now, the physical memory of a container is the sum of its user memory, the kernel memory, the page cache and the dcache. Technically, dcache is accounted into the kernel memory, then kernel memory is accounted into the physical memory, but it's not overly important. Improvements in the new 042stab04x kernelsBetter reclamation and memory balancingWhat to do if a container hit a physical memory limit? Free some pages by writing their contents to the abovementioned virtual swap. Well, not quite yet. Remember that there is also a page cache and a dcache, so the kernel can easily discard some of the pages from these caches, which is way cheaper than swapping out.The process of finding some free memory is known as reclamation. Kernel needs to decide very carefully when to start reclamation, how many and what exact pages to reclaim in every particular situation, and when it is the right time to swap out rather than discard some of the cache contents. Remember, we have four types of memory (kernel, user, dcache and page cache) and only one knob which limits the sum of all these. It would be easier for the kernel, but not for the user, to have separate limits for each type of memory. But, for the user convenience and simplicity, the kernel only have one knob for these four parameters, so it needs to balance between those four. One major improvement in 042stab040 kernel is that such balancing is now performed better. Stricter memory limitDuring the lifetime of a container, the kernel might face a situation when it needs more kernel memory, or user memory, or perhaps more dcache entries, and the memory for the container is tight (i.e. close to the limit), so it needs to either reclaim or swap. The problem is there are some situations when neither reclamation nor swapping is possible, so the kernel can either fail miserably (say by killing a process) or go beyond the limit and hope that everything will be fine and mommy won't notice. Another big improvement in 042stab040 kernel is it reduces the number of such situations, in other words, the new kernel obeys memory limit in a more strict way.PolishingFinally, the kernel is now in a pretty good shape, so we can afford some polishing, minor optimizations, and fine tuning. Such polishing was performed in a few subsystems, including checkpointing, user beancounters, netfilter, kernel NFS server and VZ disk quota.Some numbersTotally, there are 53 new patches in 042stab040.1, compared to previous 039 kernels. On top of that, 042stab042.1 adds another 30. We hope that the end result is improved stability and performance. | ||
| |||
| |||
k001 | Instead of having a nice drink in a bar, I spent this Friday night splitting the RHEL6-based OpenVZ kernel branch/repository into two, so now we have 'rhel6' and 'rhel6-testing' branches/repos. Let me explain why. When we made an initial port of OpenVZ to RHEL6 kernel and released the first kernel (in October 2010, named 042test001), I created a repository named openvz-kernel-rhel6 (or just rhel6), and this repository was marked as "development, unstable". When, after almost a year, we announced it as "testing" and then, finally, "stable" (in August 2011, named 042stab035.1). After that, all the kernels in that repository were supposed to be stable, because they are incremental improvements of the kernel we call stable. In theory it is. In practice, of course, there can always be new bugs (both introduced by us and by Red Hat folks releasing their kernel updates which we rebase to). Thus a kernel update from a repo which is supposed to be stable can do bad things. Better late than never, I have fixed the situation tonight by basically renaming "rhel6" repository into "rhel6-testing", and creating a new repository called just "rhel6". For now, I put 042stab037.1 (which is the latest kernel which has passed our internal QA) into rhel6 (aka stable), while all the other kernels, up to and including 042stab039.3, are in rhel6-testing repo. Now, very similar to what we do with RHEL5 kernels, all the new fresh-from-the-build-farm kernels will appear in rhel6-testing repo, at about the same time they go to internal QA. Then, the kernels which will have QA approval will appear in rhel6 (aka -stable) repo. What it means for you as a user is you can now choose whether to stay at the bleeding edge and have the latest stuff, or to take a conservative approach and have less frequent and delayed updates, but be more confident about kernel quality and stability. A few links: * Stable RHEL6-based OpenVZ kernels * Testing RHEL6-based OpenVZ kernels * OpenVZ yum repository setup file * Official announce of rhel6-testing | ||
| |||
| |||
k001 | And we are coming to Prague, too! This time, there will be as many as six people and two talks from us, plus we will held a memory cgroup controller meeting. The following OpenVZ/Parallels people are coming:
Two talks will be presented. Since linuxsymposium.org site is currently down, let me quote talk descriptions here. 1. Container in a file by Maxim Patlasov.
2. OpenVZ and Linux kernel testing by Andrey Vagin.
Finally, there will be a memcg meeting. Since LinuxCon will be right after the Kernel Summit, a number of kernel guys will still be there so anyone interested in cgroups can come. This meeting is a continuation of our recent discussion at Linux Plumbers (see etherpad and presentations). See you all in Prague in less than a month! | ||
| |||
| |||
k001 | Guys, I am very proud to inform you that today we mark RHEL6 kernel branch as stable. Below is a copy-paste from the relevant announce@ post. I personally highly recommend RHEL6-based OpenVZ kernel to everyone -- it is a major step forward compared to RHEL5. In the other news, Parallels has just released Virtuozzo Containers for Linux 4.7, bringing the same cool stuff (VSwap et al) to commercial customers. Despite being only a "dot" (or "minor") release, this product incorporates an impressive amount of man-hours of best Parallels engineers.
| ||
| |||
| |||
k001 | We have finally filed a number of proposals for the up-coming Containers and CGroups miniconf to be held during Linux Plumbers Conference, 7 to 9 September 2011 in Santa Rosa, CA. From those proposals, one can clearly see what are our plans regarding the mainline integration. In a few words: dcache management, memory and CPU cgroup controllers improvements, container enter, improved /proc virtualization, checkpoint/restart [mostly] in userspace (of which I have blogged recently), and making vzctl work with mainline kernel containers. Oh, and the interesting loopback-like block device to hold a container filesystem (a.k.a. ploop). Quite a lot of interesting stuff, what do you think? | ||
| |||
| |||
k001 | There is a good article at lwn.net telling about one of our latest development. We have checkpoint/restart (CPT) and live migration in OpenVZ for ages (well, OK, since 2007 or so), allowing for containers to be freely moved between physical servers without any service interruption. It is a great feature which is valued by our users. The problem is we can't merge it upstream, ie to vanilla kernel. Various people from our team worked on that, and they all gave up. Then, Oren Laadan was trying very hard to merge his CPT implementation -- unfortunately it didn't worked out very well either. The thing is, checkpointing is a complex thing, and the patch implementing it is very intrusive. Recently, our kernel team leader Pavel Emelyanov got a new idea of moving most of the checkpointing complexity out of the kernel and into user space, thus minimizing the amount of the in-kernel changes needed. In about two weeks of time he wrote a working prototype. So far the reaction is mostly positive, and he's going to submit a second RFC version for review to lkml. For more details, read the lwn.net article. After all, while I am sitting next to Pavel, Mr. Corbet ability to explain complex stuff in simple terms is way better than mine. | ||
| |||
| |||
k001 | My colleague ![[info]](http://l-stat.livejournal.com/img/userinfo.gif?v=88.8){kind=small} koct9i, whose daily job is developing and fixing OpenVZ kernel, was feeling bored last weekend, and to entertain himself he wrote a small utility called ioping. The idea is simple and straightforward: to wrote a utility similar to ping, which will show I/O latency in the same way ping shows network latency. The idea is very simple but I haven't see something like this. Actually, the tool was written to help investigating OpenVZ bug #1880).I liked ioping and worked on it a bit, too, just for run. Among some other minor stuff I have added a man page and spec file, so it is now available as an RPM package. Official project site: http://code.google.com/p/ioping/ My RPM packages and stuff: http://kir.sacred.ru/ioping/ | ||
| |||
| |||
k001 | In addition to a bunch of template updates released last week, yesterday night we released Ubuntu 11.04 templates for OpenVZ, for both x86 and x86_64 architectures. They are currently in beta and therefore available from http://wiki.openvz.org/Download/templat Make sure you use latest vzctl (3.0.26.2), otherwise vzctl enter won't work with Ubuntu 11.04 containers. As usual, report all bugs to http://bugzilla.openvz.org/ | ||
| |||
This is a livejournal community for OpenVZ users and developers. It is devoted to virtualization technology as a whole, and OpenVZ in particular. Certainly you can post your thoughts here as well.
Reading
You can read OpenVZ Blog as a web page,
RSS feed,
Atom feed etc.
If you have a livejournal account, you can
add
this journal to your friends list.
Posting
If you do not have a livejournal account yet, you can easily
create one for free.
Then, you should join
the community and post
your stuff.
