Top.Mail.Ru
? ?

Previous Entry | Next Entry

Yay to I/O limits!

Today we are releasing a somewhat small but very important OpenVZ feature: per-container disk I/O bandwidth and IOPS limiting.

OpenVZ have I/O priority feature for a while, which lets one set a per-container I/O priority -- a number from 0 to 7. This is working in a way that if two similar containers with similar I/O patterns, but different I/O priorities are run on the same system, a container with a prio of 0 (lowest) will have I/O speed of about 2-3 times less than that of a container with a prio of 7 (highest). This works for some scenarios, but not all.

So, I/O bandwidth limiting was introduced in Parallels Cloud Server, and as of today is available in OpenVZ as well. Using the feature is very easy: you set a limit for a container (in megabytes per second), and watch it obeying the limit. For example, here I try doing I/O without any limit set first:

root@host# vzctl enter 777
root@CT:/# cat /dev/urandom | pv -c - >/bigfile
 88MB 0:00:10 [8.26MB/s] [         <=>      ]
^C

Now let's set the I/O limit to 3 MB/s:

root@host# vzctl set 777 --iolimit 3M --save
UB limits were set successfully
Setting iolimit: 3145728 bytes/sec
CT configuration saved to /etc/vz/conf/777.conf
root@host# vzctl enter 777
root@CT:/# cat /dev/urandom | pv -c - >/bigfile3
39.1MB 0:00:10 [   3MB/s] [         <=>     ]
^C

If you run it yourself, you'll notice a spike of speed at the beginning, and then it goes down to the limit. This is so-called burstable limit working, it allows a container to over-use its limit (up to 3x) for a short time.

In the above example we tested writes. Reads work the same way, except when read data are in fact coming from the page cache (such as when you are reading the file which you just wrote). In this case, no actual I/O is performed, therefore no limiting.

Second feature is I/O operations per second, or just IOPS limit. For more info on what is IOPS, go read the linked Wikipedia article -- all I can say here is for traditional rotating disks the hardware capabilities are pretty limited (75 to 150 IOPS is a good guess, or 200 if you have high-end server class HDDs), while for SSDs this is much less of a problem. IOPS limit is set in the same way as iolimit (vzctl set $CTID --iopslimit NN --save), although measuring its impact is more tricky.</o>

Finally, to play with this stuff, you need:

  • vzctl 4.6 (or higher)
  • Kernel 042stab084.3 (or higher)
Note that the kernel with this feature is currently still in testing -- so if you haven't done so, it's time to read about testing kernels.

Comments

k001
Oct. 30th, 2013 03:16 pm (UTC)
Our sysadmins say it works for ZFS, too.
(Anonymous)
May. 29th, 2014 01:23 pm (UTC)
It doesn't seem to be working on zfsonlinux.

Latest Month

July 2016
S M T W T F S
     12
3456789
10111213141516
17181920212223
24252627282930
31      

Page Summary

Comments

Powered by LiveJournal.com
Designed by Tiffany Chow