Mon, 19 Feb 2007
Virtualization Overhead
In a followup to my previous article
I want to sum up the speed penalty introduced by the two virtualization
systems, KVM and lguest. The measurements are by no means
meaningful for everyone, just exact data I have measured. And yes,
it sometimes compares apples-to-oranges - read on at your own risk :-)
The test systems were the following:
- Pyrrha
- Dual AMD Opteron 2220 (4 cores total, 2.8GHz), 6GB RAM, 4-way RAID-10 disk, gigabit LAN, Gentoo Linux/AMD64.
- Terminus
- KVM-based virtual AMD64 machine running at Pyrrha, 10 GB disk image, 512 MB RAM, bridged LAN to the main Pyrrha's LAN, Fedora 6/AMD64.
- Scylla
- Pentium 4 3.0GHz HT, 1 GB RAM, single SATA drive, 100baseTx LAN, Fedora Rawhide/IA32.
- Glauke
lguest-based virtual IA32 machine running at Scylla, 10 GB disk image, 512 MB RAM, routed/masqueraded LAN to the main Scylla's LAN, Fedora 6/IA32.
| Test | Pyrrha | Terminus | KVM overhead | Scylla | Glauke | lguest overhead |
|---|---|---|---|---|---|---|
| bc | 6.286 | 5.876 | -6.52 % | 8.130 | 8.240 | 1.35 % |
| wget | 0.441 | 10.885 | 2368.25 % | 3.732 | 3.770 | 1.02 % |
| tar unpack | 15.118 | 20.322 | 34.42 % | 27.566 | 40.701 | 47.65 % |
| rm -rf | 0.538 | 0.634 | 17.84 % | 0.477 | 0.640 | 34.17 % |
| compile | 6.410 | 21.929 | 242.11 % | 126.005 | 184.833 | 46.69 % |
The numbers in the above table are in seconds (so lower is better). I ran each test five times and used the lowest time from these five runs. I did not bother to reboot between the tests or switch the system daemons off.
Description of the tests
- bc
time sh -c 'echo 2^1000000|bc >/dev/null'
A simple test of a CPU-intensive job. Why is Terminus faster than Pyrrha? Maybe the clock skew inside the guest? Or Gentoo-compiledbcbeing slower than Fedora-prebuilt one?- wget
time wget -q -O /dev/null ftp://ftp.linux.cz/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2
Network performance. KVM (having to emulate the real-world NIC) is waaay slower. However, Pyrrha has a gigabit NIC, so the baseline is 10 times off. But still, raw bandwidth used for KVM was ~22 Mbit/s, whilelguesthas filled the Scylla's 100 Mbit pipe without trouble.lguestcould be even faster in the future, if they use a bounce buffer bigger than a single page (which is what they use now).- tar unpack and rm
time tar xjf /usr/src/linux-2.6.20.tar.bz2 ; time rm -rf linux-2.6.20.tar.bz2
A simple filesystem-related test. Nothing to see here (KVM is a bit faster).- compile
make clean; time make modules > /dev/null
A simple part of kernel compile. Both the architecture and kernel config was different between Pyrrha+Terminus and Scylla+Glauke, so do not try to compare the absolute times between those two groups. Interestingly enough, KVM was much slower thanlguest.
From a subjective point of view, lguest feels faster, which was
totally unexpected. I am looking forward to the further development
(especially lguest with AMD64 support). Anyway, James Morris did
a more in-depth
measurements of the lguest network performance.