headers
George Washington Dunlap III | 2 Mar 2004 00:48
Picon

Overhead of UML-skas

Hey Jeff (and any other interested parties) -- this is George Dunlap, from
the CoVirt group at U of M.  I've been testing the overhead of running
workloads under UML compared to on the bare host, and we're getting some
unexpectedly slow numbers -- for the particular benchmark we're using,
about 105% overhead;  I was wondering if you could shed any light on the
subject: has anyone had similar experiences, and are there any obvious 
optimizations we're missing.

We're using 2.4.20-7um, on a 2.4.18 host with the skas patch.  UML has 
raw access to its own disk (via 'raw /dev/raw/raw1 /dev/hdc'), and is 
given 256M (out of 1Gig host memory).  Asyncronous I/O is enabled via the 
helper process.  COW disk mode is off.

The workload we're running is a "kernel build" benchmark, which consists
of the following commands:
  make clean ; make dep ; make bzImage.
This benchmark is run four times: once to warm up the cache, and thrice to 
get "warm cache" numbers.  All runs are taken immediately after booting 
the host kernel and then (if appropriate) the guest kernel.

We're comparing this against a 2.4.20 vanilla kernel using a copy of the 
same disk image, on the same disk.

Host 2.4.20 average:	 93 seconds
UML 2.4.20 average:	192 seconds (105% overhead)

All numbers were run on a P4 3GHz processor.

Two years ago for our ReVirt paper, we did the same test with UMLinux (now
FAUMAchine) (with some optimizations to make it not-dog-slow) on a 1.2GHz
(Continue reading)

Permalink | Reply |
headers
William Stearns | 2 Mar 2004 01:13
Picon
Favicon

Re: Overhead of UML-skas

Good evening, George,

On Mon, 1 Mar 2004, George Washington Dunlap III wrote:

> Hey Jeff (and any other interested parties) -- this is George Dunlap, from
> the CoVirt group at U of M.  I've been testing the overhead of running
> workloads under UML compared to on the bare host, and we're getting some
> unexpectedly slow numbers -- for the particular benchmark we're using,
> about 105% overhead;  I was wondering if you could shed any light on the
> subject: has anyone had similar experiences, and are there any obvious 
> optimizations we're missing.

	Do the boot time messages say it's actually going into skas mode?
	Do you have tmpfs mounted on /tmp, with enough potential space for 
all virtual machines?  
http://www.stearns.org/slartibartfast/uml-coop.current.html#host

> 
> We're using 2.4.20-7um, on a 2.4.18 host with the skas patch.  UML has 
> raw access to its own disk (via 'raw /dev/raw/raw1 /dev/hdc'), and is 
> given 256M (out of 1Gig host memory).  Asyncronous I/O is enabled via the 
> helper process.  COW disk mode is off.

	Helper process?
	BTW, any chance you have Synchronous mode turned on for ubd?  
Please make sure that's off in your kernel compile.
	Cheers,
	- Bill

---------------------------------------------------------------------------
(Continue reading)

Permalink | Reply |
headers
George Washington Dunlap III | 2 Mar 2004 01:55
Picon

Re: Overhead of UML-skas

On Mon, 1 Mar 2004, William Stearns wrote:
> 	Do the boot time messages say it's actually going into skas mode?
> 	Do you have tmpfs mounted on /tmp, with enough potential space for 
> all virtual machines?  
> http://www.stearns.org/slartibartfast/uml-coop.current.html#host
[snip]
> 	Helper process?
> 	BTW, any chance you have Synchronous mode turned on for ubd?  
> Please make sure that's off in your kernel compile.

Thanks for the quick response.

* SKAS mode: I didn't look specifically at the post messages in those
runs, but I'm using the same scripts and HD image (and getting similar
numbers for UML) for the last 3-4 months.  Plus, TT mode is disabled in
the config.  The most recent run (on a different benchmark) did say:
 Checking for the skas3 patch in the host...found
 Checking for /proc/mm...found

I've been getting those kinds of numbers for awhile; I was only shocked 
when I finally ran numbers on the host and found out how much faster it 
was. =)

* /tmp: I wasn't using tmpfs, but I had ramfs mounted on /tmp/uml, and the
TMP environment variable set to it.  Again, I didn't verify it for this
particular run, but I had verified it some time ago when I set my
environment up.  I may try tmpfs at some point and see if there's any
performance difference.

Hm, it appears that one of my co-workers changed the scripts, and it
(Continue reading)

Permalink | Reply |
headers
Jeff Dike | 2 Mar 2004 03:51

Re: Overhead of UML-skas

dunlapg <at> umich.edu said:
> Our standard mode of operation is to have ayncronous I/O turned off by
> default, because of the extra process that's created.  

Have the build on the host do it on a O_SYNC filesystem and see how fast it
goes.

> (It has to do
> with the research we're doing right now.)  I explicitly enabled it for
> this build, and verified that the helper process was being created. It
> resulted in a marked speed increase (about 6% for this benchmark, if
> you're interested). 

FWIW, I'd expect a 20-30% slowdown from the host on a kernel build.  I see
25-30% on my laptop, Matt Ayes in #uml said he saw 20% on one of his servers.

				Jeff

-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
Permalink | Reply |

Gmane