headers
David Howland | 13 Oct 2011 00:49

amd64 MP (was Re: pae MP on cherry-xenmp)

On 9/9/11 4:57 AM, Cherry G. Mathew wrote:
 > Just to say I've checked in amd64 support. Do give it a spin, and let
 > us know (on list) how it goes.
 >
 > I haven't tested this on dom0.

You, Sir, are my hero.

I'm checking in with my results so far...

I am running on an Intel Xeon platform, Debian squeeze dom0 (Xen 4.0) 
with 2 vcpu's and 2GB allocated to NetBSD domU.  My domU is running 5.1 
userland with cherry-xenmp kernel, compiled like this (code was checked 
out October 12th):

cvs checkout -r cherry-xenmp src
./build.sh -m amd64 tools
./build.sh -m amd64 kernel=XEN3_DOMU

Status: working fine, but I haven't loaded it much yet.  I am running 
GNOME in Xvnc and I'm able to poke around and browse the web just fine. 
  I'll try to build /src with -j4.

-d
Permalink | Reply |
headers
David Howland | 13 Oct 2011 01:46

Re: amd64 MP (was Re: pae MP on cherry-xenmp)

On 10/12/2011 6:49 PM, David Howland wrote:
...
> I am running on an Intel Xeon platform, Debian squeeze dom0 (Xen 4.0)
> with 2 vcpu's and 2GB allocated to NetBSD domU. My domU is running 5.1
> userland with cherry-xenmp kernel, compiled like this (code was checked
> out October 12th):

Okay, one thing I've noticed, my NetBSD domU never seems to 'idle' from 
the hypervisor's perspective.  Check out the domain list:

root <at> rackable:/etc/xen# xm vcpu-list
Name              ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0           0     0     0   -b-     814.0 0
Domain-0           0     1     1   -b-     632.4 1
Domain-0           0     2     2   r--     582.5 2
Domain-0           0     3     3   -b-     609.1 3
centos-57          5     0     7   -b-      10.2 1-3,5-7
centos-57          5     1     6   -b-       6.7 1-3,5-7
netbsd-51          3     0     6   r--   19635.9 1-3,5-7
netbsd-51          3     1     5   r--   19634.3 1-3,5-7
solaris-11         4     0     7   -b-      40.0 1-3,5-7
solaris-11         4     1     2   -b-      36.8 1-3,5-7
windows-2008r2     1     0     4   -b-     321.9 4
windows-2008r2     1     1     5   -b-     105.4 5
windows-2008r2     1     2     6   -b-     107.7 6
windows-2008r2     1     3     7   -b-     135.5 7

The 'time' column for the NetBSD domain counts up every second even 
though 'top' in netbsd-51 shows both CPU's as 100% idle.  Basically, 
what you are seeing is a timer of how long the domain has been running
(Continue reading)

Permalink | Reply |
headers
Jean-Yves Migeon | 13 Oct 2011 02:13
Picon
Favicon

Re: amd64 MP (was Re: pae MP on cherry-xenmp)

On 13.10.2011 01:46, David Howland wrote:
> The 'time' column for the NetBSD domain counts up every second even
> though 'top' in netbsd-51 shows both CPU's as 100% idle. Basically, what
> you are seeing is a timer of how long the domain has been running since
> I created it.
>
> I don't know what this means, but I report it because it seems to be
> different from all the other OS in the list.

Without looking at the code, I would say that the idle() loop for a CPU 
does not yield to hypervisor correctly. Or that something keeps kicking 
vCPUs out of it.

--

-- 
Jean-Yves Migeon
jeanyves.migeon <at> free.fr
Permalink | Reply |
headers
David Howland | 19 Oct 2011 02:47

Re: amd64 MP (was Re: pae MP on cherry-xenmp)

On 10/12/2011 8:13 PM, Jean-Yves Migeon wrote:
> On 13.10.2011 01:46, David Howland wrote:
>> The 'time' column for the NetBSD domain counts up every second even
>> though 'top' in netbsd-51 shows both CPU's as 100% idle. Basically, what
>> you are seeing is a timer of how long the domain has been running since
>> I created it.
>>
>> I don't know what this means, but I report it because it seems to be
>> different from all the other OS in the list.
>
> Without looking at the code, I would say that the idle() loop for a CPU
> does not yield to hypervisor correctly. Or that something keeps kicking
> vCPUs out of it.
>

I replaced Cherry's kernel with a standard -current kernel (non-MP) and 
the problem goes away...

root <at> rackable:/etc/xen# xm list
Name               ID   Mem VCPUs      State   Time(s)
Domain-0            0  7178     4     r----- 476622.8
centos-57           5  2048     2     -b----   1411.3
netbsd-51           7  2048     1     -b----    108.4
solaris-11          4  2048     2     -b----   4589.1
windows-2008r2      8  4096     4     -b----     35.5

NetBSD seems to be able to idle just fine.

-d
(Continue reading)

Permalink | Reply |
headers
Hugo Silva | 13 Oct 2011 12:39
Favicon

Re: amd64 MP (was Re: pae MP on cherry-xenmp)

On 10/12/11 23:49, David Howland wrote:
> On 9/9/11 4:57 AM, Cherry G. Mathew wrote:
>> Just to say I've checked in amd64 support. Do give it a spin, and let
>> us know (on list) how it goes.
>>
>> I haven't tested this on dom0.
> 
> You, Sir, are my hero.
> 
> I'm checking in with my results so far...
> 
> I am running on an Intel Xeon platform, Debian squeeze dom0 (Xen 4.0)
> with 2 vcpu's and 2GB allocated to NetBSD domU.  My domU is running 5.1
> userland with cherry-xenmp kernel, compiled like this (code was checked
> out October 12th):
> 
> cvs checkout -r cherry-xenmp src
> ./build.sh -m amd64 tools
> ./build.sh -m amd64 kernel=XEN3_DOMU
> 
> Status: working fine, but I haven't loaded it much yet.  I am running
> GNOME in Xvnc and I'm able to poke around and browse the web just fine.
>  I'll try to build /src with -j4.
> 
> -d

Is this for an amd64 domU ?

I lost the thread on this subject some weeks ago. Been setting up a pet
dom0 to test this out, and was going with i386+PAE.
(Continue reading)

Permalink | Reply |
headers
Jean-Yves Migeon | 13 Oct 2011 13:59
Picon
Favicon

Re: amd64 MP (was Re: pae MP on cherry-xenmp)

 On Thu, 13 Oct 2011 11:39:15 +0100, Hugo Silva wrote:
> Generally speaking, is the difference between the MP code for dom0 
> and
> domU big enough that the dom0 typical workload would trigger bugs 
> that a
> domU likely wouldn't?
>
> I ask to determine the best path of action: if MP-domU is likely to
> trigger the same class of bugs as MP-dom0, then it might make more 
> sense
> to run this code on a test domU (exclusively) first (not having the 
> dom0
> crash all the time helps with testing)

 Best path is to try out SMP domU first, as this will shake out all 
 frontend drivers thread-safety and virtual memory (this part is shared 
 with dom0) parts.

 Once that is stable, you can start hard-hitting the dom0. This will add 
 real device drivers to the loop, then eventually virtual backend drivers 
 when you start running multiple domUs.

--

-- 
 Jean-Yves Migeon
 jeanyves.migeon <at> free.fr
Permalink | Reply |

Gmane