headers
Artur Grabowski | 1 Feb 2010 10:08

Re: UBC?

Ariane van der Steldt <ariane <at> stack.nl> writes:

> Why are the pventries allocated from the kmem_map anyway? I think they
> should be allocated using the uvm_km_getpage instead. Or even better,
> from a pvpool like amd64.

Recursion.

caller holds lock on kernel_map. getpage pool is empty, caller wakes
up the getpage thread, goes to sleep (still holding the kernel_map
lock), getpage thread wakes up, deadlocks on the kernel_map lock. It's
not an easily detectable recursion either, so we don't panic when it
happens, we just hang.

amd64 can avoid it thanks to the direct map (no kernel_map involed when
calling getpage).

We could try some magic with allocating from a pool with NOWAIT and
then fall back to kmem_map when that fails, but the logic would become
hairy. Maybe a pool allocator with those semantics?

//art
Permalink | Reply |
headers
Owain Ainsworth | 1 Feb 2010 15:07

Re: UBC?

On Mon, Feb 01, 2010 at 10:08:08AM +0100, Artur Grabowski wrote:
> Ariane van der Steldt <ariane <at> stack.nl> writes:
> 
> > Why are the pventries allocated from the kmem_map anyway? I think they
> > should be allocated using the uvm_km_getpage instead. Or even better,
> > from a pvpool like amd64.
> 
> Recursion.
> 
> caller holds lock on kernel_map. getpage pool is empty, caller wakes
> up the getpage thread, goes to sleep (still holding the kernel_map
> lock), getpage thread wakes up, deadlocks on the kernel_map lock. It's
> not an easily detectable recursion either, so we don't panic when it
> happens, we just hang.
> 
> amd64 can avoid it thanks to the direct map (no kernel_map involed when
> calling getpage).
> 
> We could try some magic with allocating from a pool with NOWAIT and
> then fall back to kmem_map when that fails, but the logic would become
> hairy. Maybe a pool allocator with those semantics?

I think the pool allocator is doable. Will look at it when I get a spare
hour or two (may be a while ;)

As a note, why are we even using pmap_enter for dmamem_map anyway? it's
a kernel only mapping that doesn't fault or anything, why not
pmap_kenter_pa? Or is the policy that if it's a managed page, it gets
normaly entered? Admittedly I should already know this...
(Continue reading)

Permalink | Reply |
headers
Ted Unangst | 1 Feb 2010 18:41
Picon

Re: UBC?

On Mon, Feb 1, 2010 at 9:07 AM, Owain Ainsworth <zerooa <at> googlemail.com> wrote:
> On Mon, Feb 01, 2010 at 10:08:08AM +0100, Artur Grabowski wrote:
>> We could try some magic with allocating from a pool with NOWAIT and
>> then fall back to kmem_map when that fails, but the logic would become
>> hairy. Maybe a pool allocator with those semantics?
>
>
> I think the pool allocator is doable. Will look at it when I get a spare
> hour or two (may be a while ;)

Noooooooooooooooooooooooooooooooooooooooooooooooooooooo!!!

You are begging for pain.  Multi backend allocators have not fared well.
Permalink | Reply |
headers
Bob Beck | 1 Feb 2010 23:59
Picon
Picon
Favicon

Re: UBC?

On 1 February 2010 10:41, Ted Unangst <ted.unangst <at> gmail.com> wrote:

>> I think the pool allocator is doable. Will look at it when I get a spare
>> hour or two (may be a while ;)
>
> Noooooooooooooooooooooooooooooooooooooooooooooooooooooo!!!
>
> You are begging for pain.  Multi backend allocators have not fared well.
>

He hacks *accelerated X*  and gets into pmap for fun and amusement and
occasionally joins me looking a the buffer cache and NFS.. I've not
yet decided if that's because he's simply a glutton for punshiment,
feels sorry for me being in pain, or simply finds NFS and the buffer
cache less pain that what he's usually looking at...

Telling oga there will be pain involved is not a demotivator... He
probably goes to the dentist like
http://www.youtube.com/watch?v=iAKYQjpDtpA
Permalink | Reply |
headers
Owain Ainsworth | 3 Feb 2010 15:36

Re: UBC?

On Mon, Feb 01, 2010 at 03:59:24PM -0700, Bob Beck wrote:
> On 1 February 2010 10:41, Ted Unangst <ted.unangst <at> gmail.com> wrote:
> 
> >> I think the pool allocator is doable. Will look at it when I get a spare
> >> hour or two (may be a while ;)
> >
> > Noooooooooooooooooooooooooooooooooooooooooooooooooooooo!!!
> >
> > You are begging for pain. ?Multi backend allocators have not fared well.

I should mention the arm pmap bootstrap allocator (which seems to work
fine).

> >
> 
> He hacks *accelerated X*  and gets into pmap for fun and amusement and
> occasionally joins me looking a the buffer cache and NFS.. I've not
> yet decided if that's because he's simply a glutton for punshiment,
> feels sorry for me being in pain, or simply finds NFS and the buffer
> cache less pain that what he's usually looking at...

Little from column A, little from column B. Column C, the buffer cache
is about the same pain, NFS is just worse, no probably not ;).

> 
> Telling oga there will be pain involved is not a demotivator... He
> probably goes to the dentist like
> http://www.youtube.com/watch?v=iAKYQjpDtpA

I'm not sure what to say to that...
(Continue reading)

Permalink | Reply |
headers
Ted Unangst | 1 Feb 2010 18:42
Picon

Re: UBC?

On Mon, Feb 1, 2010 at 4:08 AM, Artur Grabowski <art <at> blahonga.org> wrote:
> caller holds lock on kernel_map. getpage pool is empty, caller wakes
> up the getpage thread, goes to sleep (still holding the kernel_map
> lock), getpage thread wakes up, deadlocks on the kernel_map lock. It's
> not an easily detectable recursion either, so we don't panic when it
> happens, we just hang.
>
> amd64 can avoid it thanks to the direct map (no kernel_map involed when
> calling getpage).
>
> We could try some magic with allocating from a pool with NOWAIT and
> then fall back to kmem_map when that fails, but the logic would become
> hairy. Maybe a pool allocator with those semantics?

We could keep a couple entries always available.  Then the logic goes like this:

if (alloc(NOWAIT))
    use freelist;
else
    while (freelist < minsize)
         alloc(NOWAIT).
Permalink | Reply |

Gmane