Discussion of general kernel technical issues

Brian Buhrow | 9 Aug 2012 00:07

Raidframe and disk strategy

	hello.  I've been looking at some disk performance issues lately and
trying to figure out if there's anything I can do to make it better.  (This
is under NetBSD/I386 5.1_stable with sources from July 18 2012).  During
the course of my investigations, I discovered the raidframe driver does not
implement the DIOCSSTRATEGY or DIOCGSTRATEGY ioctls.  Even more
interestingly, I notice its set to use the fcfs strategy, and has been
doing so since at least NetBSD-2.0.  The ccd(4) driver does the same thing.
Presumably, the underlying disks can use what ever strategy they use for
handling queued data, but I'm wondering if there is a particular reason the
fcfs strategy was chosen for the raidframe driver as opposed to letting the
system administrator pick the strategy?  My particular environment has a
lot of unrelated reads and writes going on simultaneously, and it occurrs
to me that using a different disk strategy than fcfs might mitigate some of
these issues.  Were bench marks done to pick the best strategy for
raidframe and/or ccd or is there some other eason I'm missing that
implementing a buffer queue strategy on top of these devices is a bad idea?
-thanks
-Brian

Greg Oster | 9 Aug 2012 00:38

Picon

Picon

Re: Raidframe and disk strategy

On Wed, 8 Aug 2012 15:07:24 -0700
buhrow <at> lothlorien.nfbcal.org (Brian Buhrow) wrote:

> 	hello.  I've been looking at some disk performance issues
> lately and trying to figure out if there's anything I can do to make
> it better.  (This is under NetBSD/I386 5.1_stable with sources from
> July 18 2012).  During the course of my investigations, I discovered
> the raidframe driver does not implement the DIOCSSTRATEGY or
> DIOCGSTRATEGY ioctls.  Even more interestingly, I notice its set to
> use the fcfs strategy, and has been doing so since at least
> NetBSD-2.0.  The ccd(4) driver does the same thing. Presumably, the
> underlying disks can use what ever strategy they use for handling
> queued data, but I'm wondering if there is a particular reason the
> fcfs strategy was chosen for the raidframe driver as opposed to
> letting the system administrator pick the strategy?  My particular
> environment has a lot of unrelated reads and writes going on
> simultaneously, and it occurrs to me that using a different disk
> strategy than fcfs might mitigate some of these issues.  Were bench
> marks done to pick the best strategy for raidframe and/or ccd or is
> there some other eason I'm missing that implementing a buffer queue
> strategy on top of these devices is a bad idea? -thanks -Brian

The "FIFO" strategy was the one that seemed to be the best tested at
the time, and since I didn't want to introduce any more variables,
that's the one I went with as a default.  Unfortunately, the RAID
labels currently don't specify the queuing strategy, so the
'autoconfig' sets won't do anything other than FIFO at this time.
Non-autoconfig sets certainly support any of the other queuing
methods in the config file, and that could certainly be used for
testing/benchmarking.

(Continue reading)

Brian Buhrow | 9 Aug 2012 01:03

Re: Raidframe and disk strategy

	hello.  Thanks for the quick reply.  My understanding is that the
disk drivers which implement selectable queueing algorithms, do so by
selecting the default strategy at boot time and then letting the user
select the strategy they want to use with the dkctl(8) utility by means of
the DIOCGSTRATEGY and DIOCSSETRATEGY ioctls.  I would propose doing the
same thing for the raidframe driver.  Right now, unless I'm seriously
missing something, it's not possible to change the disk strategy for any
raid device, whether its configured manually or with the autoconfigure
routines.  It's also not clear to me that there needs to be any changes to
the component labels to support this feature.  IN fact, unless the
administrator runs dkctl(8) against the raid device, the patches I'm
thinking of will cause raidframe to behave as it always has,just allowing
the administrator to change the queuing strategy as he can with wd or sd or
ld disks.
	I'll cook up a patch and send it out for people to fly with so that my
ramblings make more sense.
-thanks
-Brian

On Aug 8,  4:38pm, Greg Oster wrote:
} Subject: Re: Raidframe and disk strategy
} On Wed, 8 Aug 2012 15:07:24 -0700
} buhrow <at> lothlorien.nfbcal.org (Brian Buhrow) wrote:
} 
} > 	hello.  I've been looking at some disk performance issues
} > lately and trying to figure out if there's anything I can do to make
} > it better.  (This is under NetBSD/I386 5.1_stable with sources from
} > July 18 2012).  During the course of my investigations, I discovered
} > the raidframe driver does not implement the DIOCSSTRATEGY or
} > DIOCGSTRATEGY ioctls.  Even more interestingly, I notice its set to

(Continue reading)

Brian Buhrow | 9 Aug 2012 07:35

Re: Raidframe and disk strategy

	Hello.  Here is a script showing what I hav in mind.  This is
NetBSD-5.1, but I have patches for both 5.1 and -current, included below,
which do exactly the same thing.  Any reason I shouldn't commit this in the
next week or so, assuming I don't run into any show stoppers in my 5.1
production environment?
I should be able to say in a week or so how much of a performance increase
this makes.

-thanks
-Brian

Script started on Wed Aug  8 21:44:45 2012
[Before the patch]
rbl# dkctl /dev/rraid0d strategy
dkctl: /dev/rraid0d: DIOCGSTRATEGY: Inappropriate ioctl for device

[After the patch...]
rbl# dkctl /dev/rraid0d strategy
/dev/rraid0d: fcfs
rbl#dkctl /dev/rraid0d strategy priocscan
/dev/rraid0d: fcfs -> priocscan
rbl# dkctl /dev/rraid0d strategy
/dev/rraid0d: priocscan
rbl# exit

Script done on Wed Aug  8 21:45:55 2012

[Patch against 5.1 sources from July 18, 2012]

--- rf_netbsdkintf.c.51	2012-07-19 13:10:21.000000000 -0700

(Continue reading)

Thor Lancelot Simon | 9 Aug 2012 14:18

Picon

Re: Raidframe and disk strategy

On Wed, Aug 08, 2012 at 03:07:24PM -0700, Brian Buhrow wrote:
>
> Presumably, the underlying disks can use what ever strategy they use for
> handling queued data, but I'm wondering if there is a particular reason the
> fcfs strategy was chosen for the raidframe driver as opposed to letting the
> system administrator pick the strategy?  My particular environment has a

RAIDframe implements its own disk sorting algorithm (several, actually)
internally.  Why sort twice?

Thor

Greg Troxel | 9 Aug 2012 14:40

Picon

Re: Raidframe and disk strategy


Thor Lancelot Simon <tls <at> panix.com> writes:

> On Wed, Aug 08, 2012 at 03:07:24PM -0700, Brian Buhrow wrote:
>>
>> Presumably, the underlying disks can use what ever strategy they use for
>> handling queued data, but I'm wondering if there is a particular reason the
>> fcfs strategy was chosen for the raidframe driver as opposed to letting the
>> system administrator pick the strategy?  My particular environment has a
>
> RAIDframe implements its own disk sorting algorithm (several, actually)
> internally.  Why sort twice?

I don't follow this.

  - RAIDframe might not be the only user of the physical disk

  - without thinking too much, it seems that the normal strategies
    should be used to sort requests to RAIDframe, and also the normal
    strategies to sort requests made by RAIDframe to the physical
    disks.  Perhaps the latter should do fairshare between raid and
    other, and keep raid in order.

This is all nonobvious to me and I'd venture that how it really ought to
be is even a research topic.

Thor Lancelot Simon | 9 Aug 2012 15:15

Picon

Re: Raidframe and disk strategy

On Thu, Aug 09, 2012 at 08:40:51AM -0400, Greg Troxel wrote:
> 
> Thor Lancelot Simon <tls <at> panix.com> writes:
> 
> > RAIDframe implements its own disk sorting algorithm (several, actually)
> > internally.  Why sort twice?
> 
> I don't follow this.
> 
>   - RAIDframe might not be the only user of the physical disk

It should be -- at least the only user generating a significant
stream of requests -- or RAIDframe itself should use its FCFS
sorting strategy and let the underlying disk device driver sort.
I consider it PEBKAC if it's not.

>   - without thinking too much, it seems that the normal strategies
>     should be used to sort requests to RAIDframe, and also the normal
>     strategies to sort requests made by RAIDframe to the physical
>     disks.  Perhaps the latter should do fairshare between raid and
>     other, and keep raid in order.

You're ignoring the fact that RAIDframe itself sorts internally.  So that
would make a total of three sorts.

Complicating matters slightly is that we know priocscan is better than
any of RAIDframe's sorting algorithms, for a wide variety of workloads.

The intent, one assumes, is to have an ordered stream of requests
at the disk.  It doesn't matter whether they're ordered when they

(Continue reading)

Brian Buhrow | 9 Aug 2012 18:45

Re: Raidframe and disk strategy

	Hello.  I'm not going to claim to be an expert at anything here, but
grepping through the raidframe sources doesn't show me anything that says
it sorts requests from the upper layers, nor that it sorts
requests to the  underlying disks except in the case of when it's doing
reconstruction or paritymap maintenance.  That said, and while I don't have
any hard numbers yet, it looks like the patch I posted last night yields an
instant 16% improvement on throughput on one of the backup servers I run.
Could someone else try the patch and see if they see similar gains?  Below
is a shell snippet I use in /etc/rc.local to set the strategy for all the
attached raid sets on a system, in case that's useful for folks.

-thanks
-Brian

#Set the queuing strategy for raid sets currently configured (BB 8/8/2012)
raidlist="`ls /etc/raid*.conf |sed 's+/etc/++' |sed 's+\.conf++'`"
for curraid in $raidlist; do
	dkctl /dev/r$curraid"d" strategy priocscan
done

Greg Oster | 9 Aug 2012 21:16

Picon

Picon

Re: Raidframe and disk strategy

On Thu, 9 Aug 2012 09:45:18 -0700
buhrow <at> lothlorien.nfbcal.org (Brian Buhrow) wrote:

> 	Hello.  I'm not going to claim to be an expert at anything
> here, but grepping through the raidframe sources doesn't show me
> anything that says it sorts requests from the upper layers, nor that
> it sorts requests to the  underlying disks except in the case of when
> it's doing reconstruction or paritymap maintenance.  That said, and
> while I don't have any hard numbers yet, it looks like the patch I
> posted last night yields an instant 16% improvement on throughput on
> one of the backup servers I run. Could someone else try the patch and
> see if they see similar gains?  Below is a shell snippet I use
> in /etc/rc.local to set the strategy for all the attached raid sets
> on a system, in case that's useful for folks.

So there are a number of places sorting can occur here... 

 1) In the bufq_* strategy bits in RAIDframe.
 2) In the rf_diskqueue.c code, where we have fifo, cvscan, shortest
 seek time first, and two elevator algorithms as possible candidates
 for use in disk queueing.
 3) In the underlying component strategies (which may be another
 RAIDframe device, a CCD, vnd, sd, wd, or whatever).

The sorting stuff I was referring to in my previous email (where I said
it could be handled in the config file) was 2).  I wasn't even thinking
about 1).

Now if you're seeing a 16% performance boost, it's likely worth adding
the code you suggest -- it's not much code for a nice bump for folks

(Continue reading)

Simon Burge | 10 Aug 2012 03:44

Picon

Re: Raidframe and disk strategy

Brian Buhrow wrote:

> #Set the queuing strategy for raid sets currently configured (BB 8/8/2012)
> raidlist="`ls /etc/raid*.conf |sed 's+/etc/++' |sed 's+\.conf++'`"
> for curraid in $raidlist; do
> 	dkctl /dev/r$curraid"d" strategy priocscan
> done

Minor nit-pick - can I suggest the using the list of currently
configured raid sets from the hw.disknames sysctl as done in
/etc/rc.d/raidframeparity ? You can then use the same code for
autoconfigured raid sets where there is no /etc/raid*.conf .

Cheers,
Simon.

Greg Troxel | 9 Aug 2012 20:00

Picon

Re: Raidframe and disk strategy


Thor Lancelot Simon <tls <at> panix.com> writes:

> On Thu, Aug 09, 2012 at 08:40:51AM -0400, Greg Troxel wrote:
>> 
>> Thor Lancelot Simon <tls <at> panix.com> writes:
>> 
>> > RAIDframe implements its own disk sorting algorithm (several, actually)
>> > internally.  Why sort twice?
>> 
>> I don't follow this.
>> 
>>   - RAIDframe might not be the only user of the physical disk
>
> It should be -- at least the only user generating a significant
> stream of requests -- or RAIDframe itself should use its FCFS
> sorting strategy and let the underlying disk device driver sort.
> I consider it PEBKAC if it's not.

I see that as "if you don't only use the disk for rf, it's not going to
work so well" and conclude "diskstrategy isn't good enough" rather than
"don't do that" (although I won't argue it's currently a good idea).

>>   - without thinking too much, it seems that the normal strategies
>>     should be used to sort requests to RAIDframe, and also the normal
>>     strategies to sort requests made by RAIDframe to the physical
>>     disks.  Perhaps the latter should do fairshare between raid and
>>     other, and keep raid in order.
>
> You're ignoring the fact that RAIDframe itself sorts internally.  So that

(Continue reading)

Brian Buhrow | 9 Aug 2012 21:21

Re: Raidframe and disk strategy

	hello.  Having just spent some time digging around in all this stuff,
and running some real work bench marks, here's what I observed and found to
be the case.

	Raidframe does use the built-in queueing infrstructure to receive its
requests from the upper layers.  By default, it sets itself up to use the
fcfs queuing strategy and has done so since as far bak as I checked source,
(I checked NetBSD-2.0, but didn't go any earlier).  Raidframe does not
modify the queuing strategy of the component disks below itself.  For all
the disk types I checked, wd, sd and ld, the drivers use the system default
queuing strategy.  For NetBSD-4 and earlier, the default strategy seems to
be disksort.  For NetBSD-5 and later, it's priocscan.  What I noticed on
one of my systems with a raid1 raid set running NetBSD-5.1, was that  while
the number of disk requests per second was constant, indicating to me that
the subsystems were running flat out, the actual amount of data transfered
in a second was wildly variable.  This made me think the disks were spending
a lot of time seeking back and forth -- not an unreasonable thing to be
doing given that the work load consisted of a lot of processes reading
unrelated files throughout the filesystem.  However, what made me begin
looking into the issue was that if there was a single process running, it
could read pretty fast, but it didn't take many processes to get the
system thrashing all over the place.  That is when I discovered the
discrepancy between the disk strategy raidframe used versus the underlying
strategies for the components of the raid set.  My reasoning was that if
raidframe was taking requests in fcfs order, and the disks were handling
them in  priocscan order, seeking would be maximized in normal operations
because raidframe doesn't send that many requests to the underlying
components before it stops to wait for them to complete.  Raidframe is
taking requests in fcfs order, handing the ones it knows about to the disks
in either the order it got them or a slightly sorted list, and waiting for

(Continue reading)

Edgar Fuß | 16 Oct 2012 17:54

Picon

Re: Raidframe and disk strategy

> More thoughts?
Sorry, no, but two questions:

I may also be suffering from the same sorting problem.
In case I am, which channel would the affected processes be waiting on?

Instead of adapting RAIDframe's sorting to the disc drivers', would it make 
sense to adapt the other way round? There is nothing but RAIDs on my discs.

Brian Buhrow | 16 Oct 2012 19:25

Re: Raidframe and disk strategy

	hello.  What I saw before the patches I submitted for NetBSD-5, 6 and
current was that processes would get stuck in biowait and the disks were
incredibly busy without as much throughput.  By setting the raidframe
strategy to be the same as the underlying disk strategy, I got about a 15%
improvement in throughput.  NetBSD-6 and -current allow you to set the disk
strategies of wd and sd disks, NetBSD-5 only allows changing of the
strategy of wd disks.  The complete patch for -current sets the strategy of
raidframe to the default strategy for the system, rather than fcfs.
That change made it into NetBSD-6 as well, but not into NetBSD-5.
For NetBSD-5, you need to manually set the disk strategy with dkctl(8) in
/etc/rc.local or something like that.  Since dkctl can be run on the fly,
you can easily test various disk sorting strategies, comparing the
throughput of having the raidframe strategy match the disks, versus having
it use fcfs regardless of what the disks are doing, as well as testing
various disk strategies on all levels of your storage subsystem.  I have
found that for my work loads, using the priocscan sorting algorithm on both
the disks and the raid sets yields the best performance.

-Brian

On Oct 16,  5:54pm, Edgar =?iso-8859-1?B?RnXf?= wrote:
} Subject: Re: Raidframe and disk strategy
} > More thoughts?
} Sorry, no, but two questions:
} 
} I may also be suffering from the same sorting problem.
} In case I am, which channel would the affected processes be waiting on?
} 
} Instead of adapting RAIDframe's sorting to the disc drivers', would it make 
} sense to adapt the other way round? There is nothing but RAIDs on my discs.

(Continue reading)

Edgar Fuß | 16 Oct 2012 20:12

Picon

Re: Raidframe and disk strategy

> processes would get stuck in biowait
What I usually see is one of nfsd's four subthreads in biowait and the other 
three in tstile.

> The complete patch for -current sets the strategy of raidframe to the 
> default strategy for the system, rather than fcfs.
What does ``complete patch'' mean, exactly?

> That change made it into NetBSD-6 as well
Ah, great! I'm on 6.
So, as I have rf_netbsdkintf.c 1.295.6.2, I already have that change.

dkctl strategy reports priocscan on both the RAID's components and the RAID 
itself. So my problem must be something else.

David Laight | 16 Oct 2012 20:35

Picon

Re: Raidframe and disk strategy

On Tue, Oct 16, 2012 at 08:12:39PM +0200, Edgar Fu? wrote:
> > processes would get stuck in biowait
> What I usually see is one of nfsd's four subthreads in biowait and the other 
> three in tstile.

FWIW I've NFI why nfsd has this default of 4 threads.
I've seen a lot of systems where 1 works much better
(for all sorts of reasons).

	David

--

-- 
David Laight: david <at> l8s.co.uk

Edgar Fuß | 17 Oct 2012 12:03

Picon

Re: Raidframe and disk strategy

Two more questions on the subject:

> sets the strategy of raidframe to the default strategy for the system,
> rather than fcfs.
How does that play with the usual ``fifo 100'' in the ``START queue'' section?

> you can easily test various disk sorting strategies
Where can I find a description/discussion of the different strategies?

Probably your patch is the cause for our occasional NFS hangs having decreased 
from thirty seconds to a few seconds.

Brian Buhrow | 17 Oct 2012 20:26

Re: Raidframe and disk strategy

On Oct 17, 12:03pm, Edgar =?iso-8859-1?B?RnXf?= wrote:
} Subject: Re: Raidframe and disk strategy
} Two more questions on the subject:
} 
} > sets the strategy of raidframe to the default strategy for the system,
} > rather than fcfs.
} How does that play with the usual ``fifo 100'' in the ``START queue'' section?
} 
} > you can easily test various disk sorting strategies
} Where can I find a description/discussion of the different strategies?
} 
} Probably your patch is the cause for our occasional NFS hangs having decreased 
} from thirty seconds to a few seconds.
>-- End of excerpt from Edgar =?iso-8859-1?B?RnXf?=

	In answering your question, I find I have a couple for Greg.
Depending on his comments to my notes below, you may have a couple of knobs
you can tweak for performance to the raidframe system to get even more
efficiency out of the system.
	It looks like you can select a number of disk queuing strategies
within raidframe itself, something I didn't realize.  There seem to be  5
choices: fifo, cvscan, sstf, scan and cscan.
Fifo is the one we've been using for years, but the others appear to be
compiled into the system.  Unless my understanding is completely wrong,
cscan is the algorithm which most closely aligns with the priocscan buffer
queue strategy and scan  matches the traditional BSD disksort buffer queue
strategy.  To set a new disk queueing strategy for a given raid set, try
the following:

1.  If the raid set is configured by  startup scripts at boot time, then

(Continue reading)

Greg Oster | 17 Oct 2012 21:29

Picon

Picon

Re: Raidframe and disk strategy

On Wed, 17 Oct 2012 11:26:12 -0700
buhrow <at> lothlorien.nfbcal.org (Brian Buhrow) wrote:

> On Oct 17, 12:03pm, Edgar =?iso-8859-1?B?RnXf?= wrote:
> } Subject: Re: Raidframe and disk strategy
> } Two more questions on the subject:
> } 
> } > sets the strategy of raidframe to the default strategy for the
> system, } > rather than fcfs.
> } How does that play with the usual ``fifo 100'' in the ``START
> queue'' section? } 
> } > you can easily test various disk sorting strategies
> } Where can I find a description/discussion of the different
> strategies? } 
> } Probably your patch is the cause for our occasional NFS hangs
> having decreased } from thirty seconds to a few seconds.
> >-- End of excerpt from Edgar =?iso-8859-1?B?RnXf?=
> 
> 
> 	In answering your question, I find I have a couple for Greg.
> Depending on his comments to my notes below, you may have a couple of
> knobs you can tweak for performance to the raidframe system to get
> even more efficiency out of the system.
> 	It looks like you can select a number of disk queuing
> strategies within raidframe itself, something I didn't realize.
> There seem to be  5 choices: fifo, cvscan, sstf, scan and cscan.
> Fifo is the one we've been using for years, but the others appear to
> be compiled into the system.  Unless my understanding is completely
> wrong, cscan is the algorithm which most closely aligns with the
> priocscan buffer queue strategy and scan  matches the traditional BSD

(Continue reading)

Brian Buhrow | 17 Oct 2012 21:50

Re: Raidframe and disk strategy

	hello.  Thanks for the reply.  I was just loking at the raidframe code
to determine if the queueing strategy was permanent across reboots.  Also,
I apologize Edgar, cscan is not a valid queueing strategy name. The  four
choices are: fifo, cvscan, sstf and scan.

	As to the possibility of implementing persistent disk queuing
strategies, I was thinking that the easiest way might be just to add an
option to store the letter of the queuing strategy in the component label,
and then having the autoconfigure code set the queueing strategy when it
runs during the next reboot.  You can already achieve a change to the
queuing strategy on a raid set by unconfiguring it and reconfiguring it.
(Yes, that means you have to reboot for root raid sets or raid sets that
can't be unmounted at run time, but that seems safer than trying to do this
change on the fly, especially since it isn't something one would do often
under most circumstances.)

	I've got a raid set that I use for backups that I've altered to use
the scan strategy.  

Thoughts?
-Brian

Greg Oster | 17 Oct 2012 22:04

Picon

Picon

Re: Raidframe and disk strategy

On Wed, 17 Oct 2012 12:50:44 -0700
buhrow <at> lothlorien.nfbcal.org (Brian Buhrow) wrote:

> 	hello.  Thanks for the reply.  I was just loking at the
> raidframe code to determine if the queueing strategy was permanent
> across reboots.  Also, I apologize Edgar, cscan is not a valid
> queueing strategy name. The  four choices are: fifo, cvscan, sstf and
> scan.
> 
> 	As to the possibility of implementing persistent disk queuing
> strategies, I was thinking that the easiest way might be just to add
> an option to store the letter of the queuing strategy in the
> component label, and then having the autoconfigure code set the
> queueing strategy when it runs during the next reboot.  You can
> already achieve a change to the queuing strategy on a raid set by
> unconfiguring it and reconfiguring it. (Yes, that means you have to
> reboot for root raid sets or raid sets that can't be unmounted at run
> time, but that seems safer than trying to do this change on the fly,
> especially since it isn't something one would do often under most
> circumstances.)

Unconfiguring/reconfiguring is a pain...  I think what we really need
is an ioctl or similar to set the strategy (it can just pass in a
config struct with only the strategy field set -- that mechanism is
already there).  The ioctl code can then do the quiesce/change/go
song'n'dance and update component labels accordingly.  New code in the
component label code would read in the strategy (from a newly carved
out field from the component labels) and away we go.  There's lots of
free space remaning in the component labels, and this change would be
backwards compatible with older kernels.  New kernels seeing a '0' in

(Continue reading)

Edgar Fuß | 17 Oct 2012 22:55

Picon

Re: Raidframe and disk strategy

With me, the relevant RAID is not auto-configured, so I can just edit 
raid1.conf.

But my original question is still open (or I didn't understand that it was 
answered): I can set the component's sorting strategy with dkcntl sdn strategy;
I can set some RAIDframe sorting strategy with dkcntl raidn strategy. But there 
seems to be a third sorting strategy: the one I set in the START queue section.
Is this really so? If yes, at which point (relative to the two other sorting 
operations) does this sorting take place?

Brian Buhrow | 17 Oct 2012 23:18

Re: Raidframe and disk strategy

On Oct 17, 10:55pm, Edgar =?iso-8859-1?B?RnXf?= wrote:
} Subject: Re: Raidframe and disk strategy
} With me, the relevant RAID is not auto-configured, so I can just edit 
} raid1.conf.
} 
} But my original question is still open (or I didn't understand that it was 
} answered): I can set the component's sorting strategy with dkcntl sdn strategy;
} I can set some RAIDframe sorting strategy with dkcntl raidn strategy. But there 
} seems to be a third sorting strategy: the one I set in the START queue section.
} Is this really so? If yes, at which point (relative to the two other sorting 
} operations) does this sorting take place?
>-- End of excerpt from Edgar =?iso-8859-1?B?RnXf?=

	Hello. You're right.  I think the pieces ar like this:

1.  the dkctl sdn strategy sets the policy by which the system reads
and writes to and from the disk.

2.  The dkctl raidn strategy sets the policy by which the system reads
and writes to and from the logical disk, i.e. the raid volume

3.  The algorithm indicated in the /etc/raidx.conf queue strategy section
sets the policy by which the  raid driver queues requests for the
underlying components of the raid set.
I think, but don't have any hard numbers to back it up yet, that the best
performance is achieved when all the layers are using the same or similar
buffer queueing strategies.  I've already shown that I get a 15%
improvement by seting the strategies in 1 and 2 above to be the same.

	Just to be redundant, I think the data flow looks something like:

(Continue reading)

matthew green | 18 Oct 2012 08:22

Picon

re: Raidframe and disk strategy


> backwards compatible with older kernels.  New kernels seeing a '0' in
> that spot would simply pick 'fifo' as before.  One question is whether
> to use 0,1,2,3 to encode the strategy, or to use letters, or whatever.
> But that's just a minor implementation detail.

considering the random garbage we saw in component labels when
adding > 2TB support, i don't trust that we won't see anything
with some garbage in this new place.

i wonder if we need to really version this properly 8-/

.mrg.

Edgar Fuß | 19 Oct 2012 12:59

Picon

information on disc queue strategies (was: Raidframe and disk strategy)

> Where can I find a description/discussion of the different strategies?
I think this question of mine is still open.
There must be some sort of information somewhere what the rationale behind 
the different strategies is and what workload they are targeted at.

Return

Return to gmane.os.netbsd.devel.kernel.

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane