gmane.linux.kernel

Gregory Haskins | 22 May 16:02

[ANNOUNCE] sched: schedtop utility

Hi all scheduler developers,
  I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, and it appears that the perl scripts
referenced in Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I whipped up a
little utility I call "schedtop".

This utility will process statistics from /proc/schedstat such that the busiest stats will bubble up to
the top.  It can alternately be sorted by the largest stat, or by name.  Stats can be included or excluded
based on reg-ex pattern matching.

You can download the tarball here:

ftp://ftp.novell.com/dev/ghaskins/schedtop.tar.gz

I have also posted it to the opensuse build service for generating RPMS for a handful of 32/64-bit x86
distros for your convenience:

http://download.opensuse.org/repositories/home:/ghaskins/

(Note that the build is still in progress for some of the favors, so if you do not see the flavor you are looking
for, check back in a little while)

Comments/feedback/bug-fixes welcome!

Regards
-Greg

Steven Rostedt | 22 May 16:33

Re: [ANNOUNCE] sched: schedtop utility

On Thu, 22 May 2008, Gregory Haskins wrote:

> Hi all scheduler developers,
>   I had an itch to scratch w.r.t. watching the stats in
> /proc/schedstats, and it appears that the perl scripts referenced in
> Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so
> I whipped up a little utility I call "schedtop".

Greg,

Cool, I'll have to take a look when things get a little slower for me. But
when I get a chance, I'll make my comments then.

-- Steve

Ankita Garg | 2 Jun 14:48

Re: [ANNOUNCE] sched: schedtop utility

Hi Gregory,

On Thu, May 22, 2008 at 08:06:44AM -0600, Gregory Haskins wrote:
> Hi all scheduler developers,
>   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, and it appears that the perl
scripts referenced in Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I
whipped up a little utility I call "schedtop".
>

Nice tool! Helps in better visualization of the data in schedstats. 

Using the tool, realized that most of the timing related stats therein
might not be completely usable in many scenarios, as might already be
known.

Without any additional load on the system, all the stats are nice and
sane. But, as soon as I ran my particular testcase, the data
pertaining to the delta of run_delay/cpu_time went haywire! I understand
that all the values are based on top of rq->clock, which relies on tsc that 
is not synced across cpus and would result in skews/incorrect values.
But, turns out to be not so reliable data for debugging. This is
ofcourse nothing related to the tool, but for schedstat in
general...rather just adding on to the already existing woes with non-syned 
tscs 

> This utility will process statistics from /proc/schedstat such that the busiest stats will bubble up to
the top.  It can alternately be sorted by the largest stat, or by name.  Stats can be included or excluded
based on reg-ex pattern matching.
> 
> You can download the tarball here:

(Continue reading)

Peter Zijlstra | 2 Jun 15:07

Re: [ANNOUNCE] sched: schedtop utility

On Mon, 2008-06-02 at 18:18 +0530, Ankita Garg wrote:
> Hi Gregory,
> 
> On Thu, May 22, 2008 at 08:06:44AM -0600, Gregory Haskins wrote:
> > Hi all scheduler developers,
> >   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, and it appears that the perl
scripts referenced in Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I
whipped up a little utility I call "schedtop".
> >
> 
> Nice tool! Helps in better visualization of the data in schedstats. 
> 
> Using the tool, realized that most of the timing related stats therein
> might not be completely usable in many scenarios, as might already be
> known.
> 
> Without any additional load on the system, all the stats are nice and
> sane. But, as soon as I ran my particular testcase, the data
> pertaining to the delta of run_delay/cpu_time went haywire! I understand
> that all the values are based on top of rq->clock, which relies on tsc that 
> is not synced across cpus and would result in skews/incorrect values.
> But, turns out to be not so reliable data for debugging. This is
> ofcourse nothing related to the tool, but for schedstat in
> general...rather just adding on to the already existing woes with non-syned 
> tscs 

Thing is, things runtime should be calculated by using per cpu deltas.
You take a stamp when you get scheduled on the cpu and another one when
you stop running, then the delta is added to runtime.

(Continue reading)

Gregory Haskins | 2 Jun 15:20

Re: [ANNOUNCE] sched: schedtop utility

Hi Ankita,
  For some reason, I didn't get your original email.  I had to go find it on the lkml.org archives.

But anyway, see inline

>>> On Mon, Jun 2, 2008 at  9:07 AM, in message
<1212412051.6269.5.camel <at> lappy.programming.kicks-ass.net>, Peter Zijlstra
<peterz <at> infradead.org> wrote: 
> On Mon, 2008-06-02 at 18:18 +0530, Ankita Garg wrote:
>> Hi Gregory,
>> 
>> On Thu, May 22, 2008 at 08:06:44AM -0600, Gregory Haskins wrote:
>> > Hi all scheduler developers,
>> >   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, 
> and it appears that the perl scripts referenced in 
> Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I 
> whipped up a little utility I call "schedtop".
>> >
>> 
>> Nice tool! Helps in better visualization of the data in schedstats. 
>> 
>> Using the tool, realized that most of the timing related stats therein
>> might not be completely usable in many scenarios, as might already be
>> known.
>> 
>> Without any additional load on the system, all the stats are nice and
>> sane. But, as soon as I ran my particular testcase, the data
>> pertaining to the delta of run_delay/cpu_time went haywire! I understand
>> that all the values are based on top of rq->clock, which relies on tsc that 
>> is not synced across cpus and would result in skews/incorrect values.

(Continue reading)

Ankita Garg | 5 Jun 07:20

Re: [ANNOUNCE] sched: schedtop utility

Hi,

On Mon, Jun 02, 2008 at 03:07:31PM +0200, Peter Zijlstra wrote:
> On Mon, 2008-06-02 at 18:18 +0530, Ankita Garg wrote:
> > Hi Gregory,
> > 
> > On Thu, May 22, 2008 at 08:06:44AM -0600, Gregory Haskins wrote:
> > > Hi all scheduler developers,
> > >   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, and it appears that the perl
scripts referenced in Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I
whipped up a little utility I call "schedtop".
> > >
> > 
> > Nice tool! Helps in better visualization of the data in schedstats. 
> > 
> > Using the tool, realized that most of the timing related stats therein
> > might not be completely usable in many scenarios, as might already be
> > known.
> > 
> > Without any additional load on the system, all the stats are nice and
> > sane. But, as soon as I ran my particular testcase, the data
> > pertaining to the delta of run_delay/cpu_time went haywire! I understand
> > that all the values are based on top of rq->clock, which relies on tsc that 
> > is not synced across cpus and would result in skews/incorrect values.
> > But, turns out to be not so reliable data for debugging. This is
> > ofcourse nothing related to the tool, but for schedstat in
> > general...rather just adding on to the already existing woes with non-syned 
> > tscs 
> 
> Thing is, things runtime should be calculated by using per cpu deltas.

(Continue reading)

Peter Zijlstra | 19 Jun 12:27

Re: [ANNOUNCE] sched: schedtop utility

On Thu, 2008-06-05 at 10:50 +0530, Ankita Garg wrote:

> Thanks Peter for the explanation...
>
> I agree with the above and that is the reason why I did not see weird
> values with cpu_time. But, run_delay still would suffer skews as the end
> points for delta could be taken on different cpus due to migration (more
> so on RT kernel due to the push-pull operations). With the below patch,
> I could not reproduce the issue I had seen earlier. After every dequeue,
> we take the delta and start wait measurements from zero when moved to a 
> different rq.

OK, so task delay delay accounting is broken because it doesn't take
migration into account.

What you've done is make it symmetric wrt enqueue, and account it like

  cpu0      cpu1

enqueue
 <wait-d1>
dequeue
            enqueue
             <wait-d2>
            run

Where you add both d1 and d2 to the run_delay,.. right?

This seems like a good fix, however it looks like the patch will break
compilation in !CONFIG_SCHEDSTATS && !CONFIG_TASK_DELAY_ACCT, of it

(Continue reading)

Ankita Garg | 1 Jul 11:00

Re: [ANNOUNCE] sched: schedtop utility

Hi Peter,

On Thu, Jun 19, 2008 at 12:27:14PM +0200, Peter Zijlstra wrote:
> On Thu, 2008-06-05 at 10:50 +0530, Ankita Garg wrote:
> 
> > Thanks Peter for the explanation...
> >
> > I agree with the above and that is the reason why I did not see weird
> > values with cpu_time. But, run_delay still would suffer skews as the end
> > points for delta could be taken on different cpus due to migration (more
> > so on RT kernel due to the push-pull operations). With the below patch,
> > I could not reproduce the issue I had seen earlier. After every dequeue,
> > we take the delta and start wait measurements from zero when moved to a 
> > different rq.
> 
> OK, so task delay delay accounting is broken because it doesn't take
> migration into account.
> 
> What you've done is make it symmetric wrt enqueue, and account it like
> 
>   cpu0      cpu1
> 
> enqueue
>  <wait-d1>
> dequeue
>             enqueue
>              <wait-d2>
>             run
> 
> Where you add both d1 and d2 to the run_delay,.. right?

(Continue reading)

Ingo Molnar | 3 Jul 10:34

Re: [ANNOUNCE] sched: schedtop utility


* Ankita Garg <ankita <at> in.ibm.com> wrote:

> > OK, so task delay delay accounting is broken because it doesn't take
> > migration into account.

> Fixed. Pl. find the new patch below.

applied to tip/sched/devel - thanks Ankita.

	Ingo

Peter Zijlstra | 3 Jul 22:56

Re: [ANNOUNCE] sched: schedtop utility

On Thu, 2008-07-03 at 10:34 +0200, Ingo Molnar wrote:
> * Ankita Garg <ankita <at> in.ibm.com> wrote:
> 
> > > OK, so task delay delay accounting is broken because it doesn't take
> > > migration into account.
> 
> > Fixed. Pl. find the new patch below.
> 
> applied to tip/sched/devel - thanks Ankita.

ACK

Gregory Haskins | 21 Oct 18:29

Re: [ANNOUNCE] sched: schedtop utility

I finally put up a wiki for my schedtop utility

http://rt.wiki.kernel.org/index.php/Schedtop_utility

Hopefully someone out there besides myself finds this tool useful!

Regards,
-Greg

Gregory Haskins | 3 Jun 05:21

[ANNOUNCE] sched: schedtop utility v0.3

>>> On Thu, May 22, 2008 at 10:06 AM, in message <483545B4.BA47.005A.0 <at> novell.com>,
Gregory Haskins wrote: 
> Hi all scheduler developers,
>   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, 
> and it appears that the perl scripts referenced in 
> Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I 
> whipped up a little utility I call "schedtop".
> 
> This utility will process statistics from /proc/schedstat such that the 
> busiest stats will bubble up to the top.  It can alternately be sorted by the 
> largest stat, or by name.  Stats can be included or excluded based on reg-ex 
> pattern matching.
> 
> You can download the tarball here:
> 
> ftp://ftp.novell.com/dev/ghaskins/schedtop.tar.gz
> 
> I have also posted it to the opensuse build service for generating RPMS for 
> a handful of 32/64-bit x86 distros for your convenience:
> 
> http://download.opensuse.org/repositories/home:/ghaskins/
> 
> (Note that the build is still in progress for some of the favors, so if you 
> do not see the flavor you are looking for, check back in a little while)
> 
> Comments/feedback/bug-fixes welcome!
> 
> Regards
> -Greg

(Continue reading)

Gregory Haskins | 17 Jun 14:18

[ANNOUNCE] sched: schedtop utility v0.5

Gregory Haskins wrote:
>>>> On Thu, May 22, 2008 at 10:06 AM, in message <483545B4.BA47.005A.0 <at> novell.com>,
> Gregory Haskins wrote: 
>> Hi all scheduler developers,
>>   I had an itch to scratch w.r.t. watching the stats in /proc/schedstats, 
>> and it appears that the perl scripts referenced in 
>> Documentation/scheduler/sched-stats.txt do not support v14 from HEAD so I 
>> whipped up a little utility I call "schedtop".
>>
>> This utility will process statistics from /proc/schedstat such that the 
>> busiest stats will bubble up to the top.  It can alternately be sorted by the 
>> largest stat, or by name.  Stats can be included or excluded based on reg-ex 
>> pattern matching.
>>
>> You can download the tarball here:
>>
>> ftp://ftp.novell.com/dev/ghaskins/schedtop.tar.gz
>>
>> I have also posted it to the opensuse build service for generating RPMS for 
>> a handful of 32/64-bit x86 distros for your convenience:
>>
>> http://download.opensuse.org/repositories/home:/ghaskins/
>>
>> (Note that the build is still in progress for some of the favors, so if you 
>> do not see the flavor you are looking for, check back in a little while)
>>
>> Comments/feedback/bug-fixes welcome!
>>
>> Regards
>> -Greg

(Continue reading)

Return

Return to gmane.linux.kernel.

Advertisement

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane