New tracking infrastructure for Linux systems

Lukas Berk | 2 Aug 2012 15:10

[RFC] Enhanced Garbage Collection Probe Points

Hey,

I've been working on adding improved probe point within the garbage
collection system.  This will allow system administrators using various
tools to better analyze which garbage collection algorithms are
effective and java developers to better understand how (often) their
objects are being collected.

Specific probe points that I've aimed to include are:

- G1, concurrent mark sweep, parallel mark sweep, and tenured
  collections

- new generation definitions

- parallel scavenges

- parallel compaction

- object 'moves/resizes' between memory addresses

Please note that the attached patch should be appended to the
patch/systemtap.patch file.  Any feedback or suggestions would be
greatly appreciated.

Cheers,

Lukas

(Continue reading)

headers

Mark Wielaard | 3 Aug 2012 15:55

Re: [RFC] Enhanced Garbage Collection Probe Points

On Thu, 2012-08-02 at 09:10 -0400, Lukas Berk wrote:
> I've been working on adding improved probe point within the garbage
> collection system.  This will allow system administrators using various
> tools to better analyze which garbage collection algorithms are
> effective and java developers to better understand how (often) their
> objects are being collected.
> 
> Specific probe points that I've aimed to include are:
> 
> - G1, concurrent mark sweep, parallel mark sweep, and tenured
>   collections
> 
> - new generation definitions
> 
> - parallel scavenges
> 
> - parallel compaction
> 
> - object 'moves/resizes' between memory addresses
> 
> Please note that the attached patch should be appended to the
> patch/systemtap.patch file.  Any feedback or suggestions would be
> greatly appreciated.

They look interesting, but do you have some example usage for this? My
garbage collector knowledge is very rusty. Just some simple description
when which point would be hit and what interesting data can be
collected/analysed at that time would be nice to put this in context.

Did you happen to talk to the thermostat team? They might give some

(Continue reading)

headers

Lukas Berk | 3 Aug 2012 16:22

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey Mark,

* Mark Wielaard <mark <at> klomp.org> [2012-08-03 09:56]:
> On Thu, 2012-08-02 at 09:10 -0400, Lukas Berk wrote:
[...]
> 
> They look interesting, but do you have some example usage for this? My
> garbage collector knowledge is very rusty. Just some simple description
> when which point would be hit and what interesting data can be
> collected/analysed at that time would be nice to put this in context.
I ran a $stap eventcount.stp hotspot.gc_* -o eventcount.txt style script
and have attached the output.  I'll also start to document the
script(tapset?) I've been using to interpret the probe points I've been
added here.
> 
> Did you happen to talk to the thermostat team? They might give some
> feedback on what kind of information they need and whether these probe
> points would cover their needs.

Yes I have, and I'm working towards what they'd like (idealy we'd like
to track objects from allocation, space to space, and eventually watch
them be collected), however I'd like to make sure this is the correct
starting point and work my way there, one step at a time :)

Cheers,

Lukas

Finished event counting at Fri Aug  3 10:09:35 2012 EDT.

(Continue reading)

headers

Mark Wielaard | 7 Aug 2012 13:50

Re: [RFC] Enhanced Garbage Collection Probe Points

On Fri, 2012-08-03 at 10:22 -0400, Lukas Berk wrote:
> * Mark Wielaard <mark <at> klomp.org> [2012-08-03 09:56]:
> > On Thu, 2012-08-02 at 09:10 -0400, Lukas Berk wrote:
> > They look interesting, but do you have some example usage for this? My
> > garbage collector knowledge is very rusty. Just some simple description
> > when which point would be hit and what interesting data can be
> > collected/analysed at that time would be nice to put this in context.
> I ran a $stap eventcount.stp hotspot.gc_* -o eventcount.txt style script
> and have attached the output.  I'll also start to document the
> script(tapset?) I've been using to interpret the probe points I've been
> added here.

If you could post eventcount.stp with some comments that would be really
helpful (even if it is just a work in progress) to better understand the
"meaning" of the new probe points for a Garbage Collector noob like me.

Thanks,

Mark

headers

Lukas Berk | 8 Aug 2012 16:01

Re: [RFC] Enhanced Garbage Collection Probe Points

* Mark Wielaard <mjw <at> redhat.com> [2012-08-07 07:50]:
[...]
> If you could post eventcount.stp with some comments that would be really
> helpful (even if it is just a work in progress) to better understand the
> "meaning" of the new probe points for a Garbage Collector noob like me.

Sure! I've added in comments below to the output I attached in the last
email. I've also attached the tapset/script I've written and have been
using with documentation.  For those of you using sytemtap, to use the
probe points, apply my original patch to icedtea, build, and then when
using stap, $stap -I /path/to/gc_details.stp <script.stp>

Finished event counting at Fri Aug  3 10:09:35 2012 EDT.
Total time elapsed: 1021574 ms, 63 events total, 63 after filtering.
TID                       EVENT                          COUNT (RATE Hz)
---                       -----                          ---------------
java(4998)                hotspot.gc_collect_parallel    1 (0.00)
^ This is a _full_ parallel collection of all objects from the program,
typically done when called explicitly with a System.gc() or at the end
of the program.  All of my collect type of probes also have a 'cause'
parameter for what initiated the gc.

java(5003)                hotspot.gc_collect_PSScavenge_begin 10 (0.00)
java(5003)                hotspot.gc_collect_PSScavenge_end 10 (0.00)
^ This marks the beginning and end of a parallel scavenge, I've marked
the beginning and end points in case there are concurrent parallel
scavenges going on.  (scavenges are partial garbage collections, so they
should happen much more often throughout the runtime of the program)

java(5003)                hotspot.gc_collect_delete      2 (0.00)

(Continue reading)

headers

Mario Torre | 8 Aug 2012 16:34

Re: [RFC] Enhanced Garbage Collection Probe Points

Il giorno mer, 08/08/2012 alle 10.01 -0400, Lukas Berk ha scritto:
> * Mark Wielaard <mjw <at> redhat.com> [2012-08-07 07:50]:
> [...]
> > If you could post eventcount.stp with some comments that would be really
> > helpful (even if it is just a work in progress) to better understand the
> > "meaning" of the new probe points for a Garbage Collector noob like me.

To be honest, I'm no expert of this (yet! :), but the patch looks good
to me in principle, and definitely very useful for our own thermostat
use cases, so, yeah, I would like to see this in if there are no
objections.

Cheers,
Mario

headers

Jon VanAlten | 8 Aug 2012 22:37

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Lukas Berk" <lberk <at> redhat.com>
> To: distro-pkg-dev <at> openjdk.java.net
> Cc: systemtap <at> sourceware.org
> Sent: Thursday, August 2, 2012 9:10:40 AM
> Subject: [RFC] Enhanced Garbage Collection Probe Points
> 
> Hey,
> 
> I've been working on adding improved probe point within the garbage
> collection system.  This will allow system administrators using
> various
> tools to better analyze which garbage collection algorithms are
> effective and java developers to better understand how (often) their
> objects are being collected.
> 
> Specific probe points that I've aimed to include are:
> 
> - G1, concurrent mark sweep, parallel mark sweep, and tenured
>   collections
> 
> - new generation definitions
> 
> - parallel scavenges
> 
> - parallel compaction
> 
> - object 'moves/resizes' between memory addresses
>

(Continue reading)

headers

Lukas Berk | 8 Aug 2012 23:11

Re: [RFC] Enhanced Garbage Collection Probe Points

* Jon VanAlten <jvanalte <at> redhat.com> [2012-08-08 16:38]:
[...]
> 
> Hi Lukas,
> 
> I've had a look at the patch, and the rest of the thread (especially the
> tapset you posted, which made things a LOT more clear so thanks for that).
> I hope that you'll continue to refine the tapset and contribute that as
> well so that us java hackers who are brave enough to play with Systemtap
> have the nice friendly probe names and variables to work with!
Thanks for taking a look, I'll make sure to include it in future
revisions, ideally I'll have a gc_details.stp.in file that will have the
.so path change based on the system it's being compiled for. (similar to
the hotspot{,_jni}.stp.in and jstack.stp.in files already included)
> 
> The patch itself seems fine, although I'll just have to trust you that the
> probes are in the right place as I'm not really a hotspot hacker myself.
> Really I would hope that someone who is more familiar with the GC code
> base will comment from that perspective.
> 
> I assume that none of these probes require any special VM args to function?
> A related question, and this might be hard to answer being rather open
> ended, but are you aware of any VM args that would affect/interfere with
> the functioning of these probes?
The only arguments that I can think of that will immediately effect
these probes are:
-XX:+UseSerialGC
-XX:+UseParallelGC
-XX:+UseParallelOldGC
-XX:+UseParNewGC

(Continue reading)

headers

Jon VanAlten | 9 Aug 2012 00:04

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Lukas Berk" <lberk <at> redhat.com>
> To: "Jon VanAlten" <jvanalte <at> redhat.com>
> Cc: systemtap <at> sourceware.org, distro-pkg-dev <at> openjdk.java.net
> Sent: Wednesday, August 8, 2012 5:11:23 PM
> Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> 
> * Jon VanAlten <jvanalte <at> redhat.com> [2012-08-08 16:38]:
> [...]
> > 
> > Hi Lukas,
> > 
> > I've had a look at the patch, and the rest of the thread
> > (especially the
> > tapset you posted, which made things a LOT more clear so thanks for
> > that).
> > I hope that you'll continue to refine the tapset and contribute
> > that as
> > well so that us java hackers who are brave enough to play with
> > Systemtap
> > have the nice friendly probe names and variables to work with!
> Thanks for taking a look, I'll make sure to include it in future
> revisions, ideally I'll have a gc_details.stp.in file that will have
> the
> .so path change based on the system it's being compiled for. (similar
> to
> the hotspot{,_jni}.stp.in and jstack.stp.in files already included)

Yep, that would be the Right Way for it to be included!

(Continue reading)

headers

Lukas Berk | 22 Aug 2012 22:33

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey list,

I've updated and attached my patch to reflect the comments made in 
the thread above, primarily:

- split begin and end probe points where possible

- correct probe point names

- include and autoconf version of the tapset for immediate inclusion

I've also run the DaCapo benchmarking suite[1] and have attached the
results below.

In order to properly judge the performance effects (if any) my probe
points have had on hotspot, I ran three different sets of tests
(using the eclipse benchmark).  First, with icedtea configured using
--disable-systemtap, then configured with systemtap and no probes
run, and finally with the gc probes in use with a basic log(probestr)
systemtap script. (ie $stap -ve 'probe hotspot.gc_* {log(probestr)}'
-c "java -jar dacapo-9.12-bach.jar eclise -n 10". I ran each test 4
times to get a better sample size.

--disable-systemtap:
run 1: 44119 msec
run 2: 43255 msec
run 3: 44517 msec
run 4: 44017 msec
--------------------
avg: 43977 msec

(Continue reading)

headers

Jon VanAlten | 24 Aug 2012 21:59

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Lukas Berk" <lberk <at> redhat.com>
> To: distro-pkg-dev <at> openjdk.java.net
> Cc: systemtap <at> sourceware.org
> Sent: Wednesday, August 22, 2012 4:33:09 PM
> Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> 
> Hey list,
> 
> I've updated and attached my patch to reflect the comments made in
> the thread above, primarily:
> 
> - split begin and end probe points where possible
> 
> - correct probe point names
> 
> - include and autoconf version of the tapset for immediate inclusion
> 

Definite improvements!

There's some nit's I'd like to see addressed.  Mostly consistency and
clarity in comments/documentation.  Details:

We have a hotspot.stp, a hotspot_jni.stp; maybe this one should be
hotspot_gc.stp?

Adding this fluff to the existing systemtap.patch, may not be the
best way.  As I understand it, Mark is trying to get the existing

(Continue reading)

headers

Lukas Berk | 28 Aug 2012 23:22

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey Jon,

Thanks for the comments, I've resolved them or explained them inline.
* Jon VanAlten <jvanalte@...> [2012-08-24 15:59]:
> 
[...]
> We have a hotspot.stp, a hotspot_jni.stp; maybe this one should be
> hotspot_gc.stp?
Done, its now included as hotspot_gc.stp.in
> 
> Adding this fluff to the existing systemtap.patch, may not be the
> best way.  As I understand it, Mark is trying to get the existing
> stuff upstream.  Combining this into same IcedTea patch may make
> it more difficult if/when this upstreaming is eventually successful
> and needs to be removed from IcedTea.  I'd suggest introducing a
> separate patch to the icedtea sources.
Makes sense to me, I've adjusted the autoconf files accordingly to
include and apply patches/systemtap_gc.patch if icedtea is configure
with --enable-systemtap.
> 
> + *  <at> size: word size to be cleaned
> 
> (in several places)
> This wording (pun not intended) is not entirely clear.  Playing the
> ignorant reader, I am not sure if this means the wordsize that the
> jvm uses, or the total space, in words, that will be collected on
> this gc run.  Can this be clarified?
I've changed this to: "Word size of the object to be collected."
> 
> + *  <at> is_full: If TRUE, attempt a full collection of the generation

(Continue reading)

headers

Mark Wielaard | 29 Aug 2012 13:52

Re: [RFC] Enhanced Garbage Collection Probe Points

Hi Lukas,

On Tue, 2012-08-28 at 17:22 -0400, Lukas Berk wrote:
> > We have a hotspot.stp, a hotspot_jni.stp; maybe this one should be
> > hotspot_gc.stp?
> Done, its now included as hotspot_gc.stp.in

I like it this way. Thanks.

> > Adding this fluff to the existing systemtap.patch, may not be the
> > best way.  As I understand it, Mark is trying to get the existing
> > stuff upstream.  Combining this into same IcedTea patch may make
> > it more difficult if/when this upstreaming is eventually successful
> > and needs to be removed from IcedTea.  I'd suggest introducing a
> > separate patch to the icedtea sources.
> Makes sense to me, I've adjusted the autoconf files accordingly to
> include and apply patches/systemtap_gc.patch if icedtea is configure
> with --enable-systemtap.

That also looks good. For 6 I split up the original systemtap.patch into
separate patches too. See
http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2012-August/019851.html
Eventually all those patches should go directly into the forest (or
Andrew Hughes will kill me for polluting the tree with even more
patches) but lets do all those in one step later. For now the separate
patch is great.

Please do post and ping systemtap_gc.patch to
hotspot-gc-dev <at> openjdk.java.net one last time. It would be really good
if some hotspot gc hacker could comment. We can always try.

(Continue reading)

headers

Jon VanAlten | 30 Aug 2012 03:19

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey Lukas,

Nice improvement.  I've snipped the stuff that looks all-good now.

> > 
> > + *  <at> size: word size to be cleaned
> > 
> > (in several places)
> > This wording (pun not intended) is not entirely clear.  Playing the
> > ignorant reader, I am not sure if this means the wordsize that the
> > jvm uses, or the total space, in words, that will be collected on
> > this gc run.  Can this be clarified?
> I've changed this to: "Word size of the object to be collected."

So, this maybe is not a great improvement.  Does this mean that in the
gc run there will be a single object collected?  Is that single object
 <at> size words big?  I think I know better that this is not what you
mean here, but it reads unclear.  Let me ask my question another way,
is  <at> size a target for this collection run?  Or is it the size of the
region to be collected?

This also introduces a terminology collision for java-centric folks.
I think you maybe mean 'region' or 'generation' rather than 'object',
which has a specific meaning in Java-land.  This is also an issue
below...

If the meaning of this value is what I think it is, can I suggest:

"The number of words of memory that this collection will try to reclaim"

(Continue reading)

headers

Lukas Berk | 31 Aug 2012 22:06

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey,

[...]
> > > this gc run.  Can this be clarified?
> > I've changed this to: "Word size of the object to be collected."
> 
> So, this maybe is not a great improvement.  Does this mean that in the
> gc run there will be a single object collected?  Is that single object
>  <at> size words big?  I think I know better that this is not what you
> mean here, but it reads unclear.  Let me ask my question another way,
> is  <at> size a target for this collection run?  Or is it the size of the
> region to be collected?
> 
> This also introduces a terminology collision for java-centric folks.
> I think you maybe mean 'region' or 'generation' rather than 'object',
> which has a specific meaning in Java-land.  This is also an issue
> below...
> 
> If the meaning of this value is what I think it is, can I suggest:
> 
> "The number of words of memory that this collection will try to reclaim"

As per our discussion on irc, I've changed this to "The collection
should achieve a minimum region of available memory to allow for an
allocation of 'size'." This should satisfy all possible situations where
where  <at> size is used.

> 
> > > 
> > > + * probe - gc_collect_parallel_scavenge

(Continue reading)

headers

Jon VanAlten | 24 Sep 2012 20:16

Re: [RFC] Enhanced Garbage Collection Probe Points


> 
> > > Is this ok to commit?
> > > 
> > 
> > With caveats noted above, I say yes :)
> > 
> 
> Thanks! Please let me know if there is anything else you'd like to
> know,
> updated patch is attached.
> 

Hi,

This has been in my TODO for some time, so I am very sorry not to
have responded earlier.  From looking at the patch, it seems okay
now.  I intend to push this to HEAD on your behalf, unless someone
else has some reason why not.  (Now is your chance to speak up
about that, if you have such a reason!).  But, I do feel responsible
to build with it myself and verify that things seem to be working as
intended.  Various other things have been jumping my work queue, but
I may have time this week to give this a sanity check and finally
get it into hg.  Thanks for persisting!

cheers,
jon

headers

Lukas Berk | 23 Oct 2012 18:53

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey,

* Jon VanAlten <jvanalte <at> redhat.com> [2012-09-24 14:16]:
> 
> > 
> > > > Is this ok to commit?
> > > > 
> > > 
> > > With caveats noted above, I say yes :)
> > > 
> > 
> > Thanks! Please let me know if there is anything else you'd like to
> > know,
> > updated patch is attached.
> > 
> 
> Hi,
> 
> This has been in my TODO for some time, so I am very sorry not to
> have responded earlier.  From looking at the patch, it seems okay
> now.  I intend to push this to HEAD on your behalf, unless someone
> else has some reason why not.  (Now is your chance to speak up
> about that, if you have such a reason!).  But, I do feel responsible
> to build with it myself and verify that things seem to be working as
> intended.  Various other things have been jumping my work queue, but
> I may have time this week to give this a sanity check and finally
> get it into hg.  Thanks for persisting!
> 
> cheers,
> Jon

(Continue reading)

headers

Jon VanAlten | 25 Oct 2012 07:02

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Lukas Berk" <lberk@...>
> To: "Jon VanAlten" <jvanalte@...>
> Cc: systemtap@..., distro-pkg-dev@...
> Sent: Tuesday, October 23, 2012 12:53:30 PM
> Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> 
> Hey,
> 
> * Jon VanAlten <jvanalte@...> [2012-09-24 14:16]:
> > 
> > > 
> > > > > Is this ok to commit?
> > > > > 
> > > > 
> > > > With caveats noted above, I say yes :)
> > > > 
> > > 
> > > Thanks! Please let me know if there is anything else you'd like
> > > to
> > > know,
> > > updated patch is attached.
> > > 
> > 
> > Hi,
> > 
> > This has been in my TODO for some time, so I am very sorry not to
> > have responded earlier.  From looking at the patch, it seems okay
> > now.  I intend to push this to HEAD on your behalf, unless someone

(Continue reading)

headers

Andrew Hughes | 25 Oct 2012 15:08

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Lukas Berk" <lberk <at> redhat.com>
> > To: "Jon VanAlten" <jvanalte <at> redhat.com>
> > Cc: systemtap <at> sourceware.org, distro-pkg-dev <at> openjdk.java.net
> > Sent: Tuesday, October 23, 2012 12:53:30 PM
> > Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> > 
> > Hey,
> > 
> > * Jon VanAlten <jvanalte <at> redhat.com> [2012-09-24 14:16]:
> > > 
> > > > 
> > > > > > Is this ok to commit?
> > > > > > 
> > > > > 
> > > > > With caveats noted above, I say yes :)
> > > > > 
> > > > 
> > > > Thanks! Please let me know if there is anything else you'd like
> > > > to
> > > > know,
> > > > updated patch is attached.
> > > > 
> > > 
> > > Hi,
> > >

(Continue reading)

headers

Jon VanAlten | 30 Oct 2012 02:49

Re: [RFC] Enhanced Garbage Collection Probe Points


> > > 
> > > I've updated the patch slightly to reflect a correction in
> > > psScavenge.cpp that was causing a build error when icedtea-debug
> > > wasn't
> > > specified.  The rest of the patch is unchanged from before.
> > > 
> > 
> > And I've now pushed this.  I know it's preferred that new work
> > goes to forest in general, but this was discussed earlier when
> > patch was originally posted, and afaict the feeling was it
> > would be better to move all systemtap related patches there in
> > one step some time later[1].
> > 
> > Thanks Lukas for this contribution!
> > 
> 
> Why has this been pushed?  We don't want more patches adding to
> IcedTea7.
> Please revert.
> 

I'm sorry for the mix-up.  Andrew and I discussed this further
offline (sorry, this did not happen in public, but neither was it
intentionally private).

We reached the conclusion (I believe) that it would be best to
wait and hear from Mark about plans referred to in email archive
link below.  Mark, do you have any comment here?  Would it be
for the best to revert here and apply instead to 7 forest?

(Continue reading)

headers

Mark Wielaard | 30 Oct 2012 09:03

Re: [RFC] Enhanced Garbage Collection Probe Points

On Mon, Oct 29, 2012 at 09:49:17PM -0400, Jon VanAlten wrote:
> We reached the conclusion (I believe) that it would be best to
> wait and hear from Mark about plans referred to in email archive
> link below.  Mark, do you have any comment here?  Would it be
> for the best to revert here and apply instead to 7 forest?

Adding patches directly to the tree is fine with me.
My only hesitation was my own confusion since the default
configure/make setup doesn't pick up a patched forest.
You don't have that issue with patches, which are directly
applied. If we are going for a complete forest setup it
might make sense to also add the tapsets and testsuite
directly there.

I'll try and figure it all out again and also port the existing
patches to the forest. Hints and tips appreciated.

Cheers,

Mark

headers

Andrew Hughes | 30 Oct 2012 13:37

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> 
> > > > 
> > > > I've updated the patch slightly to reflect a correction in
> > > > psScavenge.cpp that was causing a build error when
> > > > icedtea-debug
> > > > wasn't
> > > > specified.  The rest of the patch is unchanged from before.
> > > > 
> > > 
> > > And I've now pushed this.  I know it's preferred that new work
> > > goes to forest in general, but this was discussed earlier when
> > > patch was originally posted, and afaict the feeling was it
> > > would be better to move all systemtap related patches there in
> > > one step some time later[1].
> > > 
> > > Thanks Lukas for this contribution!
> > > 
> > 
> > Why has this been pushed?  We don't want more patches adding to
> > IcedTea7.
> > Please revert.
> > 
> 
> I'm sorry for the mix-up.  Andrew and I discussed this further
> offline (sorry, this did not happen in public, but neither was it
> intentionally private).
> 
> We reached the conclusion (I believe) that it would be best to

(Continue reading)

headers

Jon VanAlten | 1 Nov 2012 19:32

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Andrew Hughes" <gnu.andrew <at> redhat.com>

<SNIP>

> > > 
> > > Why has this been pushed?  We don't want more patches adding to
> > > IcedTea7.
> > > Please revert.
> > > 
> > 
> > I'm sorry for the mix-up.  Andrew and I discussed this further
> > offline (sorry, this did not happen in public, but neither was it
> > intentionally private).
> > 
> > We reached the conclusion (I believe) that it would be best to
> > wait and hear from Mark about plans referred to in email archive
> > link below.  Mark, do you have any comment here?  Would it be
> > for the best to revert here and apply instead to 7 forest?
> > 
> > Also, I've backported this changeset to icedtea6, and it should
> > also probably go somehow to 8.  Can someone refresh me, as a
> > rather seldom-contributor here, what the correct repos are for
> > these?
> > 
> 
> Patch needs to be applied to:
> 
> http://icedtea.classpath.org/hg/icedtea6/

(Continue reading)

headers

Mark Wielaard | 1 Nov 2012 20:05

Re: [RFC] Enhanced Garbage Collection Probe Points

On Thu, 2012-11-01 at 14:32 -0400, Jon VanAlten wrote:
> I would also prefer the 7 stuff to go to the forest, but...
> 
> > From: "Mark Wielaard" <mark <at> klomp.org>
> 
> <SNIP>
> > 
> > Adding patches directly to the tree is fine with me.
> > My only hesitation was my own confusion since the default
> > configure/make setup doesn't pick up a patched forest.
> > You don't have that issue with patches, which are directly
> > applied. If we are going for a complete forest setup it
> > might make sense to also add the tapsets and testsuite
> > directly there.
> > 
> > I'll try and figure it all out again and also port the existing
> > patches to the forest. Hints and tips appreciated.
> 
> So about this patch and 7, I'm getting mixed messages here.

What is the mixed message? I am fine with adding patches directly to the
forest and even adding the other stuff like the tapsets and the
staptests there. I just don't know the workflow when I would work
against the forest directly and not just have patches in the tree. And I
just haven't found the time to create a new workflow. I am sure it is
just a different ./configure flag, just haven't figured out which one.
So, how do people work against the forest directly? What configure flags
do they use? What does your edit/build/test/commit/pull cycle look like?

Basically the only step I seem to miss is how to make sure the tree is

(Continue reading)

headers

Jon VanAlten | 1 Nov 2012 23:11

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Mark Wielaard" <mark <at> klomp.org>
> To: "Jon VanAlten" <jvanalte <at> redhat.com>
> Cc: "Andrew Hughes" <gnu.andrew <at> redhat.com>, systemtap <at> sourceware.org,
distro-pkg-dev <at> openjdk.java.net, "Lukas Berk"
> <lberk <at> redhat.com>
> Sent: Thursday, November 1, 2012 3:05:45 PM
> Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> 
> On Thu, 2012-11-01 at 14:32 -0400, Jon VanAlten wrote:
> > I would also prefer the 7 stuff to go to the forest, but...
> > 
> > > From: "Mark Wielaard" <mark <at> klomp.org>
> > 
> > <SNIP>
> > > 
> > > Adding patches directly to the tree is fine with me.
> > > My only hesitation was my own confusion since the default
> > > configure/make setup doesn't pick up a patched forest.
> > > You don't have that issue with patches, which are directly
> > > applied. If we are going for a complete forest setup it
> > > might make sense to also add the tapsets and testsuite
> > > directly there.
> > > 
> > > I'll try and figure it all out again and also port the existing
> > > patches to the forest. Hints and tips appreciated.
> > 
> > So about this patch and 7, I'm getting mixed messages here.
>

(Continue reading)

headers

Andrew Hughes | 2 Nov 2012 15:53

Re: [RFC] Enhanced Garbage Collection Probe Points

----- Original Message -----
> On Thu, 2012-11-01 at 14:32 -0400, Jon VanAlten wrote:
> > I would also prefer the 7 stuff to go to the forest, but...
> > 
> > > From: "Mark Wielaard" <mark <at> klomp.org>
> > 
> > <SNIP>
> > > 
> > > Adding patches directly to the tree is fine with me.
> > > My only hesitation was my own confusion since the default
> > > configure/make setup doesn't pick up a patched forest.
> > > You don't have that issue with patches, which are directly
> > > applied. If we are going for a complete forest setup it
> > > might make sense to also add the tapsets and testsuite
> > > directly there.
> > > 
> > > I'll try and figure it all out again and also port the existing
> > > patches to the forest. Hints and tips appreciated.
> > 
> > So about this patch and 7, I'm getting mixed messages here.
> 

Mark, is this the whole of your patch or only part of it?

http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/75982791ddb6

i.e. could that be added to forest and systemtap.patch removed from IcedTea7?

> What is the mixed message? I am fine with adding patches directly to
> the

(Continue reading)

headers

Mark Wielaard | 2 Nov 2012 16:03

Re: [RFC] Enhanced Garbage Collection Probe Points

On Fri, 2012-11-02 at 10:53 -0400, Andrew Hughes wrote:
> Mark, is this the whole of your patch or only part of it?
> 
> http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/75982791ddb6
> 
> i.e. could that be added to forest and systemtap.patch removed from IcedTea7?

Yes it is, plus the followup patch to add the testcase I wrote for it:
http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/26351ce8c4b0

Cheers,

Mark

headers

Andrew Hughes | 30 Oct 2012 13:37

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> 
> > > > 
> > > > I've updated the patch slightly to reflect a correction in
> > > > psScavenge.cpp that was causing a build error when
> > > > icedtea-debug
> > > > wasn't
> > > > specified.  The rest of the patch is unchanged from before.
> > > > 
> > > 
> > > And I've now pushed this.  I know it's preferred that new work
> > > goes to forest in general, but this was discussed earlier when
> > > patch was originally posted, and afaict the feeling was it
> > > would be better to move all systemtap related patches there in
> > > one step some time later[1].
> > > 
> > > Thanks Lukas for this contribution!
> > > 
> > 
> > Why has this been pushed?  We don't want more patches adding to
> > IcedTea7.
> > Please revert.
> > 
> 
> I'm sorry for the mix-up.  Andrew and I discussed this further
> offline (sorry, this did not happen in public, but neither was it
> intentionally private).
> 
> We reached the conclusion (I believe) that it would be best to

(Continue reading)

headers

Jon VanAlten | 25 Oct 2012 07:02

Re: [RFC] Enhanced Garbage Collection Probe Points


----- Original Message -----
> From: "Lukas Berk" <lberk <at> redhat.com>
> To: "Jon VanAlten" <jvanalte <at> redhat.com>
> Cc: systemtap <at> sourceware.org, distro-pkg-dev <at> openjdk.java.net
> Sent: Tuesday, October 23, 2012 12:53:30 PM
> Subject: Re: [RFC] Enhanced Garbage Collection Probe Points
> 
> Hey,
> 
> * Jon VanAlten <jvanalte <at> redhat.com> [2012-09-24 14:16]:
> > 
> > > 
> > > > > Is this ok to commit?
> > > > > 
> > > > 
> > > > With caveats noted above, I say yes :)
> > > > 
> > > 
> > > Thanks! Please let me know if there is anything else you'd like
> > > to
> > > know,
> > > updated patch is attached.
> > > 
> > 
> > Hi,
> > 
> > This has been in my TODO for some time, so I am very sorry not to
> > have responded earlier.  From looking at the patch, it seems okay
> > now.  I intend to push this to HEAD on your behalf, unless someone

(Continue reading)

headers

Lukas Berk | 31 Aug 2012 22:06

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey,

[...]
> > > this gc run.  Can this be clarified?
> > I've changed this to: "Word size of the object to be collected."
> 
> So, this maybe is not a great improvement.  Does this mean that in the
> gc run there will be a single object collected?  Is that single object
>  <at> size words big?  I think I know better that this is not what you
> mean here, but it reads unclear.  Let me ask my question another way,
> is  <at> size a target for this collection run?  Or is it the size of the
> region to be collected?
> 
> This also introduces a terminology collision for java-centric folks.
> I think you maybe mean 'region' or 'generation' rather than 'object',
> which has a specific meaning in Java-land.  This is also an issue
> below...
> 
> If the meaning of this value is what I think it is, can I suggest:
> 
> "The number of words of memory that this collection will try to reclaim"

As per our discussion on irc, I've changed this to "The collection
should achieve a minimum region of available memory to allow for an
allocation of 'size'." This should satisfy all possible situations where
where  <at> size is used.

> 
> > > 
> > > + * probe - gc_collect_parallel_scavenge

(Continue reading)

headers

Lukas Berk | 28 Aug 2012 23:22

Re: [RFC] Enhanced Garbage Collection Probe Points

Hey Jon,

Thanks for the comments, I've resolved them or explained them inline.
* Jon VanAlten <jvanalte <at> redhat.com> [2012-08-24 15:59]:
> 
[...]
> We have a hotspot.stp, a hotspot_jni.stp; maybe this one should be
> hotspot_gc.stp?
Done, its now included as hotspot_gc.stp.in
> 
> Adding this fluff to the existing systemtap.patch, may not be the
> best way.  As I understand it, Mark is trying to get the existing
> stuff upstream.  Combining this into same IcedTea patch may make
> it more difficult if/when this upstreaming is eventually successful
> and needs to be removed from IcedTea.  I'd suggest introducing a
> separate patch to the icedtea sources.
Makes sense to me, I've adjusted the autoconf files accordingly to
include and apply patches/systemtap_gc.patch if icedtea is configure
with --enable-systemtap.
> 
> + *  <at> size: word size to be cleaned
> 
> (in several places)
> This wording (pun not intended) is not entirely clear.  Playing the
> ignorant reader, I am not sure if this means the wordsize that the
> jvm uses, or the total space, in words, that will be collected on
> this gc run.  Can this be clarified?
I've changed this to: "Word size of the object to be collected."
> 
> + *  <at> is_full: If TRUE, attempt a full collection of the generation

(Continue reading)