Andreas Sewe | 16 May 2012 10:56
Picon
Favicon

Obtaining methods executed at least once?

Hi all,

I hope this is the right mailing list to ask such questions (if not,
please point me to the right one), as I couldn't find an obvious
analogue to lists like jikesrvm-researchers.

Is the a way (preferably using "diagnostic" rather than "develop"
options) to obtain a list of all methods executed at least once?
Currently I am considering "-XX:-UseInterpreter -XX:+LogCompilation",
but this seems to be an awfully indirect way to achieve my goal -- and
it may not even give correct results if the only call of m() is inlined
during compilation.

Any help is greatly appreciated.

Best wishes,

Andreas

David Holmes | 16 May 2012 12:11
Picon
Favicon

Re: Obtaining methods executed at least once?

Andreas,

I've cc'd this to hotspot-dev which is the right place to ask.

Please drop the discuss list from further replies.

David

On 16/05/2012 6:56 PM, Andreas Sewe wrote:
> Hi all,
>
> I hope this is the right mailing list to ask such questions (if not,
> please point me to the right one), as I couldn't find an obvious
> analogue to lists like jikesrvm-researchers.
>
> Is the a way (preferably using "diagnostic" rather than "develop"
> options) to obtain a list of all methods executed at least once?
> Currently I am considering "-XX:-UseInterpreter -XX:+LogCompilation",
> but this seems to be an awfully indirect way to achieve my goal -- and
> it may not even give correct results if the only call of m() is inlined
> during compilation.
>
> Any help is greatly appreciated.
>
> Best wishes,
>
> Andreas

Keith McGuigan | 16 May 2012 13:52
Picon
Favicon

Re: Obtaining methods executed at least once?


Have you considered using a JVMTI agent and catching the method 
entry/return events?  You'd have to do some work in the agent to ignore 
duplicates though.

--
- Keith

On 5/16/2012 6:11 AM, David Holmes wrote:
> Andreas,
>
> I've cc'd this to hotspot-dev which is the right place to ask.
>
> Please drop the discuss list from further replies.
>
> David
>
> On 16/05/2012 6:56 PM, Andreas Sewe wrote:
>> Hi all,
>>
>> I hope this is the right mailing list to ask such questions (if not,
>> please point me to the right one), as I couldn't find an obvious
>> analogue to lists like jikesrvm-researchers.
>>
>> Is the a way (preferably using "diagnostic" rather than "develop"
>> options) to obtain a list of all methods executed at least once?
>> Currently I am considering "-XX:-UseInterpreter -XX:+LogCompilation",
>> but this seems to be an awfully indirect way to achieve my goal -- and
>> it may not even give correct results if the only call of m() is inlined
>> during compilation.
(Continue reading)

Andreas Sewe | 16 May 2012 15:03
Picon
Favicon

Re: Obtaining methods executed at least once?

Hi Keith,

> Have you considered using a JVMTI agent and catching the method
> entry/return events?  You'd have to do some work in the agent to ignore
> duplicates though.

yes, I did consider using JVMTI. Unfortunately, there's no proper event
for this, and handling all JVMTI_EVENT_METHOD_ENTRYs is very expensive,
even if it is "just" looking up jmethodID's in a map most of the time.

Thus I was thinking about forcing compilation (-XX:-UseInterpreter) and
then using the compilation log. I am unsure, however, if this really
gives me all methods, i.e., if -XX:-UseInterpreter is respected all the
time.

Best wishes,

Andreas

Krystal Mok | 16 May 2012 16:25
Picon
Gravatar

Re: Obtaining methods executed at least once?

Hi Andreas,


There are a couple of ways off the top of my head. Their viability depends on the original purpose of getting the list of executed methods.

HotSpot keeps the invocation count (at least in the interpreter) for all Java methods. It may not be precise, because it'll decay periodically, or it may be set to certain values for triggering standard compilation, etc. But if you only care about zero and non-zero, this counter would be sufficient.

The problem is: how to get the list of invoked methods?
There's no flag within the VM that I'm aware of that would print the list of invoked methods, unfortunately.

You could try the Serviceability Agent. See this example that I just made: https://gist.github.com/2710667#file_print_invoked_methods.java

It can attach to your target Java process, and extract all invoked methods.

Caveat:

1. If you're on JDK6, you may need to include these two VM flags in your target Java application:
-XX:-UseFastEmptyMethods -XX:-UseFastAccessorMethods
Otherwise empty methods and accessor methods will not show up in the list, because the "fast" version doesn't increment the invocation counter for these methods.
In JDK7 these two flags default to false, so you don't have to bother setting them to false explicitly.

2. If any of your classes are unloaded when you use this tool, then their methods won't show up in the output.

3. If you'd like the list to be printed before process exit, then an easy way to do this is to set up a shutdown hook with a Java agent, and make this hook run the tool above (in a new process). You may miss some methods that are only run in the shutdown hook, but I guess that's an acceptable tradeoff.

- Kris

On Wed, May 16, 2012 at 6:11 PM, David Holmes <david.holmes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
Andreas,

I've cc'd this to hotspot-dev which is the right place to ask.

Please drop the discuss list from further replies.

David

On 16/05/2012 6:56 PM, Andreas Sewe wrote:
Hi all,

I hope this is the right mailing list to ask such questions (if not,
please point me to the right one), as I couldn't find an obvious
analogue to lists like jikesrvm-researchers.

Is the a way (preferably using "diagnostic" rather than "develop"
options) to obtain a list of all methods executed at least once?
Currently I am considering "-XX:-UseInterpreter -XX:+LogCompilation",
but this seems to be an awfully indirect way to achieve my goal -- and
it may not even give correct results if the only call of m() is inlined
during compilation.

Any help is greatly appreciated.

Best wishes,

Andreas

Andreas Sewe | 16 May 2012 16:39
Picon
Favicon

Re: Obtaining methods executed at least once?

Hi,

> There are a couple of ways off the top of my head. Their viability
> depends on the original purpose of getting the list of executed methods.
> 
> HotSpot keeps the invocation count (at least in the interpreter) for all
> Java methods. It may not be precise, because it'll decay periodically,
> or it may be set to certain values for triggering standard compilation,
> etc. But if you only care about zero and non-zero, this counter would be
> sufficient.

Yes, I only care about that.

The question is whether I can force interpretation for all methods (so
that the counts are incremented) by doing -XX:-UseCompiler?

> The problem is: how to get the list of invoked methods?
> There's no flag within the VM that I'm aware of that would print the
> list of invoked methods, unfortunately.
> 
> You could try the Serviceability Agent. See this example that I just
> made: https://gist.github.com/2710667#file_print_invoked_methods.java
> 
> It can attach to your target Java process, and extract all invoked methods.

Neat. :-)

> Caveat:
> 
> 3. If you'd like the list to be printed before process exit, then an
> easy way to do this is to set up a shutdown hook with a Java agent, and
> make this hook run the tool above (in a new process). You may miss some
> methods that are only run in the shutdown hook, but I guess that's an
> acceptable tradeoff.

Yes, that's fine. Thanks.

Best wishes,

Andreas

Krystal Mok | 16 May 2012 16:48
Picon
Gravatar

Re: Obtaining methods executed at least once?

On Wed, May 16, 2012 at 10:39 PM, Andreas Sewe <sewe-ragVMc8tzzHbA/2l3k8Gwdu65OH5Cw0ybI1oV63ZYDM@public.gmane.org> wrote:
The question is whether I can force interpretation for all methods (so
that the counts are incremented) by doing -XX:-UseCompiler?

I don't think you have to anything to "force" a Java method to be interpreted first, because that's the default behavior already. Since you only care about "non-zero" invocation counts, running a method in the interpreter for even just once is enough, and it doesn't matter if the method got compiled later.

So long as you don't do -Xcomp sort of stuff, you should get the correct list of invoked methods this way. And with the -XX:-UseFastEmptyMethods -XX:-UseFastAccessorMethods caveat.

Hopefully I'm not missing anything... :-)

- Kris
 
Best wishes,

Andreas

Ben Evans | 16 May 2012 14:04
Picon
Gravatar

Re: Obtaining methods executed at least once?

Hi Andreas,

OK, I'll bite with a couple of questions:

1) Why do you want this information? Is it purely informative, or are
you planning to make use of it in some way? If you're planning to use
it - what uses do you have in mind?

2) What do you mean by "inlined at compilation". When dealing with
HotSpot, always remember that there are two separate things which
could be thought of as 'compilation' - there's the process which
creates classfiles (e.g. javac) and there's the process which creates
machine code from hot methods - JIT compilation. It isn't clear to me
which of these two processes you mean.

Thanks,

Ben

On Wed, May 16, 2012 at 7:11 AM, David Holmes <david.holmes@...> wrote:
> Andreas,
>
> I've cc'd this to hotspot-dev which is the right place to ask.
>
> Please drop the discuss list from further replies.
>
> David
>
>
> On 16/05/2012 6:56 PM, Andreas Sewe wrote:
>>
>> Hi all,
>>
>> I hope this is the right mailing list to ask such questions (if not,
>> please point me to the right one), as I couldn't find an obvious
>> analogue to lists like jikesrvm-researchers.
>>
>> Is the a way (preferably using "diagnostic" rather than "develop"
>> options) to obtain a list of all methods executed at least once?
>> Currently I am considering "-XX:-UseInterpreter -XX:+LogCompilation",
>> but this seems to be an awfully indirect way to achieve my goal -- and
>> it may not even give correct results if the only call of m() is inlined
>> during compilation.
>>
>> Any help is greatly appreciated.
>>
>> Best wishes,
>>
>> Andreas

Andreas Sewe | 16 May 2012 14:15
Picon
Favicon

Re: Obtaining methods executed at least once?

Hi Ben,

> OK, I'll bite with a couple of questions:

Fair enough.

> 1) Why do you want this information? Is it purely informative, or are
> you planning to make use of it in some way? If you're planning to use
> it - what uses do you have in mind?

I am gathering some statistics on a few benchmarks. In particular, I am
interested in the *fraction* of methods compiled (at the various levels,
if tiered compilation is used). Thus, I also need to know the overall
number of methods executed (whether they have been compiled or not).

> 2) What do you mean by "inlined at compilation". When dealing with
> HotSpot, always remember that there are two separate things which
> could be thought of as 'compilation' - there's the process which
> creates classfiles (e.g. javac) and there's the process which creates
> machine code from hot methods - JIT compilation. It isn't clear to me
> which of these two processes you mean.

I meant the JIT.

The problem is that you want to count a method as compiled even if is
only compiled in a "nested" context due to inlining. (Of course, you
don't want to count the method multiple times if it is inlined at
multiple places either.) But as far as I can see, the
-XX:+LogCompilation log contains enough information to determine all
methods that have been compiled, be it as a root method or deeper into
the inline tree.

Hope this clarifies things for you.

Best wishes,

Andreas


Gmane