headers
Russel Winder | 13 Aug 09:49

[groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Am I the only person worried about the Groovy build time.  1502 took 93
minutes, 1503 took 152 minutes.

-------- Forwarded Message --------
> From: build-support@...
> Reply-To: dev@...
> To: build-support@..., dierk.koenig@...,
> mguillemot@..., scm@...
> Subject: [groovy-scm] groovy build.1503 Build Successful
> Date: Wed, 13 Aug 2008 01:39:56 +0200 (CEST)

> BUILD COMPLETE - build.1503
>            Date of build
> 08/12/2008 23:06:37
>            Time to build
> 152 minutes 46 seconds
>            Last changed
> 08/12/2008 23:03:14
> 
> 

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
(Continue reading)

Permalink | Reply |
headers
Jochen Theodorou | 13 Aug 11:32
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> Am I the only person worried about the Groovy build time.  1502 took 93
> minutes, 1503 took 152 minutes.

you have to consider, that the server is doing other things too. The 
time the build server took is no real indication for how long the build 
really takes. I guess you target your javadoc problem

bye blackdrag

--

-- 
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Russel Winder | 13 Aug 17:16

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Wed, 2008-08-13 at 11:32 +0200, Jochen Theodorou wrote:
> Russel Winder schrieb:
> > Am I the only person worried about the Groovy build time.  1502 took 93
> > minutes, 1503 took 152 minutes.
> 
> you have to consider, that the server is doing other things too. The 
> time the build server took is no real indication for how long the build 
> really takes. I guess you target your javadoc problem
> 

Indeed.  I think the problem is entirely in the documentation build.  On
a twin-Xeon 2.33GHz machine with 4GB memory and doing nothing else other
than showing System Monitor:

|> time ant install
 . . . 
BUILD SUCCESSFUL
Total time: 7 minutes 0 seconds

real	7m0.603s
user	5m45.406s
sys	0m24.714s

This is of course slow because the files are provided over a Gigabit
Ethernet NFS mount rather than local files.  Then after a clean to
ensure the same start point:

|> ANT_OPTS=-Xmx512m time ant installRepo
 . . .
BUILD SUCCESSFUL
(Continue reading)

Permalink | Reply |
headers
Martin C. Martin | 13 Aug 17:21

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Can you run a profiler on it?  No one else seems to be able to reproduce 
your problem.  Just -Xprof should tell you something interesting.

Best,
Martin

Russel Winder wrote:
> On Wed, 2008-08-13 at 11:32 +0200, Jochen Theodorou wrote:
>> Russel Winder schrieb:
>>> Am I the only person worried about the Groovy build time.  1502 took 93
>>> minutes, 1503 took 152 minutes.
>> you have to consider, that the server is doing other things too. The 
>> time the build server took is no real indication for how long the build 
>> really takes. I guess you target your javadoc problem
>>
> 
> Indeed.  I think the problem is entirely in the documentation build.  On
> a twin-Xeon 2.33GHz machine with 4GB memory and doing nothing else other
> than showing System Monitor:
> 
> |> time ant install
>  . . . 
> BUILD SUCCESSFUL
> Total time: 7 minutes 0 seconds
> 
> real	7m0.603s
> user	5m45.406s
> sys	0m24.714s
> 
> This is of course slow because the files are provided over a Gigabit
(Continue reading)

Permalink | Reply |
headers
Russel Winder | 13 Aug 18:06

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Wed, 2008-08-13 at 11:21 -0400, Martin C. Martin wrote:
> Can you run a profiler on it?  No one else seems to be able to reproduce 
> your problem.  Just -Xprof should tell you something interesting.

Run with -Xprof now happening, results in 1 hour 40 mins ;-)

Actually I am not the only person who can reproduce this, Canoo's Cruise
Control engine can as well, even if no one else admits it happens to
them !

Is it really the case that no-one else has this problem building the
documents?

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Russel Winder | 14 Aug 09:45

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Wed, 2008-08-13 at 17:06 +0100, Russel Winder wrote:

> Run with -Xprof now happening, results in 1 hour 40 mins ;-)

The delay in reporting back is because I cannot seem to get it to run in
a way that avoids the Out of Memory exception.  No amount of -Xmx... in
the exported ANT_OPTS stops it when being timed.

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Jochen Theodorou | 13 Aug 20:26
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> On Wed, 2008-08-13 at 11:32 +0200, Jochen Theodorou wrote:
>> Russel Winder schrieb:
>>> Am I the only person worried about the Groovy build time.  1502 took 93
>>> minutes, 1503 took 152 minutes.
>> you have to consider, that the server is doing other things too. The 
>> time the build server took is no real indication for how long the build 
>> really takes. I guess you target your javadoc problem
>>
> 
> Indeed.  I think the problem is entirely in the documentation build.  On
> a twin-Xeon 2.33GHz machine with 4GB memory and doing nothing else other
> than showing System Monitor:
> 
> |> time ant install
>  . . . 
> BUILD SUCCESSFUL
> Total time: 7 minutes 0 seconds
> 
> real	7m0.603s
> user	5m45.406s
> sys	0m24.714s
> 
> This is of course slow because the files are provided over a Gigabit
> Ethernet NFS mount rather than local files.

is there a way for you to test this with a local hard disc? Gigabit 
ethernet is sure fast enough to transfer the data, but if for some 
reason the time per access is long, then it would be still slow.
(Continue reading)

Permalink | Reply |
headers
Russel Winder | 14 Aug 11:12

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Jochen,

On Wed, 2008-08-13 at 20:26 +0200, Jochen Theodorou wrote:

> > |> time ant install
> >  . . . 
> > BUILD SUCCESSFUL
> > Total time: 7 minutes 0 seconds
> > 
> > real	7m0.603s
> > user	5m45.406s
> > sys	0m24.714s
> > 
> > This is of course slow because the files are provided over a Gigabit
> > Ethernet NFS mount rather than local files.
> 
> is there a way for you to test this with a local hard disc? Gigabit 
> ethernet is sure fast enough to transfer the data, but if for some 
> reason the time per access is long, then it would be still slow.

Using the local disc:

|> time ant install
 . . . 
BUILD SUCCESSFUL
Total time: 3 minutes 38 seconds

real	3m39.028s
user	5m41.089s
sys	0m12.893s
(Continue reading)

Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 12:34
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> Jochen,
> 
> On Wed, 2008-08-13 at 20:26 +0200, Jochen Theodorou wrote:
> 
>>> |> time ant install
>>>  . . . 
>>> BUILD SUCCESSFUL
>>> Total time: 7 minutes 0 seconds
>>>
>>> real	7m0.603s
>>> user	5m45.406s
>>> sys	0m24.714s
>>>
>>> This is of course slow because the files are provided over a Gigabit
>>> Ethernet NFS mount rather than local files.
>> is there a way for you to test this with a local hard disc? Gigabit 
>> ethernet is sure fast enough to transfer the data, but if for some 
>> reason the time per access is long, then it would be still slow.
> 
> Using the local disc:
> 
> |> time ant install
>  . . . 
> BUILD SUCCESSFUL
> Total time: 3 minutes 38 seconds
> 
> real	3m39.028s
> user	5m41.089s
> sys	0m12.893s
(Continue reading)

Permalink | Reply |
headers
Russel Winder | 14 Aug 14:07

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Thu, 2008-08-14 at 12:34 +0200, Jochen Theodorou wrote:

> I am puzzled... NFS takes 7min, local disc takes 3min.. what is it that 
> takes 108min? It is the same machine, or not? on my machine I am getting 
> about the same times (around 4min)... but isn't that ok?

The problem was the javadoc target.

All the 7min 3min was showing was that compiling and jaring up the
groovy code is reasonably fast and faster on local disc than NFS. 

> I think "ant installRepo" is doing much more than "ant install". If it 
> is what I rmember, then it creates the javadoc again, two times for 
> example. ah, wait, that was "ant deploy"

ant installRepo does a little bit more than install but the crucial time
factor was the javadoc task.

> anyway... "time ant javadoc":
> 
> BUILD SUCCESSFUL
> Total time: 38 seconds
> 
> real	0m38.576s
> user	0m56.856s
> sys	0m1.564s

Sorry but this I do not believe at least not if you are using JDK
1.6.0_06.  This is exactly the things that was taking 100mins for me on
my twin-Xeon box.  I cannot see the same thing taking 38s on you box
(Continue reading)

Permalink | Reply |
headers
Russel Winder | 14 Aug 14:21

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Thu, 2008-08-14 at 13:07 +0100, Russel Winder wrote:

> > anyway... "time ant javadoc":
> > 
> > BUILD SUCCESSFUL
> > Total time: 38 seconds
> > 
> > real	0m38.576s
> > user	0m56.856s
> > sys	0m1.564s
> 
> Sorry but this I do not believe at least not if you are using JDK
> 1.6.0_06.  This is exactly the things that was taking 100mins for me on
> my twin-Xeon box.  I cannot see the same thing taking 38s on you box
> even if it is a Cray XMP -- mostly because javadoc is singularly
> single-threaded.

OK, I was wrong.  I change my mind it could easily have taken 38s..  My
new version takes just over 1m :-)))))))

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Russel Winder | 14 Aug 14:58

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]


We have success, total build time for a complete from scratch
installRepo is down from 108mins to 10mins.

Ant no ANT_OPTS=-Xmx512m either :-)

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 16:54
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> We have success, total build time for a complete from scratch
> installRepo is down from 108mins to 10mins.
> 
> Ant no ANT_OPTS=-Xmx512m either :-)

hehe, ok, so the problem is solved now ;)

bye blackdrag

--

-- 
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Martin C. Martin | 14 Aug 17:06

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Is there anything that can be checked into subversion to fix it? 
Failing that, a better error message?  It would be great if the next 
person didn't have to wrestle with this for weeks on and off too.  :)

Best,
Martin

Jochen Theodorou wrote:
> Russel Winder schrieb:
>> We have success, total build time for a complete from scratch
>> installRepo is down from 108mins to 10mins.
>>
>> Ant no ANT_OPTS=-Xmx512m either :-)
> 
> hehe, ok, so the problem is solved now ;)
> 
> bye blackdrag
> 

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 18:28
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Martin C. Martin schrieb:
> Is there anything that can be checked into subversion to fix it? Failing 
> that, a better error message?  It would be great if the next person 
> didn't have to wrestle with this for weeks on and off too.  :)

Russel checked the changes already in. the properties file can now be 
used to increase the memory. A better error message is not possible, 
since it does not fail... it just takes long.

bye blackdrag

--

-- 
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Russel Winder | 14 Aug 18:16

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Thu, 2008-08-14 at 16:54 +0200, Jochen Theodorou wrote:
> Russel Winder schrieb:
> > We have success, total build time for a complete from scratch
> > installRepo is down from 108mins to 10mins.
> > 
> > Ant no ANT_OPTS=-Xmx512m either :-)
> 
> hehe, ok, so the problem is solved now ;)

I think so.  Canoo's build took 48min which is way better than 150.

Do you want me to make the same change to the 1.5.x branch?
--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 18:26
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> On Thu, 2008-08-14 at 16:54 +0200, Jochen Theodorou wrote:
>> Russel Winder schrieb:
>>> We have success, total build time for a complete from scratch
>>> installRepo is down from 108mins to 10mins.
>>>
>>> Ant no ANT_OPTS=-Xmx512m either :-)
>> hehe, ok, so the problem is solved now ;)
> 
> I think so.  Canoo's build took 48min which is way better than 150.
> 
> Do you want me to make the same change to the 1.5.x branch?

if the problem is there too, then yes.

bye blackdrag

--

-- 
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Russel Winder | 14 Aug 18:32

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Thu, 2008-08-14 at 18:26 +0200, Jochen Theodorou wrote:

> if the problem is there too, then yes.

No idea I'm afraid, I have never built that branch -- I build and use
HEAD or get the released 1.5.6 from the Maven repository.

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 20:28
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> On Thu, 2008-08-14 at 18:26 +0200, Jochen Theodorou wrote:
> 
>> if the problem is there too, then yes.
> 
> No idea I'm afraid, I have never built that branch -- I build and use
> HEAD or get the released 1.5.6 from the Maven repository.

I would say just do it... you could of course, if you have time, make a 
checkout and see if the problem happens there too... but since it seems 
to be a general problem and since it might occur there any time I think 
just doing the change should be fine.

bye blackdrag

--

-- 
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Permalink | Reply |
headers
Jochen Theodorou | 14 Aug 15:51
Gravatar

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

Russel Winder schrieb:
> On Thu, 2008-08-14 at 12:34 +0200, Jochen Theodorou wrote:
>> anyway... "time ant javadoc":
>>
>> BUILD SUCCESSFUL
>> Total time: 38 seconds
>>
>> real	0m38.576s
>> user	0m56.856s
>> sys	0m1.564s
> 
> Sorry but this I do not believe at least not if you are using JDK
> 1.6.0_06.  This is exactly the things that was taking 100mins for me on
> my twin-Xeon box.  I cannot see the same thing taking 38s on you box
> even if it is a Cray XMP -- mostly because javadoc is singularly
> single-threaded.

I have

java version "1.6.0"
OpenJDK Runtime Environment (build 1.6.0-b09)
OpenJDK 64-Bit Server VM (build 1.6.0-b09, mixed mode)

and it really takes only 38s, the groovydoc task is much slower for me 
than your the javadoc. The ant javadoc task includes building the docs 
and jaring them up. No compilation, nothing. And its not because my 
system is faster than yours, but I guess javadoc gets in my case enough 
memory to not to need to do a garbage collection run.

> I have amended the build file to spawn the javadoc task just as the
(Continue reading)

Permalink | Reply |
headers
Russel Winder | 14 Aug 11:19

Re: [groovy-dev] [Fwd: [groovy-scm] groovy build.1503 Build Successful]

On Wed, 2008-08-13 at 20:26 +0200, Jochen Theodorou wrote:

> hmmm... do I read that 97% as that the tool used 97% CPU power, which 
> means one CPU/core was almost at 100%? On a 2.33GHz machine for this 
> long!? Also... is that huge numer of pagefaults there normal?

I should have added that the javadoc process is distinctly sequential
and single threaded -- yes it maxes out 1 of my eight processors leaving
the rest idle.

--

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
Permalink | Reply |

Gmane