Developing PHP

Ferenc Kovacs | 3 Nov 2011 01:46

Re: [PHP-DEV] CI for 5.4

On Wed, Sep 7, 2011 at 11:29 AM, Hannes Magnusson <
hannes.magnusson <at> gmail.com> wrote:

> On Wed, Sep 7, 2011 at 03:53, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
> > On Wed, Sep 7, 2011 at 3:29 AM, Stas Malyshev <smalyshev <at> sugarcrm.com>
> wrote:
> >> Hi!
> >>
> >> Since we started to pay real attention to our unit tests now, I wonder
> if we
> >> could set up some kind of frequently-running CI system that could be
> used to
> >> screen commits and identify breakage early? That'd help with 5.4
> process I
> >> think.
> >> We have http://gcov.php.net/ but it doesn't run with the frequency I'd
> like
> >> and since it says the run takes 44 hours it's kind of understandable.
> So I
> >> wonder if we could have something that just builds it and runs unit
> tests
> >> and we could see it in the same format as on gcov? Ideally after each
> commit
> >> would be nice, but say once an hour or two (even fullest unit tests run
> >> should take more than that, I think) would be OK too. If we could have
> two
> >> of them, like Linux & Windows, it'd be even better, but at least one
> would
> >> be nice.
> >> What do you think?

(Continue reading)

headers

Ferenc Kovacs | 3 Nov 2011 18:04

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 1:46 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Wed, Sep 7, 2011 at 11:29 AM, Hannes Magnusson <
> hannes.magnusson <at> gmail.com> wrote:
>
>> On Wed, Sep 7, 2011 at 03:53, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>> > On Wed, Sep 7, 2011 at 3:29 AM, Stas Malyshev <smalyshev <at> sugarcrm.com>
>> wrote:
>> >> Hi!
>> >>
>> >> Since we started to pay real attention to our unit tests now, I wonder
>> if we
>> >> could set up some kind of frequently-running CI system that could be
>> used to
>> >> screen commits and identify breakage early? That'd help with 5.4
>> process I
>> >> think.
>> >> We have http://gcov.php.net/ but it doesn't run with the frequency
>> I'd like
>> >> and since it says the run takes 44 hours it's kind of understandable.
>> So I
>> >> wonder if we could have something that just builds it and runs unit
>> tests
>> >> and we could see it in the same format as on gcov? Ideally after each
>> commit
>> >> would be nice, but say once an hour or two (even fullest unit tests run
>> >> should take more than that, I think) would be OK too. If we could have
>> two

(Continue reading)

headers

Klaus Silveira | 3 Nov 2011 18:10

Re: [PHP-DEV] CI for 5.4

What are the current obstacles in a possible Jenkins implementation for the
PHP codebase?

--

-- 
Klaus Silveira
(011) 8564-2492
www.klaussilveira.com

headers

Ferenc Kovacs | 3 Nov 2011 19:01

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 6:10 PM, Klaus Silveira <contato <at> klaussilveira.com>wrote:

> What are the current obstacles in a possible Jenkins implementation for
> the PHP codebase?
>
>
> --
> Klaus Silveira
> (011) 8564-2492
> www.klaussilveira.com
>

Sorry, I don't really follow you.
As I mentioned in my previous email and the RFC, the php-src(all of the
supported branches: 5.3, 5.4 and trunk) is already integrated into Jenkins.
I also listed the current problems/limitations in
https://wiki.php.net/rfc/jenkins#current_problems and there were other
obstacles (extending the run-tests.php to generate a test result format
supported by Jenkins), but most of them were solved (except those in the
current_problems section).
Of course there are ways to improve the current setup, I listed those ideas
at https://wiki.php.net/rfc/jenkins#future_plans

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

headers

Stefan Marr | 3 Nov 2011 19:43

Re: [PHP-DEV] CI for 5.4

Hi Ferenc:

On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:

> Of course there are ways to improve the current setup, I listed those ideas
> at https://wiki.php.net/rfc/jenkins#future_plans

Very nice.
I don't know Jenkins, but would it be possible to mail the author of a change in case it breaks something? I
just had such a case, where my local setup was insufficient to catch it. 

Thanks a lot
Stefan

-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php

headers

Klaus Silveira | 3 Nov 2011 19:47

Re: [PHP-DEV] CI for 5.4

Yes, it is. But not only email notifications are possible, other
interesting integrations are possible. For example, Jabber.

https://wiki.jenkins-ci.org/display/JENKINS/Instant+Messaging+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Jabber+Plugin

On Thu, Nov 3, 2011 at 4:43 PM, Stefan Marr <php <at> stefan-marr.de> wrote:

> Hi Ferenc:
>
> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>
> > Of course there are ways to improve the current setup, I listed those
> ideas
> > at https://wiki.php.net/rfc/jenkins#future_plans
>
> Very nice.
> I don't know Jenkins, but would it be possible to mail the author of a
> change in case it breaks something? I just had such a case, where my local
> setup was insufficient to catch it.
>
> Thanks a lot
> Stefan
>
>
> --
> Stefan Marr
> Software Languages Lab
> Vrije Universiteit Brussel
> Pleinlaan 2 / B-1050 Brussels / Belgium

(Continue reading)

headers

Ferenc Kovacs | 3 Nov 2011 20:15

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <php <at> stefan-marr.de> wrote:

> Hi Ferenc:
>
> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>
> > Of course there are ways to improve the current setup, I listed those
> ideas
> > at https://wiki.php.net/rfc/jenkins#future_plans
>
> Very nice.
> I don't know Jenkins, but would it be possible to mail the author of a
> change in case it breaks something? I just had such a case, where my local
> setup was insufficient to catch it.
>
> Thanks a lot
> Stefan
>
>
It is, and as I mentioned in the RFC the email notification can be enabled
when needed, I didn't wanted to do that before introducing the RFC.
There is a slight problem however: if we don't build each and every
revision (which can happen, if two commit happens before the build starts,
which can happen more easily if another build is in progress for that job,
because then jenkins will just queue the build, not execute it right away
for obvious reasons), so there is a slight chance for "blaming" the wrong
person.
The usual setup is that you have a mailing list/alias which always gets a
mail about the build results (which can also be customized in detail, when
to mail, etc.) and optionally you can notify the person whose commit break

(Continue reading)

headers

Stefan Marr | 3 Nov 2011 20:33

Re: [PHP-DEV] CI for 5.4

Hi:

On 03 Nov 2011, at 20:15, Ferenc Kovacs wrote:

> The usual setup is that you have a mailing list/alias which always gets a mail about the build results
(which can also be customized in detail, when to mail, etc.) and optionally you can notify the person whose
commit break the build.

If I get a personal notification via mail, if I break something, that kind of guarantees my attention. Using
IRC is to distracting, so that is not an option for me. Sorry.

If I am the only one who would use it, then don't bother. But if I can simply opt-in for it somewhere, that would
be great.

Thanks to people like Antony, obviously stupid bugs get caught pretty fast.

Thanks
Stefan

-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>

(Continue reading)

headers

Klaus Silveira | 3 Nov 2011 20:41

Re: [PHP-DEV] CI for 5.4

That's kind of a general setup, Stefan. Sending an email to the commiter
that broke the build with details of the build process, as well sending an
email to a mailing list.

I'll be looking into this Jenkins issue (JENKINS-5931). Maybe having an
access control based on IRC operators and voices?

On Thu, Nov 3, 2011 at 5:33 PM, Stefan Marr <php <at> stefan-marr.de> wrote:

> Hi:
>
> On 03 Nov 2011, at 20:15, Ferenc Kovacs wrote:
>
> > The usual setup is that you have a mailing list/alias which always gets
> a mail about the build results (which can also be customized in detail,
> when to mail, etc.) and optionally you can notify the person whose commit
> break the build.
>
> If I get a personal notification via mail, if I break something, that kind
> of guarantees my attention. Using IRC is to distracting, so that is not an
> option for me. Sorry.
>
> If I am the only one who would use it, then don't bother. But if I can
> simply opt-in for it somewhere, that would be great.
>
> Thanks to people like Antony, obviously stupid bugs get caught pretty fast.
>
> Thanks
> Stefan
>

(Continue reading)

headers

Ferenc Kovacs | 3 Nov 2011 20:49

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 8:41 PM, Klaus Silveira <contato <at> klaussilveira.com>wrote:

> That's kind of a general setup, Stefan. Sending an email to the commiter
> that broke the build with details of the build process, as well sending an
> email to a mailing list.
>
> I'll be looking into this Jenkins issue (JENKINS-5931). Maybe having an
> access control based on IRC operators and voices?
>

yeah, using voices would work, but then that would mean that it is only a
read only channel for most people, which imo would be a bad move.
I mean it would be really nice, if people could drop by to our qa related
irc room and start talking about that pesky build failure.
another solution would be to set up a custom irc bot, independently from
jenkins, which could fetch the build results through the mailing list or
rss and send it to the channel.
of course extending the jenkins irc bot is also a possibility, the easiest
way would be having a configuration option for disabling the whole control
stuff, as we don't really need the interactivity there, you can use the web
interface for that.

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

headers

Felipe Pena | 3 Nov 2011 22:22

Re: Re: [PHP-DEV] CI for 5.4

2011/11/3 Ferenc Kovacs <tyra3l <at> gmail.com>:
> On Thu, Nov 3, 2011 at 8:41 PM, Klaus Silveira <contato <at> klaussilveira.com>wrote:
>
>> That's kind of a general setup, Stefan. Sending an email to the commiter
>> that broke the build with details of the build process, as well sending an
>> email to a mailing list.
>>
>> I'll be looking into this Jenkins issue (JENKINS-5931). Maybe having an
>> access control based on IRC operators and voices?
>>
>
> yeah, using voices would work, but then that would mean that it is only a
> read only channel for most people, which imo would be a bad move.
> I mean it would be really nice, if people could drop by to our qa related
> irc room and start talking about that pesky build failure.
> another solution would be to set up a custom irc bot, independently from
> jenkins, which could fetch the build results through the mailing list or
> rss and send it to the channel.
> of course extending the jenkins irc bot is also a possibility, the easiest
> way would be having a configuration option for disabling the whole control
> stuff, as we don't really need the interactivity there, you can use the web
> interface for that.
>
> --
> Ferenc Kovács
>  <at> Tyr43l - http://tyrael.hu
>

Nice work with this Jenkins stuff! The system seems pretty useful.
I like the idea to have a mailing list for build result notification,

(Continue reading)

headers

Klaus Silveira | 3 Nov 2011 22:25

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

Yes, it's a wonderful setup. Great work, Ferenc! :D

This does give a nice motivation to write more tests and increase the code
coverage, doesn't it?

On Thu, Nov 3, 2011 at 7:22 PM, Felipe Pena <felipensp <at> gmail.com> wrote:

> 2011/11/3 Ferenc Kovacs <tyra3l <at> gmail.com>:
> > On Thu, Nov 3, 2011 at 8:41 PM, Klaus Silveira <
> contato <at> klaussilveira.com>wrote:
> >
> >> That's kind of a general setup, Stefan. Sending an email to the commiter
> >> that broke the build with details of the build process, as well sending
> an
> >> email to a mailing list.
> >>
> >> I'll be looking into this Jenkins issue (JENKINS-5931). Maybe having an
> >> access control based on IRC operators and voices?
> >>
> >
> > yeah, using voices would work, but then that would mean that it is only a
> > read only channel for most people, which imo would be a bad move.
> > I mean it would be really nice, if people could drop by to our qa related
> > irc room and start talking about that pesky build failure.
> > another solution would be to set up a custom irc bot, independently from
> > jenkins, which could fetch the build results through the mailing list or
> > rss and send it to the channel.
> > of course extending the jenkins irc bot is also a possibility, the
> easiest
> > way would be having a configuration option for disabling the whole

(Continue reading)

headers

Ferenc Kovacs | 3 Nov 2011 22:29

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 10:25 PM, Klaus Silveira
<contato <at> klaussilveira.com>wrote:

> Yes, it's a wonderful setup. Great work, Ferenc! :D
>
> This does give a nice motivation to write more tests and increase the code
> coverage, doesn't it?
>

Yes, and first of all, to fix the currently failing tests.
We also had a discussion with Hannes, that the build machines could also be
used for hunting down bugs, for which the developer doesn't have an
environment to test and reproduce the problem.

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

headers

Ferenc Kovacs | 3 Nov 2011 22:26

[PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 10:22 PM, Felipe Pena <felipensp <at> gmail.com> wrote:

> 2011/11/3 Ferenc Kovacs <tyra3l <at> gmail.com>:
> > On Thu, Nov 3, 2011 at 8:41 PM, Klaus Silveira <
> contato <at> klaussilveira.com>wrote:
> >
> >> That's kind of a general setup, Stefan. Sending an email to the commiter
> >> that broke the build with details of the build process, as well sending
> an
> >> email to a mailing list.
> >>
> >> I'll be looking into this Jenkins issue (JENKINS-5931). Maybe having an
> >> access control based on IRC operators and voices?
> >>
> >
> > yeah, using voices would work, but then that would mean that it is only a
> > read only channel for most people, which imo would be a bad move.
> > I mean it would be really nice, if people could drop by to our qa related
> > irc room and start talking about that pesky build failure.
> > another solution would be to set up a custom irc bot, independently from
> > jenkins, which could fetch the build results through the mailing list or
> > rss and send it to the channel.
> > of course extending the jenkins irc bot is also a possibility, the
> easiest
> > way would be having a configuration option for disabling the whole
> control
> > stuff, as we don't really need the interactivity there, you can use the
> web
> > interface for that.
> >

(Continue reading)

headers

Rasmus Lerdorf | 3 Nov 2011 23:33

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On 11/03/2011 02:26 PM, Ferenc Kovacs wrote:
> We could set up the email notification any time, we would just have to
> agree where to send (internals, php-qa, creating a dedicated mailing list?)
> and when to send (as I added to the RFC, having failing tests can be a
> problem from the notification POV).
> We have to find a good middle way, else we will either miss the
> notifications, or we will be overwhelmed, so people will just ignore it.

Ideally they should go to internals if you can get them to be non-noisy.

As long as it only reports a test status change from the previous run,
hopefully that wouldn't create much noise.

-Rasmus

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php

headers

Ferenc Kovacs | 3 Nov 2011 23:47

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 11:33 PM, Rasmus Lerdorf <rasmus <at> lerdorf.com> wrote:

> On 11/03/2011 02:26 PM, Ferenc Kovacs wrote:
> > We could set up the email notification any time, we would just have to
> > agree where to send (internals, php-qa, creating a dedicated mailing
> list?)
> > and when to send (as I added to the RFC, having failing tests can be a
> > problem from the notification POV).
> > We have to find a good middle way, else we will either miss the
> > notifications, or we will be overwhelmed, so people will just ignore it.
>
> Ideally they should go to internals if you can get them to be non-noisy.
>

true

>
> As long as it only reports a test status change from the previous run,
> hopefully that wouldn't create much noise.

yeah, and that is a little bit of a problem, because as I mentioned by
default the jenkins way is that your tests passes:
http://jenkins.361315.n4.nabble.com/Test-failure-baseline-td386476.html
so we have three option as far as I can see:
A, it will report every test failure, but we fix all of our tests so we are
cool. (my preference)
B, same as A, but until we fix all of the tests, we mark the failing test
as XFAIL, which won't be counted as failed test (I'm against it, as time
taught us that we are lazy, and XFAILs will be kept that way. :/)
C, Somebody(me) do some scripting, and our currently failing tests won't

(Continue reading)

headers

Rasmus Lerdorf | 3 Nov 2011 23:58

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On 11/03/2011 03:47 PM, Ferenc Kovacs wrote:
> A, it will report every test failure, but we fix all of our tests so we
> are cool. (my preference)

But even if we do that, when a test does fail, it may take a couple of
days to fix it. It would be nice if we didn't get an internals email for
every commit that triggers a build during that time. So even in this
ideal case I think the notification mechanism needs to be smarter.

-Rasmus

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php

headers

Ferenc Kovacs | 4 Nov 2011 00:12

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 11:58 PM, Rasmus Lerdorf <rasmus <at> lerdorf.com> wrote:

> On 11/03/2011 03:47 PM, Ferenc Kovacs wrote:
> > A, it will report every test failure, but we fix all of our tests so we
> > are cool. (my preference)
>
> But even if we do that, when a test does fail, it may take a couple of
> days to fix it. It would be nice if we didn't get an internals email for
> every commit that triggers a build during that time. So even in this
> ideal case I think the notification mechanism needs to be smarter.
>
> -Rasmus
>

using the email-ext plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin one can bind
email notification on the following events:
- Failure
- Unstable
- Still-Failing
- Success
- Fixed
- Still-Unstable
- (Before-Build)

So we could configure to only send emails on Failure and Fixed(I ignored
the Unstable on purpose for the sake of simplicity, of course we also use
this for option C).
That way we would only see the notification on the first error, and the
next mail would be when we finally fixed it.

(Continue reading)

headers

Stefan Marr | 4 Nov 2011 00:20

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

Hi:

On 04 Nov 2011, at 00:12, Ferenc Kovacs wrote:

> I almost forget to mention, but the email notification also supports defining different recipients for
each event, so for example the commiters could still get the notification for each of their
commits/builds until the build is back to normal, that way the list wouldn't be spammed, but some active
contributors would be still continuously bugged.

Does that meant that the committer could get an unconditional email with the result of his/her commit?

Sounds very reasonable to me.

Best regards
Stefan

-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php

(Continue reading)

headers

Ferenc Kovacs | 4 Nov 2011 00:21

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Fri, Nov 4, 2011 at 12:20 AM, Stefan Marr <php <at> stefan-marr.de> wrote:

> Hi:
>
> On 04 Nov 2011, at 00:12, Ferenc Kovacs wrote:
>
> > I almost forget to mention, but the email notification also supports
> defining different recipients for each event, so for example the commiters
> could still get the notification for each of their commits/builds until the
> build is back to normal, that way the list wouldn't be spammed, but some
> active contributors would be still continuously bugged.
>
> Does that meant that the committer could get an unconditional email with
> the result of his/her commit?
>
> Sounds very reasonable to me.
>
>
yep.

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

headers

Ferenc Kovacs | 4 Nov 2011 02:00

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Fri, Nov 4, 2011 at 12:21 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Fri, Nov 4, 2011 at 12:20 AM, Stefan Marr <php <at> stefan-marr.de> wrote:
>
>> Hi:
>>
>> On 04 Nov 2011, at 00:12, Ferenc Kovacs wrote:
>>
>> > I almost forget to mention, but the email notification also supports
>> defining different recipients for each event, so for example the commiters
>> could still get the notification for each of their commits/builds until the
>> build is back to normal, that way the list wouldn't be spammed, but some
>> active contributors would be still continuously bugged.
>>
>> Does that meant that the committer could get an unconditional email with
>> the result of his/her commit?
>>
>> Sounds very reasonable to me.
>>
>>
> yep.
>
>
> --
> Ferenc Kovács
>  <at> Tyr43l - http://tyrael.hu
>

(Continue reading)

headers

Ferenc Kovacs | 5 Nov 2011 10:42

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Fri, Nov 4, 2011 at 2:00 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Fri, Nov 4, 2011 at 12:21 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>
>>
>>
>> On Fri, Nov 4, 2011 at 12:20 AM, Stefan Marr <php <at> stefan-marr.de> wrote:
>>
>>> Hi:
>>>
>>> On 04 Nov 2011, at 00:12, Ferenc Kovacs wrote:
>>>
>>> > I almost forget to mention, but the email notification also supports
>>> defining different recipients for each event, so for example the commiters
>>> could still get the notification for each of their commits/builds until the
>>> build is back to normal, that way the list wouldn't be spammed, but some
>>> active contributors would be still continuously bugged.
>>>
>>> Does that meant that the committer could get an unconditional email with
>>> the result of his/her commit?
>>>
>>> Sounds very reasonable to me.
>>>
>>>
>> yep.
>>
>>
>> --

(Continue reading)

headers

Ferenc Kovacs | 5 Nov 2011 13:16

Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Sat, Nov 5, 2011 at 10:42 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Fri, Nov 4, 2011 at 2:00 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>
>>
>>
>> On Fri, Nov 4, 2011 at 12:21 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>>
>>>
>>>
>>> On Fri, Nov 4, 2011 at 12:20 AM, Stefan Marr <php <at> stefan-marr.de> wrote:
>>>
>>>> Hi:
>>>>
>>>> On 04 Nov 2011, at 00:12, Ferenc Kovacs wrote:
>>>>
>>>> > I almost forget to mention, but the email notification also supports
>>>> defining different recipients for each event, so for example the commiters
>>>> could still get the notification for each of their commits/builds until the
>>>> build is back to normal, that way the list wouldn't be spammed, but some
>>>> active contributors would be still continuously bugged.
>>>>
>>>> Does that meant that the committer could get an unconditional email
>>>> with the result of his/her commit?
>>>>
>>>> Sounds very reasonable to me.
>>>>
>>>>

(Continue reading)

headers

Hannes Magnusson | 5 Nov 2011 16:23

[PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Sat, Nov 5, 2011 at 10:42, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

> I just noticed that the mail will be sent on each configuration build
> (debian 32bit, debian 64bit, freebsd 32bit, freebsd 64bit), which is of
> course a little bit too much, I'm looking into resolving this, so to only
> send email on the parent build result. Until that is fixed, I won't enable
> the email notifications for the other jobs (5.4, trunk)

If a build fails only on one specific platform, that will need to be
mailed out too.

-Hannes

--

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

headers

Ferenc Kovacs | 5 Nov 2011 16:35

Re: Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4

On Sat, Nov 5, 2011 at 4:23 PM, Hannes Magnusson <bjori <at> php.net> wrote:

> On Sat, Nov 5, 2011 at 10:42, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>
> > I just noticed that the mail will be sent on each configuration build
> > (debian 32bit, debian 64bit, freebsd 32bit, freebsd 64bit), which is of
> > course a little bit too much, I'm looking into resolving this, so to only
> > send email on the parent build result. Until that is fixed, I won't
> enable
> > the email notifications for the other jobs (5.4, trunk)
>
> If a build fails only on one specific platform, that will need to be
> mailed out too.
>
>
If any configuration job fails(for example
http://ci.qa.php.net/job/php-src-5.3-matrix-build/architecture=x86,os=linux-debian-6.0/),
it will make the whole job (
http://ci.qa.php.net/job/php-src-5.3-matrix-build/), so thats not a problem.
The parent build status reflects the configuration jobs, it will be only
successful if all of the configuration builds were successful too.

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

headers

Ferenc Kovacs | 3 Nov 2011 20:43

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 8:33 PM, Stefan Marr <php <at> stefan-marr.de> wrote:

> Hi:
>
> On 03 Nov 2011, at 20:15, Ferenc Kovacs wrote:
>
> > The usual setup is that you have a mailing list/alias which always gets
> a mail about the build results (which can also be customized in detail,
> when to mail, etc.) and optionally you can notify the person whose commit
> break the build.
>
> If I get a personal notification via mail, if I break something, that kind
> of guarantees my attention. Using IRC is to distracting, so that is not an
> option for me. Sorry.
>
>
agree, this is why this option is present.

> If I am the only one who would use it, then don't bother. But if I can
> simply opt-in for it somewhere, that would be great.
>

I think it would be useful, I just wanted to mention the only downside that
I can think of.
Of course if everybody else would be against the idea, you could still
subscribe to the mailing list where the general report goes, and filter the
messages to only receive those mail to your inbox which reports a build
which was triggered by you (if I remember correctly, by default the email
would contain who started the build, either through SCM or manually).

(Continue reading)

headers

Ferenc Kovacs | 7 Nov 2011 11:14

Re: [PHP-DEV] CI for 5.4

On Thu, Nov 3, 2011 at 8:15 PM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <php <at> stefan-marr.de> wrote:
>
>> Hi Ferenc:
>>
>> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>>
>> > Of course there are ways to improve the current setup, I listed those
>> ideas
>> > at https://wiki.php.net/rfc/jenkins#future_plans
>>
>> Very nice.
>> I don't know Jenkins, but would it be possible to mail the author of a
>> change in case it breaks something? I just had such a case, where my local
>> setup was insufficient to catch it.
>>
>> Thanks a lot
>> Stefan
>>
>>
> It is, and as I mentioned in the RFC the email notification can be enabled
> when needed, I didn't wanted to do that before introducing the RFC.
> There is a slight problem however: if we don't build each and every
> revision (which can happen, if two commit happens before the build starts,
> which can happen more easily if another build is in progress for that job,
> because then jenkins will just queue the build, not execute it right away
> for obvious reasons), so there is a slight chance for "blaming" the wrong

(Continue reading)

headers

Ferenc Kovacs | 7 Nov 2011 16:42

Re: [PHP-DEV] CI for 5.4

On Mon, Nov 7, 2011 at 11:14 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Thu, Nov 3, 2011 at 8:15 PM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>
>>
>>
>> On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <php <at> stefan-marr.de> wrote:
>>
>>> Hi Ferenc:
>>>
>>> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>>>
>>> > Of course there are ways to improve the current setup, I listed those
>>> ideas
>>> > at https://wiki.php.net/rfc/jenkins#future_plans
>>>
>>> Very nice.
>>> I don't know Jenkins, but would it be possible to mail the author of a
>>> change in case it breaks something? I just had such a case, where my local
>>> setup was insufficient to catch it.
>>>
>>> Thanks a lot
>>> Stefan
>>>
>>>
>> It is, and as I mentioned in the RFC the email notification can be
>> enabled when needed, I didn't wanted to do that before introducing the RFC.
>> There is a slight problem however: if we don't build each and every

(Continue reading)

headers

Ferenc Kovacs | 8 Nov 2011 11:15

Re: [PHP-DEV] CI for 5.4

On Mon, Nov 7, 2011 at 4:42 PM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:

>
>
> On Mon, Nov 7, 2011 at 11:14 AM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>
>>
>>
>> On Thu, Nov 3, 2011 at 8:15 PM, Ferenc Kovacs <tyra3l <at> gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <php <at> stefan-marr.de> wrote:
>>>
>>>> Hi Ferenc:
>>>>
>>>> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>>>>
>>>> > Of course there are ways to improve the current setup, I listed those
>>>> ideas
>>>> > at https://wiki.php.net/rfc/jenkins#future_plans
>>>>
>>>> Very nice.
>>>> I don't know Jenkins, but would it be possible to mail the author of a
>>>> change in case it breaks something? I just had such a case, where my local
>>>> setup was insufficient to catch it.
>>>>
>>>> Thanks a lot
>>>> Stefan
>>>>

(Continue reading)

headers

Dirk Haun | 8 Nov 2011 11:36

Re: [PHP-DEV] CI for 5.4

Quoting Ferenc Kovacs <tyra3l <at> gmail.com>:

> There is only three failing tests left on the debian slaves:
>
> For 5.3
> ext/phar/tests/phar_oo_005.phpt.Phar and RecursiveDirectoryIterator 0.01 20
> ext/spl/tests/bug60082.phpt.Bug #60082 (100% CPU / when using references
> with ArrayObject(&$ref))

FWIW, the test for Bug #60082 also seems to let the PHP CLI running,  
even after "make test" has finished. I've reported this here:  
https://bugs.php.net/bug.php?id=60225

bye, Dirk

--

-- 
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php

headers

Ferenc Kovacs | 8 Nov 2011 11:53

Re: [PHP-DEV] CI for 5.4

On Tue, Nov 8, 2011 at 11:36 AM, Dirk Haun <dirk <at> haun-online.de> wrote:

> Quoting Ferenc Kovacs <tyra3l <at> gmail.com>:
>
>  There is only three failing tests left on the debian slaves:
>>
>> For 5.3
>> ext/phar/tests/phar_oo_005.**phpt.Phar and RecursiveDirectoryIterator
>> 0.01 20
>> ext/spl/tests/bug60082.phpt.**Bug #60082 (100% CPU / when using
>> references
>> with ArrayObject(&$ref))
>>
>
> FWIW, the test for Bug #60082 also seems to let the PHP CLI running, even
> after "make test" has finished. I've reported this here:
> https://bugs.php.net/bug.php?**id=60225<https://bugs.php.net/bug.php?id=60225>
>
>
The run-tests.php should kill it when it timeouts(it does a proc_terminate).
On FreeBSD I experienced that the timeout never happen, but in you case it
seems it does, but it seems that the proc_terminate didn't killed the test.
Maybe that is related to ext/standard/tests/general_functions/bug39322.phpt
"Bug #39322 (proc_terminate() loosing process resource)"
Albeit that (https://bugs.php.net/bug.php?id=39322) should be fixed,
according to Rasmus.
Rasmus do you have any idea why is that test failing on 5.4 and trunk for
debian x86?
see:
http://ci.qa.php.net/job/php-src-5.4-matrix-tests/88/architecture=x86,os=linux-debian-6.0/testReport/ext_standard_tests_general_functions_bug39322/phpt/Bug__39322__proc_terminate___loosing_process_resource_/

(Continue reading)

headers

Ferenc Kovacs | 26 Nov 2011 01:05

Re: [PHP-DEV] CI for 5.4

Yesterday we were talking about enabling more extensions and adding pecl
> into the mix:
> http://www.mail-archive.com/internals <at> lists.php.net/msg54220.html
> I think it would be a good idea to try to fix the currently failing tests.
> http://ci.qa.php.net/job/php-src-5.3-matrix-tests/35/testReport/?
> http://ci.qa.php.net/job/php-src-5.4-matrix-tests/88/testReport/?
> http://ci.qa.php.net/job/php-src-trunk-matrix-tests/52/testReport/?
>
> There is only three failing tests left on the debian slaves:
>
> For 5.3
> ext/phar/tests/phar_oo_005.phpt.Phar and RecursiveDirectoryIterator 0.01
> 20
> ext/spl/tests/bug60082.phpt.Bug #60082 (100% CPU / when using references
> with ArrayObject(&$ref))
>
> For 5.4 and trunk
> ext/standard/tests/general_functions/bug39322.phpt.Bug #39322
> (proc_terminate() loosing process resource)
>
> For the freebsd failures, some of them can be tackled from the test (the
> locale related tests), but I see some things that need fixing in the code,
> rather than in the tests:
> https://bugs.php.net/bug.php?id=60186
> https://bugs.php.net/bug.php?id=60222
>
> I think I also come up with a solution for the xfail issue (where do we
> count those):
> I would add support to run-tests to be able only run the XFAILS, or skip
> them.

(Continue reading)

headers

Ferenc Kovacs | 2 Dec 2011 00:39

Re: [PHP-DEV] CI for 5.4

Hi,

Recent activity:

Alexey Shein dug through the failing symfony2 tests, and it turned out that
3 tests were bugged, he went ahead and fixed it (
https://github.com/symfony/symfony/pull/2750), one from the other two is a
sporadic test, it can fail randomly, the last one fails in our specific
setup (we don't do make install so their test won't find our php binary
at PHP_BINDIR).

The new Jenkins LTS got released, so I went ahead and updated our cluster,
there were a couple of hiccups, but everything should be back to normal now.

I also simplified our setup, I merged the build and test jobs, this will
make our life much easier.
This also made possible to simplify our build.xml, and I used that chance
to complicate it a little bit more :P
I added a condition and an environment variable to use gmake (gnu make) on
FreeBSD instead of make, which also made it possible to add -j2(a little
bit concurrency) to our make process.

The biggest news is that we polished up my original JUnit patch for
run-tests.php and with the help of Felipe, we have that in the svn
repository.
Felipe, Alexey, thanks again!

--

-- 
Ferenc Kovács
 <at> Tyr43l - http://tyrael.hu

(Continue reading)