Lars Wirzenius | 10 Mar 23:35 2013
Picon

All candidates: Development and technical issues and challenges

Gegerly, Moray, and Lucas:

We're currently in the middle of a freeze for the next release. We've
been in this release since June 30. That's over eight months. That's a
long time, even for a project the size of Debian. Releasing when we're
ready is all well and good, but it's a problem when it takes us so long
to get ready.

In your opinion, what are the fundamental reasons the release freeze is so
long, and so painful, and what do you propose to do, as DPL, to fix them?

What other development process problems do you see, ones that do not
directly affect the freeze, but make the development of Debian less
productive or less interesting?

Finally, what are the main technical challenges you see for Debian in
the next five years and what should we, as a project, do about them?

--

-- 
http://www.cafepress.com/trunktees -- geeky funny T-shirts
http://gtdfh.branchable.com/ -- GTD for hackers
Lucas Nussbaum | 11 Mar 14:19 2013
Picon

Re: All candidates: Development and technical issues and challenges

> In your opinion, what are the fundamental reasons the release freeze is so
> long, and so painful, and what do you propose to do, as DPL, to fix them?

The release process is hard to grasp. The most visible side of it is the 
number of RC bugs (which I contributed to increase :p) As a DPL, I will 
continue to support QA tests, because identifying bugs early is much better 
than running into them late in the release process. I will also explore ways 
to make fixing RC bugs more rewarding. For example, in the Debian Project 
News, we could list the most efficient RC bug squashers.

But then, while in the ideal world, all RC bugs would be fixed, it's not that
much of a problem to release with some RC bugs. There are other critical
part of the release process: to release, we need an installer, release notes, 
etc. Those are parts of the work that are unfortunately not very rewarding and 
visible, and for which we have always had problems to find volunteers.
There's probably a small margin of improvement for the release team to explain
what are the requirements for releasing (a "why releasing is hard" 
presentation). As a DPL, I will encourage/help them on that path, and also try 
to recruit people to work where it's most needed (installer, release notes, 
release team, etc.).

On a more personal opinion, I would welcome more exploration of gradual 
freezes. I understand that it's important to freeze early packages that affect 
the installer, and maybe also the base system. But freezing all packages with 
the hope that it will force people to fix RC bugs in other people's packages 
does not work: many people just stop working on Debian during the freeze.
This was discussed a bit already, but I'm not sure that we really were 
throughout in the discussion. As a DPL, I will reopen that discussion at a 
good time (after the release, and not just after the release).

(Continue reading)

Paul Wise | 11 Mar 14:49 2013
Picon

Re: All candidates: Development and technical issues and challenges

On Mon, Mar 11, 2013 at 9:19 PM, Lucas Nussbaum wrote:

> to make fixing RC bugs more rewarding. For example, in the Debian Project
> News, we could list the most efficient RC bug squashers.

Just discussed this in #debian-publicity, if you can write a query to
run against UDD, the publicity team is definitely interested in doing
that.

--

-- 
bye,
pabs

http://wiki.debian.org/PaulWise

Lucas Nussbaum | 11 Mar 15:02 2013
Picon

Re: All candidates: Development and technical issues and challenges

On 11/03/13 at 21:49 +0800, Paul Wise wrote:
> On Mon, Mar 11, 2013 at 9:19 PM, Lucas Nussbaum wrote:
> 
> > to make fixing RC bugs more rewarding. For example, in the Debian Project
> > News, we could list the most efficient RC bug squashers.
> 
> Just discussed this in #debian-publicity, if you can write a query to
> run against UDD, the publicity team is definitely interested in doing
> that.

What is easy to do, is get the top RC bugs *closers* for the last n
days. But there are more ways to be an efficient RC bug squasher.

Query for the top 10 closers in March:

udd=> select done, count(*) from bugs where severity >= 'serious' and
last_modified >= '2013-03-01' and status='done' group by done
 order by count desc limit 10;
                   done                    | count 
-------------------------------------------+-------
 Michael Gilbert <mgilbert <at> debian.org>     |     8
 LaMont Jones <lamont <at> debian.org>          |     6
 Julien Cristau <jcristau <at> debian.org>      |     4
 Ludovico Cavedon <cavedon <at> debian.org>     |     4
 Raphaël Hertzog <hertzog <at> debian.org>      |     4
 gregor herrmann <gregoa <at> debian.org>       |     3
 Sebastian Ramacher <sramacher <at> debian.org> |     3
 Abou Al Montacir <abou.almontacir <at> sfr.fr> |     3
 Arno Töll <arno <at> debian.org>               |     3
 Sébastien Villemot <sebastien <at> debian.org> |     3
(Continue reading)

Moray Allan | 11 Mar 20:30 2013

Re: All candidates: Development and technical issues and challenges

On 2013-03-11 01:35, Lars Wirzenius wrote:
> In your opinion, what are the fundamental reasons the release freeze 
> is so
> long, and so painful, and what do you propose to do, as DPL, to fix 
> them?

On one level I'm cautious about answering this.  I don't think that a 
DPL should try to impose policies on teams like the Release Team, or 
push their own ideas on this kind of topic rather than trying neutrally 
to push forward a project discussion.

However, to give some of my own views, since you ask for it:

Background: It seems to me that part of the problem comes from the 
Release Team's own past success.  Individual maintainers have got used 
to Debian releases happening more or less painlessly, and just assume 
that the Release Team will get on with the work, and release when it's 
ready.  But, of course, the release should be the responsibility of all 
maintainers -- and if too many of us just look after their own packages 
and leave the Release Team and a few helpers to do the rest, we will be 
waiting until the slowest maintainer has fixed the hardest bug.  At the 
same time, as a freeze gets longer, and without a specific immediate 
deadline, it becomes harder to motivate people to put energy into 
helping.

Earlier removals: I wonder if removing RC-buggy packages much earlier 
in the freeze would help -- even if it's logically no different to 
saying they will be removed later, it might wake up maintainers into 
bug-fixing action faster, and especially maintainers of packages that 
are removed due to their dependencies.
(Continue reading)

Paul Wise | 12 Mar 05:17 2013
Picon

Re: All candidates: Development and technical issues and challenges

On Tue, Mar 12, 2013 at 3:30 AM, Moray Allan wrote:

> Earlier removals: I wonder if removing RC-buggy packages much earlier in the
> freeze would help -- even if it's logically no different to saying they will
> be removed later, it might wake up maintainers into bug-fixing action
> faster, and especially maintainers of packages that are removed due to their
> dependencies.

Removing packages in the freeze is way too late, they should be
removed from testing in an (semi-)automated fashion during the whole
release cycle. IIRC the release team are planning on doing this and
have done it manually in the past.

> Flag up RC bugs: To tackle things earlier in the cycle, perhaps we could
> push use of some tools[1] that more actively flag up new RC-buggy packages
> to users of testing?

There is apt-listbugs but does that work for things like PackageKit or
software-center?

--

-- 
bye,
pabs

http://wiki.debian.org/PaulWise

Moray Allan | 12 Mar 12:35 2013

Re: All candidates: Development and technical issues and challenges

On 2013-03-12 07:17, Paul Wise wrote:
> Removing packages in the freeze is way too late, they should be
> removed from testing in an (semi-)automated fashion during the whole
> release cycle. IIRC the release team are planning on doing this and
> have done it manually in the past.

Indeed -- I should really have said something like "much earlier in the 
release cycle".

> There is apt-listbugs but does that work for things like PackageKit 
> or
> software-center?

I'll be interested to hear what tools already exist that I've missed.  
Then the challenge is to get them onto more machines in such a way that 
people pay attention to what they say.  Normally people are 
installing/updating packages because they're trying to achieve some 
goal, and they won't tend to interrupt that to debug issues that might 
be mentioned in messages there.  For some contributors, a popup 
notification about new RC bugs like existing "upgrades are available" 
ones might be useful, though clearly for many users this would just be 
an unhelpful worry.

--

-- 
Moray

Serafeim Zanikolas | 12 Mar 13:14 2013
Picon

[OT] flag RC-buggy packages to users of testing

[M-F-T set to 628835 <at> bugs.debian.org, as I believe this is becoming OT]

On Tue, Mar 12, 2013 at 12:17:29PM +0800, Paul Wise wrote:
[..]
> On Tue, Mar 12, 2013 at 3:30 AM, Moray Allan wrote:
[..]
> > Flag up RC bugs: To tackle things earlier in the cycle, perhaps we could
> > push use of some tools[1] that more actively flag up new RC-buggy packages
> > to users of testing?
> 
> There is apt-listbugs but does that work for things like PackageKit or
> software-center?

software-center is unaware of apt-listbugs and PackageKit explicitly disables
it (because it otherwise hangs while apt-listbugs waits for console-based
input).

Two things need to happen:

- apt-listbugs design should be revised for invocation by programs (as opposed
  to manual/interactive invocation)

- high level package managers must learn how to interact with apt-listbugs

--

-- 
Every great idea is worthless without someone to do the work. --Neil Williams
anarcat | 19 Mar 00:41 2013
Picon

Re: All candidates: Development and technical issues and challenges

On 2013-03-11, Moray Allan wrote:
> When to release: I would also note that we should continue to be
> flexible about "-ignore" tags where appropriate.  In some cases
> leaving a package in the release with RC bugs is more useful to users
> than removing it altogether.  Indeed, we always release with quite a
> large number of non-RC bugs, some of which make the packages in
> question unusable for large groups of users.  At any point in the
> freeze we should ask not only about the state of the frozen release,
> but how it compares to the previous release.  Maybe it doesn't even
> need to be a single date -- we could badge the new release as ready
> for the desktop before we close it off as final and suggest that
> people upgrade their servers.

I think this is a great point, and I would like to push it a little
further.

"When to release" seems really important. As things stand right now, we
have about 70 packages (assuming one package per RC bug) blocking the
release of 38000 packages[1]. That is 0.2% of the archive. It's really
small. About 0.1% if we look at the ones that don't have a fix yet (49).

Shouldn't we be releasing 'as is' at some point and just accept that
some bugs will be fixed in a stable release later?

Shouldn't we "release early, release often"? 

I agree that releasing with, say, 1000 RC bugs is crazy, but maybe
waiting forever for the last 100 packages is also nonsense.

Maybe we could discriminate on the package's priorities. For example,
(Continue reading)

Lucas Nussbaum | 19 Mar 08:14 2013
Picon

Re: All candidates: Development and technical issues and challenges

On 18/03/13 at 19:41 -0400, anarcat wrote:
> On 2013-03-11, Moray Allan wrote:
> > When to release: I would also note that we should continue to be
> > flexible about "-ignore" tags where appropriate.  In some cases
> > leaving a package in the release with RC bugs is more useful to users
> > than removing it altogether.  Indeed, we always release with quite a
> > large number of non-RC bugs, some of which make the packages in
> > question unusable for large groups of users.  At any point in the
> > freeze we should ask not only about the state of the frozen release,
> > but how it compares to the previous release.  Maybe it doesn't even
> > need to be a single date -- we could badge the new release as ready
> > for the desktop before we close it off as final and suggest that
> > people upgrade their servers.
> 
> I think this is a great point, and I would like to push it a little
> further.
> 
> "When to release" seems really important. As things stand right now, we
> have about 70 packages (assuming one package per RC bug) blocking the
> release of 38000 packages[1]. That is 0.2% of the archive. It's really
> small. About 0.1% if we look at the ones that don't have a fix yet (49).
> 
> Shouldn't we be releasing 'as is' at some point and just accept that
> some bugs will be fixed in a stable release later?

That's what we do. We use 'wheezy-ignore' for some bugs.

But I think that it would be inappropriate to comment further on the
release team's work. I think they are doing a great job.

(Continue reading)

Michael Gilbert | 20 Mar 06:15 2013
Picon

Re: All candidates: Development and technical issues and challenges

On Tue, Mar 19, 2013 at 3:14 AM, Lucas Nussbaum wrote:
>> Maybe we could discriminate on the package's priorities. For example,
>> about a third of the 49 packages *really* blocking the release (not
>> waiting for a transition) are from "extra"[2]. Only 5 bugs affect
>> required, important or standard packages. We could focus on those and
>> tell the "extra packages" to hurry up or be shipped with packages that
>> will need to be fixed in a point release... or simply removed.
>
> That's something I already commented on in
> https://lists.debian.org/debian-vote/2013/03/msg00020.html:
>
>    Another possible area of improvement is the focusing on the more
>    important RC bugs. One way to achieve that would be to remove as many
>    leaf/not-so-popular packages as possible at the start of the freeze.
>    If they get fixed, they could get back in.

Even the suggestion of a testing removal can evoke negative feelings
for those affected (sometimes from those on the sidelines too).  A
recent example:
http://bugs.debian.org/703258

Do you have any thoughts on addressing the social aspect of this
approach?  In actuality, a testing removal is really not a big deal
since the package can come right back once the RC bug is fixed.  Even
so, some see removals as a kind of judgement on themselves as
maintainer.  What can be said or done to qualm the fear and anxiety?

Best wishes,
Mike

(Continue reading)

Lucas Nussbaum | 20 Mar 12:01 2013
Picon

Re: All candidates: Development and technical issues and challenges

On 20/03/13 at 01:15 -0400, Michael Gilbert wrote:
> On Tue, Mar 19, 2013 at 3:14 AM, Lucas Nussbaum wrote:
> >> Maybe we could discriminate on the package's priorities. For example,
> >> about a third of the 49 packages *really* blocking the release (not
> >> waiting for a transition) are from "extra"[2]. Only 5 bugs affect
> >> required, important or standard packages. We could focus on those and
> >> tell the "extra packages" to hurry up or be shipped with packages that
> >> will need to be fixed in a point release... or simply removed.
> >
> > That's something I already commented on in
> > https://lists.debian.org/debian-vote/2013/03/msg00020.html:
> >
> >    Another possible area of improvement is the focusing on the more
> >    important RC bugs. One way to achieve that would be to remove as many
> >    leaf/not-so-popular packages as possible at the start of the freeze.
> >    If they get fixed, they could get back in.
> 
> Even the suggestion of a testing removal can evoke negative feelings
> for those affected (sometimes from those on the sidelines too).  A
> recent example:
> http://bugs.debian.org/703258
> 
> Do you have any thoughts on addressing the social aspect of this
> approach?  In actuality, a testing removal is really not a big deal
> since the package can come right back once the RC bug is fixed.  Even
> so, some see removals as a kind of judgement on themselves as
> maintainer.  What can be said or done to qualm the fear and anxiety?

It is the release team's role to decide what's inside testing, and I
think that it's important to respect that.
(Continue reading)

gregor herrmann | 20 Mar 20:15 2013
X-Face
Picon

Re: All candidates: Development and technical issues and challenges

On Wed, 20 Mar 2013 01:15:58 -0400, Michael Gilbert wrote:

> Even the suggestion of a testing removal can evoke negative feelings
> for those affected (sometimes from those on the sidelines too).  A
> recent example:
> http://bugs.debian.org/703258

There seems to be a misunderstanding; at least my surprise was not
caused by the proposal to remove the package from testing (actually,
I mentioned this as the most probably option in my first mail in the
original bug report), but by the fact that I wouldn't have known
about the removal bug without the later CC by Jonathan.

> Do you have any thoughts on addressing the social aspect of this
> approach?

In this case, "X-Debbugs-Cc: $original_bug|$maintainer_address" would
have been enough :)

Cheers,
gregor

--

-- 
 .''`.  Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
 : :' : Debian GNU/Linux user, admin, and developer  -  http://www.debian.org/
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   NP: Jimi Hendrix: Message To Love
Gergely Nagy | 16 Mar 12:59 2013

Re: All candidates: Development and technical issues and challenges

Lars Wirzenius <liw <at> liw.fi> writes:

> Gegerly, Moray, and Lucas:
>
> We're currently in the middle of a freeze for the next release. We've
> been in this release since June 30. That's over eight months. That's a
> long time, even for a project the size of Debian. Releasing when we're
> ready is all well and good, but it's a problem when it takes us so long
> to get ready.
>
> In your opinion, what are the fundamental reasons the release freeze is so
> long, and so painful, and what do you propose to do, as DPL, to fix them?

The fundamental reason is that fixing bugs isn't all that rewarding in
many cases, and considerably harder than doing packaging, especially in
cases where one can't rely on upstream help either (for any of the
million reasons).

What we could and should do (and this includes everyone, not just the
DPL) is to make upstreams, downstreams and everyone else involved
realise that if we work together, we'll release faster, and if we
release faster, we'll have more up to date software, which benefits
everyone.

I've heard many an upstreams (and users of downstreams too) complain
about how old packages in Debian are. I myself am annoyed too about this
from time to time, when I'm wearing my syslog-ng upstream hat, for
example. But the only proper way to make things better if we combine our
knowledge.

(Continue reading)


Gmane