headers
Richard Boulton | 21 Feb 2008 16:52

Branching for 1.1

Xapian 1.0 was released a little over 9 months ago now.  We've been 
adding features and fixing bugs throughout the 1.0 release series, and 
have now got a reasonably stable and solid release with 1.0.5.  There 
are still some outstanding bugs which are fixed in SVN but aren't yet in 
any release, and more are likely to be discovered over the coming 
months, so we'll be wanting to make at least a 1.0.6 release at some 
point, and I'd be surprised if there wasn't at least one further release 
after that in the 1.0 series.  I don't expect that we are likely to want 
to add any major new functionality to the 1.0 series now, though.

However, since 1.0.5 we've implemented various things on HEAD which 
probably don't belong in a 1.0.X release: in particular, the replication 
code, while an API addition (and thus, technically possible to include 
in 1.0.6) is still a bit immature, and it would be good to give it a few 
months of development time to settle before freezing it into a release.

I therefore think it is time to make a branch for continued development 
of the 1.0 release series, and to dedicate HEAD to development towards 
the 1.1 release series.

The appropriate branchpoint is probably just before we started adding 
replication support.  There would then be a few further changes to apply 
to the branch from HEAD which relate to bug fixes.

I'm happy to make such a branch, but thought I'd check if other 
developers (especially Olly) agree that now is a sensible time to make 
it, or have other suggestions.

--

-- 
Richard
(Continue reading)

Permalink | Reply |
headers
Olly Betts | 22 Feb 2008 11:28
Favicon
Gravatar

Re: Branching for 1.1

On Thu, Feb 21, 2008 at 03:52:52PM +0000, Richard Boulton wrote:
> However, since 1.0.5 we've implemented various things on HEAD which 
> probably don't belong in a 1.0.X release: in particular, the replication 
> code, while an API addition (and thus, technically possible to include 
> in 1.0.6) is still a bit immature, and it would be good to give it a few 
> months of development time to settle before freezing it into a release.

Hmm, "a few months" sounds a little worrying.  I've not really had a
play with it, but it would be bad if this held up 1.1.0.  Perhaps we
should consider creating a branch for it?

> I'm happy to make such a branch, but thought I'd check if other 
> developers (especially Olly) agree that now is a sensible time to make 
> it, or have other suggestions.

I'd prefer to hold off on branching until we're ready to make a 1.0.6
release.  My past experience of this sort of thing suggests that
tracking which changes are on which release branches is something of
a pain, so I'd rather do the branch creation, backporting of appropriate
changes, and 1.0.6 release over a short interval (and not while I'm
busily trying to emmigrate!)

Cheers,
    Olly
Permalink | Reply |
headers
Richard Boulton | 22 Feb 2008 11:48

Re: Branching for 1.1

Olly Betts wrote:
> On Thu, Feb 21, 2008 at 03:52:52PM +0000, Richard Boulton wrote:
>> However, since 1.0.5 we've implemented various things on HEAD which 
>> probably don't belong in a 1.0.X release: in particular, the replication 
>> code, while an API addition (and thus, technically possible to include 
>> in 1.0.6) is still a bit immature, and it would be good to give it a few 
>> months of development time to settle before freezing it into a release.
> 
> Hmm, "a few months" sounds a little worrying.  I've not really had a
> play with it, but it would be bad if this held up 1.1.0.  Perhaps we
> should consider creating a branch for it?

A few months is probably an overestimate - I needed to change the API in 
replication.h only a couple of days ago (to allow for logging of the 
changes involved in a replication operation), so it's feeling less API 
stable to me than it might otherwise do.  However, I'd known that that 
change was likely to be necessary for a while, and I don't know of any 
other changes which are likely to be needed to the replication API, so I 
don't think it's likely to hold up 1.1.0.

(There are a few things I'd like to improve about the implementation of 
the replication stuff - but they're just to improve test coverage and 
reorganise the code a bit, so don't need to block a release.)

>> I'm happy to make such a branch, but thought I'd check if other 
>> developers (especially Olly) agree that now is a sensible time to make 
>> it, or have other suggestions.
> 
> I'd prefer to hold off on branching until we're ready to make a 1.0.6
> release.  My past experience of this sort of thing suggests that
(Continue reading)

Permalink | Reply |
headers
Olly Betts | 22 Feb 2008 12:39
Favicon
Gravatar

Re: Branching for 1.1

On Fri, Feb 22, 2008 at 10:48:55AM +0000, Richard Boulton wrote:
> Fair point - but assuming we don't want the replication stuff in 1.0.6, 
> the branch point is either going to have to be at the point before the 
> replication stuff was added to HEAD, or we're going to have to remove 
> the replication stuff from HEAD temporarily to make a release, then 
> branch and add it back.

Yes, we want to branch from a revision just before the replication stuff
started to be committed, and then cherry pick changes since then to
backport.  But that's as true tomorrow as it is today - it doesn't
really argue for branching at any particular time (except perhaps for
branching before making the replication changes, but we didn't).  There
may be a few more changes we want to backport if we branch later, but
then they'd need to be backported anyway.

> I've just come up with another reason for branching now: I've just been 
> working on allowing Xapian::Stem to be subclassed from Python (to allow 
> a custom stemmer to be supplied to the queryparser) - it turns out that 
> Xapian::Stem::operator() isn't virtual, which is needs to be to allow 
> subclassing from Python.  I can't see any way to fix this without an API 
> change, so it needs to wait for the 1.1 series.

Will this actually work?  Subclassing our reference counted classes
doesn't really work in general...

Ideally Xapian::Stem shouldn't be reference counted, but Martin was
highly resistant to my snowball patch to move towards making the
stemming algorithm variables stack based rather than persistent in a
struct between invocations (which would allow Xapian::Stem to just wrap
an enum).
(Continue reading)

Permalink | Reply |
headers
Richard Boulton | 22 Feb 2008 12:57

Re: Branching for 1.1

Olly Betts wrote:
> Will this actually work?  Subclassing our reference counted classes
> doesn't really work in general...

Ah - it works, in that it passes my tests, but I'd better look into 
what's going on here a bit more carefully...

I think it's just about okay though: the subclass is a subclass of the 
interface class, not of the reference counted internals, so the subclass 
just inherits the same reference counted internals as the base class 
(and can choose not to initialise them by using the base class 
constructor, it if wishes).

I'm not sure whether it's safe if the copy constructor or assignment 
operator on the subclass are used, though - I'll double-check that.

> Ideally Xapian::Stem shouldn't be reference counted, but Martin was
> highly resistant to my snowball patch to move towards making the
> stemming algorithm variables stack based rather than persistent in a
> struct between invocations (which would allow Xapian::Stem to just wrap
> an enum).
 >
> IIRC, his main argument seemed to be for compatibility with a
> closed-source snowball compiler he has, so I'm not very sympathetic, but
> forking the language doesn't seem ideal.  It's not ideal that we have a
> forked snowball compiler, but at least those changes can hopefully be
> merged back fairly easily at some point (I don't think any are likely to
> be controversial).

Agreed, on all counts...
(Continue reading)

Permalink | Reply |
headers
Olly Betts | 22 Feb 2008 13:09
Favicon
Gravatar

Re: Branching for 1.1

On Fri, Feb 22, 2008 at 11:57:12AM +0000, Richard Boulton wrote:
> I'm not sure whether it's safe if the copy constructor or assignment 
> operator on the subclass are used, though - I'll double-check that.

I think it will fail, because we expect to be able to do things like:

    Xapian::Stem copy_of_stemmer = stemmer;

And that doesn't work if stemmer is a subclass of Xapian::Stem.

Mind you, making Xapian::Stem wrap an enum doesn't help here either...

Cheers,
    Olly
Permalink | Reply |
headers
Richard Boulton | 22 Feb 2008 13:20

Re: Branching for 1.1

Olly Betts wrote:
> On Fri, Feb 22, 2008 at 11:57:12AM +0000, Richard Boulton wrote:
>> I'm not sure whether it's safe if the copy constructor or assignment 
>> operator on the subclass are used, though - I'll double-check that.
> 
> I think it will fail, because we expect to be able to do things like:
> 
>     Xapian::Stem copy_of_stemmer = stemmer;
> 
> And that doesn't work if stemmer is a subclass of Xapian::Stem.
> 
> Mind you, making Xapian::Stem wrap an enum doesn't help here either...

I agree.  Argh.

One API for this, then, would be to change Xapian::Stem to be an 
abstract class, and move the implementation to a Xapian::SnowballStem 
class.  However, this would break too much existing code to be a 
reasonable solution, in the near term, anyway.

What about the following:
  - add a Xapian::BaseStem class, which is abstract.
  - make Xapian::Stem inherit from this.
  - change TermGenerator and QueryParser to take a Xapian::BaseStem 
instead of a stem class.

User code would continue to work (after a recompile), but users could 
also create their own subclasses (of BaseStem) and use them with 
TermGenerator and QueryParser.
(Continue reading)

Permalink | Reply |
headers
Richard Boulton | 22 Feb 2008 12:15

Re: Branching for 1.1

I should add that, as far as I can tell from our current plans, the 
change from 1.0 to 1.1 should be a lot less traumatic than the change 
from 0.9 to 1.0.  There's a fairly short list of planned deprecations, 
all of which have simple replacements (or need no replacement, because 
the originals did nothing!).  Other than these, code for 1.0 should 
simply need a recompile to work with 1.1.  So, it should be easy for 
users to upgrade to 1.1 if they want new features, and therefore I don't 
think there'll be a need to maintain 1.0 for long (if at all) after 1.1 
is released, and there is no need to backport new features to 1.0 after 
we've branched for 1.1 - just backporting bugfixes should suffice.

I'd hope (and I get the impression Olly strongly agrees here) that it 
only takes at most a couple of months to get from the point at which we 
branch for 1.1 to the point at which we release.  I think the 
replication stuff won't jeopardise this, and the other branches I'd like 
to merge will only be merged as and when they're stable enough to be 
released.

I think we should only do very small new features directly on HEAD while 
waiting for 1.1 - any more ambitious new features which we hope to get 
into 1.1 should be done in branches (for example, some work I'm about to 
start on for supporting geolocation searches).  Putting the replication 
stuff directly on HEAD was probably a mistake, but it's done now, and 
I'm not sure moving it to a branch would be worth the effort.

--

-- 
Richard
Permalink | Reply |

Gmane