Mark de Wever | 15 Feb 2012 20:56
Picon
Picon
Favicon

Wesnoth project hosting

Hi,

At the FOSDEM we discussed again about whether or not moving to git and
thus from the location our source repository is hosted. The discussion
is listed [1] under Git.

Personally I see not too much advantage by Git, git-svn works good
enough for me. However IMO the uptime of GNA is getting lower over the
years (it might be selective memory as well). So moving to another
place to host our source repository might be a good idea. Am I the only
one who thinks it would be a good idea to try to find a more reliable
hosting party? Note moving to another hosting party does not mean
switching to Git per se.

How do we feel about moving to Git? As said I see not too much advantage
by Git, but I'm not opposed to move. If so are there preferences over
the hosting party? Rhonda also said she can probably host Git for us.

If we decide to move, do we only want to move the source repository or
also the bug-tracker and mailing lists? I haven't investigated whether
or not it's possible to export our bug-tracker's contents.

[1] http://wiki.wesnoth.org/FOSDEM2012#Mordante.27s_.60email.27
--

-- 
Regards,
Mark de Wever aka Mordante/SkeletonCrew
Bruno Wolff III | 15 Feb 2012 21:22
Picon

Re: Wesnoth project hosting

On Wed, Feb 15, 2012 at 20:56:36 +0100,
  Mark de Wever <koraq <at> xs4all.nl> wrote:
> 
> At the FOSDEM we discussed again about whether or not moving to git and
> thus from the location our source repository is hosted. The discussion
> is listed [1] under Git.

Does GNA have any plans to support git in the future?
Eric S. Raymond | 15 Feb 2012 22:07
Picon
Gravatar

Re: Wesnoth project hosting

Bruno Wolff III <bruno <at> wolff.to>:
> On Wed, Feb 15, 2012 at 20:56:36 +0100,
>   Mark de Wever <koraq <at> xs4all.nl> wrote:
> > 
> > At the FOSDEM we discussed again about whether or not moving to git and
> > thus from the location our source repository is hosted. The discussion
> > is listed [1] under Git.
> 
> Does GNA have any plans to support git in the future?

I'm a GNA admin, though not active.  Short answer: no.  Longer answer: no,
and the odds that will change look slim to me.  The GNA codebase is old
and inflexible, and the man-hours required to add features of that
magnitude aren't available.
--

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
Eric S. Raymond | 15 Feb 2012 22:04
Picon
Gravatar

Re: Wesnoth project hosting

Mark de Wever <koraq <at> xs4all.nl>:
> At the FOSDEM we discussed again about whether or not moving to git and
> thus from the location our source repository is hosted. The discussion
> is listed [1] under Git.
> 
> Personally I see not too much advantage by Git, git-svn works good
> enough for me. However IMO the uptime of GNA is getting lower over the
> years (it might be selective memory as well).

It probably isn't.  I've noticed it as well.

>                               So moving to another
> place to host our source repository might be a good idea. Am I the only
> one who thinks it would be a good idea to try to find a more reliable
> hosting party? Note moving to another hosting party does not mean
> switching to Git per se.

No, but if we're going to change repo hosting anyway the additional overhead
of changing VCSes is much more tolerable by comparison.  Better one clean 
break than two traumas.

> How do we feel about moving to Git? As said I see not too much advantage
> by Git, but I'm not opposed to move. If so are there preferences over
> the hosting party? Rhonda also said she can probably host Git for us.

I would be in favor of moving to git.  If nothing else it would reduce 
incremental update times a lot.

The project would have one unusual advantage in such a move: me!  As the
author and maintainer of reposurgeon, and having done several svn-to-git
(Continue reading)

Iurii Chernyi | 15 Feb 2012 22:17

Re: Wesnoth project hosting

Hello,

On Wed, Feb 15, 2012 at 10:04 PM, Eric S. Raymond <esr <at> thyrsus.com> wrote:
> I have written a tool that can scrape the GNA bugtracker.  The problem isn't
> export but import - loading it into anywhere else.  It is probably not
> yet feasible to move the tracker.

In the bugtracker, we have bugs, comments, patches, attached files -
the data model is rather small. If we pick the new bugtracker ahead of
time, it should be possible to prepare for migration with minimal
losses of data, writing the code for import and testing it. Even if
we'll import via 'standard' input channels, we'll successfully salvage
most of the content. If the new bugtracker will have some bulk import
capabilities (or if it'd be possible to negotiate for such a
capability or host our own bugtracker instance), even better.

So, I think that we should start figuring out 'where to move'.

--

-- 
Cheers, Iurii Chernyi
Timotei Dolean | 15 Feb 2012 22:32
Picon
Favicon
Gravatar

Re: Wesnoth project hosting

As far as I and other devs checked, unfortunately Github doesn't offer 
attachment functionality - which is vital for our bugs (think of replays 
for example).

On the other hand, Sourceforge.net might be a good contender.

Timo

On 15-Feb-12 23:17, Iurii Chernyi wrote:
> Hello,
>
> On Wed, Feb 15, 2012 at 10:04 PM, Eric S. Raymond<esr <at> thyrsus.com>  wrote:
>> I have written a tool that can scrape the GNA bugtracker.  The problem isn't
>> export but import - loading it into anywhere else.  It is probably not
>> yet feasible to move the tracker.
> In the bugtracker, we have bugs, comments, patches, attached files -
> the data model is rather small. If we pick the new bugtracker ahead of
> time, it should be possible to prepare for migration with minimal
> losses of data, writing the code for import and testing it. Even if
> we'll import via 'standard' input channels, we'll successfully salvage
> most of the content. If the new bugtracker will have some bulk import
> capabilities (or if it'd be possible to negotiate for such a
> capability or host our own bugtracker instance), even better.
>
> So, I think that we should start figuring out 'where to move'.
>
Ignacio Riquelme Morelle | 15 Feb 2012 22:33
Picon
Gravatar

Re: Wesnoth project hosting

On Wednesday 15 February 2012 18:04:51 Eric S. Raymond wrote:
> Mark de Wever <koraq <at> xs4all.nl>:
> > Personally I see not too much advantage by Git, git-svn works good
> > enough for me. However IMO the uptime of GNA is getting lower over the
> > years (it might be selective memory as well).
> 
> It probably isn't.  I've noticed it as well.

I haven't really had any persistent problems with gna.org's service for a long 
while myself, other than the recent, prolonged downtime we all know of, and 
that can be mostly attributed to poor planning. Better staffed services with 
fresh codebases occasionally have to deal with things like distributed denial 
of service attacks or security breaches, so I'm not sure there's a real 
difference for a project like Wesnoth where there isn't constant 24/7 
activity.

That isn't to say there isn't a _problem_. I just think there's too much 
emphasis on a rather insubstantial benefit along that line.

I have repeatedly expressed concerns for moving to Git on IRC, so I'll try to 
summarize them here for those who have not (been able) paying attention:

1) With SVN, the bulk of the initial download on 'svn checkout' appears to be 
the snapshot of HEAD that serves as the client's reference to resolve 
differences against the BASE revision. The size of the uncompressed source 
distribution tarball we offer through SF.net should be a good reference for 
this size. (I'm deliberately ignoring metadata and the visible file duplicates 
in the checkout tree.)

Using Git, users (i.e. developers, prospective contributors, early adopters 
(Continue reading)

Eric S. Raymond | 16 Feb 2012 05:22
Picon
Gravatar

Re: Wesnoth project hosting

Ignacio Riquelme Morelle <shadowm2006 <at> gmail.com>:
> In the matter of CIA.vc hooks for Git, all those I have seen so far have 
> annoying limitations such as the inability to present real merges in a useful 
> fashion, commits being reported out of order, or no tag object reports. I 
> believe I have seen CIA reporting commits to hosted KDE projects addressing 
> some of these points, but I haven't found the hook's source yet.

Talk to me.  I maintain the CIA hooks that are in the git project
repo.  If you know of a more advanced version, I'm willing to merge
those features and changes.

> 3) The learning curve and organization. At the beginning (and even
> afterwards, as new people join the project) it will be very chaotic
> unless someone steps forward and provides users with links to
> documentation (since IME even code developers fail to RTFM at
> times), and most importantly, style guidelines covering commit
> style/format, usage of branches, real merges vs. rebasing, and so
> on.

I know how to handle this.  I've managed several large svn-to-git
transitions.
--

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
_______________________________________________
Wesnoth-dev mailing list
Wesnoth-dev <at> gna.org
https://mail.gna.org/listinfo/wesnoth-dev
(Continue reading)

Ilor | 16 Feb 2012 10:17
Picon

Re: Wesnoth project hosting

On 15 February 2012 22:33, Ignacio Riquelme Morelle
<shadowm2006 <at> gmail.com> wrote:
> 1) With SVN, the bulk of the initial download on 'svn checkout' appears to be
> the snapshot of HEAD that serves as the client's reference to resolve
> differences against the BASE revision. The size of the uncompressed source
> distribution tarball we offer through SF.net should be a good reference for
> this size. (I'm deliberately ignoring metadata and the visible file duplicates
> in the checkout tree.)
>
> Using Git, users (i.e. developers, prospective contributors, early adopters
> and players taking advantage of tags) will have to download the entire history
> the first time in one go (git clone doesn't currently have a mechanism to
> resume interrupted operations).
>
> Taking my git-svn tree as a reference, that would be something around 1.7 GiB.
> I wouldn't have been able to download this much when I first joined, and I'm
> not able to at the moment (nor for the foreseeable future) either.

Hi all,

Isn't most of the 1.7GB music, with code and images etc. taking up a
relatively small amount of space? Wouldn't it be possible to use the
opportunity of a repo switch to also separate music and other binary
blobs into a different repository to make the core repo smaller and
easier to work with?

Ilor
Mark de Wever | 16 Feb 2012 20:26
Picon
Picon
Favicon

Re: Wesnoth project hosting

On Thu, Feb 16, 2012 at 10:17:03AM +0100, Ilor wrote:
> Isn't most of the 1.7GB music, with code and images etc. taking up a
> relatively small amount of space? Wouldn't it be possible to use the
> opportunity of a repo switch to also separate music and other binary
> blobs into a different repository to make the core repo smaller and
> easier to work with?

We discussed this option, but IIRC boucman had experience with it and
wasn't too thrilled about it. I also wonder how big the burden is to
download 1.7 GB of data.

--

-- 
Regards,
Mark de Wever aka Mordante/SkeletonCrew
Fabian Mueller | 16 Feb 2012 20:45
Picon
Picon

Re: Wesnoth project hosting

On 02/16/2012 10:17 AM, Ilor wrote:
> Hi all,
>
> Isn't most of the 1.7GB music, with code and images etc. taking up a
> relatively small amount of space? Wouldn't it be possible to use the
> opportunity of a repo switch to also separate music and other binary
> blobs into a different repository to make the core repo smaller and
> easier to work with?
>
> Ilor
>
Hi Ilor,
I have made this suggestion several times in the past since
the huge amount of data is a real problem when you work with several
local copies of the source tree.

If I understood git might solve that problem without a split of the 
repository,
because local forks are cheap with it.

Fabian
Ignacio Riquelme Morelle | 16 Feb 2012 21:05
Picon
Gravatar

Re: Wesnoth project hosting

On Thursday 16 February 2012 16:45:51 Fabian Mueller wrote:
> If I understood git might solve that problem without a split of the
> repository,
> because local forks are cheap with it.

I'm not really sure what that's supposed to solve. Forking does not imply 
separate code and resource repositories, and you cannot make usable partial 
clones of a existing repository with Git (although you can make shallow 
clones, which are not compatible with push operations, and you still get all 
files in the tree).

--

-- 
Regards
  Ignacio Riquelme Morelle <shadowmaster>
_______________________________________________
Wesnoth-dev mailing list
Wesnoth-dev <at> gna.org
https://mail.gna.org/listinfo/wesnoth-dev
jeremy rosen | 17 Feb 2012 08:52
Picon
Picon

Re: Wesnoth project hosting

quick reply on the sub-project aspect...

sub-modules exist in git but they are not very practical... they are
OK for lightly coupled projects (music and translations) but I
wouldn't recommand it for data/ because WML must be kept closely in
sync with code and I wouldn't recommand it for image/ either because
it needs to be kept closely in sync with data/

for those who know git : git submodule is just a smart wrapper around
a versionned file in .git that contains the URL of a repository and
the id of the commit to use in that repository for the current state
of the parent repo. Nothing fancy but it does the job.

IIRC it doesnt do things like automatically pulling the sub-repo and
things like that. You handle it more or less like two separate git
repo except that you can easily find out what commit is needed in the
child with a particular commit in the parent.

Boucman
Martin Renold | 18 Feb 2012 14:41
Picon
Picon
Gravatar

Re: Wesnoth project hosting

On Thu, Feb 16, 2012 at 05:05:58PM -0300, Ignacio Riquelme Morelle wrote:
> On Thursday 16 February 2012 16:45:51 Fabian Mueller wrote:
> > If I understood git might solve that problem without a split of the
> > repository, because local forks are cheap with it.
> 
> I'm not really sure what that's supposed to solve. Forking does not imply 
> separate code and resource repositories, and you cannot make usable partial 
> clones of a existing repository with Git (although you can make shallow 
> clones, which are not compatible with push operations, and you still get all 
> files in the tree).

If you are concerned only about local disk space (and activity) git helps
there either way.  Local branches eliminate some of the need for multiple
svn sandboxes.  But if you do use them, git will use hardlinks ('git clone')
or symlinks ('git-new-workdir') to share the repository data.

--

-- 
Martin Renold
Mark de Wever | 16 Feb 2012 20:15
Picon
Picon
Favicon

Re: Wesnoth project hosting

On Wed, Feb 15, 2012 at 06:33:03PM -0300, Ignacio Riquelme Morelle wrote:
> On Wednesday 15 February 2012 18:04:51 Eric S. Raymond wrote:
> > Mark de Wever <koraq <at> xs4all.nl>:
> > > Personally I see not too much advantage by Git, git-svn works good
> > > enough for me. However IMO the uptime of GNA is getting lower over the
> > > years (it might be selective memory as well).
> > 
> > It probably isn't.  I've noticed it as well.
> 
> I haven't really had any persistent problems with gna.org's service for a long 
> while myself, other than the recent, prolonged downtime we all know of, and 
> that can be mostly attributed to poor planning. Better staffed services with 
> fresh codebases occasionally have to deal with things like distributed denial 
> of service attacks or security breaches, so I'm not sure there's a real 
> difference for a project like Wesnoth where there isn't constant 24/7 
> activity.

True we don't need to have 24/7 support, I just have the feeling the
quality of GNA is getting lower and esr's post confirms that.

> I have repeatedly expressed concerns for moving to Git on IRC, so I'll try to 
> summarize them here for those who have not (been able) paying attention:
> 
> 1) With SVN, the bulk of the initial download on 'svn checkout' appears to be 
> the snapshot of HEAD that serves as the client's reference to resolve 
> differences against the BASE revision. The size of the uncompressed source 
> distribution tarball we offer through SF.net should be a good reference for 
> this size. (I'm deliberately ignoring metadata and the visible file duplicates 
> in the checkout tree.)
> 
(Continue reading)

Mark de Wever | 15 Feb 2012 22:42
Picon
Picon
Favicon

Re: Wesnoth project hosting

On Wed, Feb 15, 2012 at 04:04:51PM -0500, Eric S. Raymond wrote:
> > place to host our source repository might be a good idea. Am I the only
> > one who thinks it would be a good idea to try to find a more reliable
> > hosting party? Note moving to another hosting party does not mean
> > switching to Git per se.
> 
> No, but if we're going to change repo hosting anyway the additional overhead
> of changing VCSes is much more tolerable by comparison.  Better one clean 
> break than two traumas.

True, however I want to consider moving even if we decide not to move to
Git.

> On the other hand, one of the things that experience tells me is that
> a Wesnoth migration would be a huge, messy project.  The sheer size
> of the repo would produce a scale problem in itself - a full download of
> all branches runs my development machine out of disk space!

I assume that is before you do a git repacking operation. Our git-svn
repo is less than 2 GB and AFAIK only misses the website branch.

> This doesn't mean it can't be done.  It does mean it can't be done
> *quickly*.  But we probably want to wait until I ship reposurgeon 2.0
> anyway; I'm still chasing some bugs in the svn support.

Haven't looked at reposurgeon yet, but if that tool can make it easier
the better :-)

> > If we decide to move, do we only want to move the source repository or
> > also the bug-tracker and mailing lists? I haven't investigated whether
(Continue reading)

Oleg Tsarev | 16 Feb 2012 05:15
Picon

Re: Wesnoth project hosting

Hi guys,

Let me summarize points about project hosting (which I found on wiki & letters):
1) Project hosting should both code hosting & issue/bug tracking

Github allows this:
https://github.com/battle-for-wesnoth/svn/issues?milestone=1&state=open

2) Project hosting should track references between code & issues

https://github.com/blog/831-issues-2-0-the-next-generation

3) Git has a high learning curve (important for not-a-developers)

We can has a main subversion repository and upload data to git by
svn-post-commit hooks.
git svn dcommit allows to commit to svn directly
git allows manage separated branches, and commit their to svn when
branch is completed.

4) Github has size limitation

(NOTE: just for a non-free projects, as you can easy see
https://github.com/battle-for-wesnoth/svn - complete BfW repository on
github)

5) We are going to host addons on git/similar.

github allows this

(Continue reading)

Eric S. Raymond | 16 Feb 2012 05:29
Picon
Gravatar

Re: Wesnoth project hosting

Mark de Wever <koraq <at> xs4all.nl>:
> > On the other hand, one of the things that experience tells me is that
> > a Wesnoth migration would be a huge, messy project.  The sheer size
> > of the repo would produce a scale problem in itself - a full download of
> > all branches runs my development machine out of disk space!
> 
> I assume that is before you do a git repacking operation. Our git-svn
> repo is less than 2 GB and AFAIK only misses the website branch.

I was talking about a svn download. :-)

> > This doesn't mean it can't be done.  It does mean it can't be done
> > *quickly*.  But we probably want to wait until I ship reposurgeon 2.0
> > anyway; I'm still chasing some bugs in the svn support.
> 
> Haven't looked at reposurgeon yet, but if that tool can make it easier
> the better :-)

reposurgeon is a repository surgery tool; one of its main uses is for
high-quality repository conversions.  See http://catb.org/~esr/reposurgeon/

The 2.x version will have direct support for reading (though not
writing) Subversion repos.  Already, even with its known bugs, it's
the best svn-to-git converter in existence - because it's the only one
that can handle mixed-branch commits.

> > However, I think we should probably begin preparing for a move.  Gna
> > is understaffed and undermaintained, and the Gna codebase is creaking
> > and old. I would feel much more secure if the repo were migrated to
> > somewhere like github or gitorious.
(Continue reading)


Gmane