headers
Teemu Ikonen | 3 Apr 10:58

Branches to patches

Stefano Zacchiroli wrote:
> However in general it is not possible to
> require that all patches apply properly to pristine source. In some
> (rare fortunately) cases two patches will conflict with each other on
> the pristine source and you need to make one depend on the other or the
> other way round.

I recently started to use git in some of my Debian packages and
thought about this issue a bit.

When maintaining packages with a version control system, having each
"feature" in a separate branch (branched from the upstream version) is
clearly superior to a stack of patches. You retain separate history
for every branch and can easily develop features independently,
without having to worry about conflicts.

When making a release, all the feature branches are merged to an
integration branch. The end product of the packaging should be this
well-integrated source, but it would be nice to also have the separate
features easily extractable from the source package as well. Usually
this is achieved by having the package contain a quilt patch series or
similar. Converting a set of branches to a stack of patches is easy if
there are no conflicts, but of course this is not always the case.

A brute force method of making a patch stack which has all the feature
patches applicable to pristine source and ends up with the complete
integrated source would be to apply feature patch #1, revert it with
the same patch reversed, apply feature patch #2, revert it, etc. and
finally to apply a patch containing all the features, plus additional
integration changes. The patch stack would then contain every change
(Continue reading)

Permalink | Reply |
headers
James Westby | 3 Apr 12:38

Re: Branches to patches

On Thu, 2008-04-03 at 10:58 +0200, Teemu Ikonen wrote:
> Stefano Zacchiroli wrote:
> > However in general it is not possible to
> > require that all patches apply properly to pristine source. In some
> > (rare fortunately) cases two patches will conflict with each other on
> > the pristine source and you need to make one depend on the other or the
> > other way round.
> I've made a python program (actually a rather thin wrapper around git
> and various tools from patchutils) which makes this transformation
> from an upstream branch, a set of feature branches and an integration
> branch stored in git to a series of patch files. It's called
> git-genpatches and is attached to this mail. The revert patches
> between feature patches are made by python program minrevert (also
> attached) which takes patches A and B and outputs a patch reverting
> those hunks in patch A which overlap with hunks in patch B.

Hi,

This is interesting, thanks for sharing it.

I made a simple test case locally, and it doesn't quite seem to do what
I expect.

The quite series ends up looking like

  first patch
  remove the change in first patch that will conflict with a change in
     the second patch
  second patch
  revert all the changes made by second patch and the remaining changes
(Continue reading)

Permalink | Reply |
headers
Teemu Ikonen | 3 Apr 13:06

Re: Branches to patches

On Thu, Apr 3, 2008 at 12:38 PM, James Westby <jw+debian@...> wrote:
> On Thu, 2008-04-03 at 10:58 +0200, Teemu Ikonen wrote:
> > I've made a python program (actually a rather thin wrapper around git
>  > and various tools from patchutils) which makes this transformation
>  > from an upstream branch, a set of feature branches and an integration
>  > branch stored in git to a series of patch files. It's called
>  > git-genpatches and is attached to this mail. The revert patches
>  > between feature patches are made by python program minrevert (also
>  > attached) which takes patches A and B and outputs a patch reverting
>  > those hunks in patch A which overlap with hunks in patch B.
>  I made a simple test case locally, and it doesn't quite seem to do what
>  I expect.

Well, as I wrote, the code is more like a proof of concept :)

>  The quite series ends up looking like
>
>   first patch
>   remove the change in first patch that will conflict with a change in
>      the second patch
>   second patch
>   revert all the changes made by second patch and the remaining changes
>      made by the first patch.
>
>  It seems like the logic is just slightly wrong, or I may be using it
>  wrong, but it means that a "quilt push -a" just leaves me with the
>  original upstream file.

I fixed some bugs yesterday after not looking at the code for a while,
so I probably introduced more bugs than I fixed. I don't have access
(Continue reading)

Permalink | Reply |
headers
James Westby | 3 Apr 13:14

Re: Branches to patches

On Thu, 2008-04-03 at 13:06 +0200, Teemu Ikonen wrote:
> I fixed some bugs yesterday after not looking at the code for a while,
> so I probably introduced more bugs than I fixed. I don't have access
> to the code ATM, but I'll have a look at it in the evening. If there's
> interest I'll also publish the git repo for this somewhere.
> 
> And, if you have a publicly accessible repository I could test the
> code against, that would be great. Making artificial test cases for
> this is both tedious, and apparently does not catch all mistakes.

Oh, this was the simplest artificial test case that I could think
of. "upstream" is a file with the numbers 1 to 30, one per line.
Patch 1 adds 1.5 and 15.5, patch 2 add 29.5 and 14.5, so each
there are two hunks that conflict and two that don't.

Thanks,

James
Permalink | Reply |
headers
John Goerzen | 3 Apr 15:25

Re: Branches to patches

On Thu April 3 2008 3:58:47 am Teemu Ikonen wrote:

> When maintaining packages with a version control system, having each
> "feature" in a separate branch (branched from the upstream version) is
> clearly superior to a stack of patches. You retain separate history
> for every branch and can easily develop features independently,
> without having to worry about conflicts.
>
> When making a release, all the feature branches are merged to an
> integration branch. The end product of the packaging should be this
> well-integrated source, but it would be nice to also have the separate

My question here is: how do you keep the feature branches up-to-date with 
upstream?  Are you rebasing or merging?

I have a couple of simple scripts that will do a git merge in each feature 
branch, and then another one that will merge each feature branch to master.  
This has been working well for me.

The problem with rebasing, of course, is that it makes it difficult for 
others to work on the project with you.

-- John
Permalink | Reply |
headers
Russ Allbery | 4 Apr 02:54
Favicon

Re: Branches to patches

John Goerzen <jgoerzen <at> complete.org> writes:

> My question here is: how do you keep the feature branches up-to-date
> with upstream?  Are you rebasing or merging?
>
> I have a couple of simple scripts that will do a git merge in each
> feature branch, and then another one that will merge each feature branch
> to master.  This has been working well for me.
>
> The problem with rebasing, of course, is that it makes it difficult for 
> others to work on the project with you.

Out of curiosity (and this is one of the places where I'm just a Git
newbie), why would I ever want to rebase and not merge?  I'm currently
merging and it works great.  I'm not sure what the advantage of rebasing
instead would be.

--

-- 
Russ Allbery (rra <at> stanford.edu)             <http://www.eyrie.org/~eagle/>
Permalink | Reply |
headers
Yaroslav Halchenko | 4 Apr 03:15
Gravatar

Re: Branches to patches


> Out of curiosity (and this is one of the places where I'm just a Git
> newbie), why would I ever want to rebase and not merge?  I'm currently
> merging and it works great.  I'm not sure what the advantage of
> rebasing instead would be.
there are many use cases (there are some in git rebase --help), where
rebase is the only option (like moving branch X which is based on Y, to
be based on some other branch Z -- for instance feature X is apparently
not to be distributed in public domain, thus should be based on branch Y
which is for 'private use only', or vise versa -- and for that branch
has to be rebased, and if smth was merged into it from X, then
rebasing onto Y would bring X into it).

But in this case for feature branches imho a branch without multiple
merges from upstream branch looks much cleaner and it is easy to rebase
it if needed against any ref in repository (e.g. you need to apply that
branch to test if it solves issues in the maintained version of package
within etch) -- another option would be cherry picking but rebasing
would be cleaner imho and easier.

Am I missing some other critical use cases?

--

-- 
Yaroslav Halchenko
Research Assistant, Psychology Department, Rutgers-Newark
Student  Ph.D. @ CS Dept. NJIT
Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
        101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
WWW:     http://www.linkedin.com/in/yarik
(Continue reading)

Permalink | Reply |
headers
martin f krafft | 4 Apr 10:53
Favicon

Re: Branches to patches

also sprach Yaroslav Halchenko <debian <at> onerussian.com> [2008.04.04.0315 +0200]:
> But in this case for feature branches imho a branch without multiple
> merges from upstream branch looks much cleaner and it is easy to rebase
> it if needed against any ref in repository (e.g. you need to apply that
> branch to test if it solves issues in the maintained version of package
> within etch) -- another option would be cherry picking but rebasing
> would be cleaner imho and easier.

Both rebasing and cherry-picking don't work well with published
repos because they change commit IDs. For everything you publish,
just stick to merging. This is my suggestion and your mileage may,
of course, vary.

Cherry-picking is great for pulling individual commits from
upstream.

Rebasing is great to keep the stuff you work on up-to-date with what
others are doing, and to minimise diffs between upstream and your
working, throwaway branch. It is *not* suitable for cooperation.

--

-- 
 .''`.   martin f. krafft <madduck <at> debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

"nothing can cure the soul but the senses,
 just as nothing can cure the senses but the soul."
                                                        -- oscar wilde
(Continue reading)

Permalink | Reply |
headers
Yaroslav Halchenko | 4 Apr 15:47
Gravatar

Re: Branches to patches


> Both rebasing and cherry-picking don't work well with published
> repos because they change commit IDs.
I don't see why cherry-picking is an issue -- it just replicates commit
(patch) into current branch from another branch. It is quite different
from rebasing where commits are moved, thus ref for that branch is
changed in non-fast-forward fashion

> Cherry-picking is great for pulling individual commits from
> upstream.
or introducing individual commits back into upstream (if you are
upstream and want to cherry pick individual fixes from some feature
branches of debian-fixes branches)

> Rebasing is great to keep the stuff you work on up-to-date with what
> others are doing, and to minimise diffs between upstream and your
> working
imho diff should (must) stay the same if you are diffing head of
upstream (or the latest split point, ie the point where upstream was
either merged from last time, or rebased against) and the feature
branch regardless if you kept feature branch under constant rebasing or
merging from upstream.

--

-- 
Yaroslav Halchenko
Research Assistant, Psychology Department, Rutgers-Newark
Student  Ph.D. @ CS Dept. NJIT
Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
        101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
WWW:     http://www.linkedin.com/in/yarik
(Continue reading)

Permalink | Reply |
headers
martin f krafft | 3 Apr 16:15
Favicon

Re: Branches to patches

also sprach John Goerzen <jgoerzen@...> [2008.04.03.1525 +0200]:
> The problem with rebasing, of course, is that it makes it
> difficult for others to work on the project with you.

Don't rebase published branches. It can't be said enough.
Merging is fine. Messy histories are what make true hackers. :)

--

-- 
 .''`.   martin f. krafft <madduck@...>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

"once ... in the wilds of afghanistan, i lost my corkscrew, and we
 were forced to live on nothing but food and water for days."
                                -- w. c. fields, "my little chickadee"

_______________________________________________
vcs-pkg-discuss mailing list
vcs-pkg-discuss@...
http://lists.alioth.debian.org/mailman/listinfo/vcs-pkg-discuss
Permalink | Reply |
headers
martin f krafft | 21 Aug 16:31
Favicon

Re: Branches to patches

also sprach Teemu Ikonen <tpikonen <at> gmail.com> [2008.04.03.1058 +0200]:
> I've made a python program (actually a rather thin wrapper around git
> and various tools from patchutils) which makes this transformation
> from an upstream branch, a set of feature branches and an integration
> branch stored in git to a series of patch files. It's called
> git-genpatches and is attached to this mail. The revert patches
> between feature patches are made by python program minrevert (also
> attached) which takes patches A and B and outputs a patch reverting
> those hunks in patch A which overlap with hunks in patch B.

You will want to check out topgit...

--

-- 
 .''`.   martin f. krafft <madduck <at> debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

"men always want to be a woman's first love.
 women have a more subtle instinct:
 what they like is to be a man's last romance."
                                                        -- oscar wilde

also sprach Teemu Ikonen <tpikonen <at> gmail.com> [2008.04.03.1058 +0200]:
> I've made a python program (actually a rather thin wrapper around git
> and various tools from patchutils) which makes this transformation
> from an upstream branch, a set of feature branches and an integration
> branch stored in git to a series of patch files. It's called
> git-genpatches and is attached to this mail. The revert patches
(Continue reading)

Permalink | Reply |
headers
Teemu Ikonen | 21 Aug 17:10

Re: Branches to patches

On Thu, Aug 21, 2008 at 4:31 PM, martin f krafft <madduck@...> wrote:
> also sprach Teemu Ikonen <tpikonen@...> [2008.04.03.1058 +0200]:
>> I've made a python program (actually a rather thin wrapper around git
>> and various tools from patchutils) which makes this transformation
>> from an upstream branch, a set of feature branches and an integration
>> branch stored in git to a series of patch files. It's called
>> git-genpatches and is attached to this mail. The revert patches
>> between feature patches are made by python program minrevert (also
>> attached) which takes patches A and B and outputs a patch reverting
>> those hunks in patch A which overlap with hunks in patch B.
>
> You will want to check out topgit...

I already have, but TopGit does not really solve the same problem,
namely serializing a set of possibly conflicting heads with the end
result being identical to the state of an integration branch. At least
it does not do this automatically without the need to create extra
merged bases.

TopGit is nice though, but it still needs some work. A 'tg clone'
command which would pull also the bases automatically would be nice to
have. Also, is there a trick to get the "ours" merge strategy used in
.top* -files to work with octopus merge?

Teemu
Permalink | Reply |
headers
martin f krafft | 27 Aug 15:42

Re: Branches to patches

also sprach Teemu Ikonen <tpikonen@...> [2008.08.21.1610 +0100]:
> I already have, but TopGit does not really solve the same problem,
> namely serializing a set of possibly conflicting heads with the end
> result being identical to the state of an integration branch. At least
> it does not do this automatically without the need to create extra
> merged bases.

But it does so automatically, and I have been talking to Petr about
adding support for temporary merge bases in case you only want
one-time integration of a set of branches (as for distro packaging,
for instance).

> TopGit is nice though, but it still needs some work. A 'tg clone'
> command which would pull also the bases automatically would be
> nice to have.

Petr is aware of this and working on it, but it seems like he's
busy. It should be fairly trivial to add, if you want to have
a shot.

> Also, is there a trick to get the "ours" merge strategy used in
> .top* -files to work with octopus merge?

It is my understanding that the ours strategy works with any number
of heads, so it should work fine with the octopus merge. This is
what the git-merge(1) manpage has to say about it:

  This resolves any number of heads, but the result of the merge is
  always the current branch head.
(Continue reading)

Permalink | Reply |
headers
Teemu Ikonen | 27 Aug 21:09

Re: Branches to patches

On Wed, Aug 27, 2008 at 3:42 PM, martin f krafft <madduck@...> wrote:
> also sprach Teemu Ikonen <tpikonen@...> [2008.08.21.1610 +0100]:
>> I already have, but TopGit does not really solve the same problem,
>> namely serializing a set of possibly conflicting heads with the end
>> result being identical to the state of an integration branch. At least
>> it does not do this automatically without the need to create extra
>> merged bases.
>
> But it does so automatically, and I have been talking to Petr about
> adding support for temporary merge bases in case you only want
> one-time integration of a set of branches (as for distro packaging,
> for instance).

Ok, after reading the documentation again this seems to be the case.

So my integration (or debian) branch should be a TopGit branch which
depends on all the applied feature branches + upstream, and I should
do the conflict resolution when the base for it is merged?

>> Also, is there a trick to get the "ours" merge strategy used in
>> .top* -files to work with octopus merge?
>
> It is my understanding that the ours strategy works with any number
> of heads, so it should work fine with the octopus merge. This is
> what the git-merge(1) manpage has to say about it:
>
>  This resolves any number of heads, but the result of the merge is
>  always the current branch head.

When I'm trying to an octopus merge, git fails on the .top* files.
(Continue reading)

Permalink | Reply |
headers
martin f krafft | 28 Aug 00:12
Favicon

Re: Branches to patches

also sprach Teemu Ikonen <tpikonen@...> [2008.08.27.2009 +0100]:
> So my integration (or debian) branch should be a TopGit branch
> which depends on all the applied feature branches + upstream, and
> I should do the conflict resolution when the base for it is
> merged?

Yes.

--

-- 
 .''`.   martin f. krafft <madduck@...>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

officer, arrest that man! he's whistling a copyrighted song.

_______________________________________________
vcs-pkg-discuss mailing list
vcs-pkg-discuss@...
http://lists.alioth.debian.org/mailman/listinfo/vcs-pkg-discuss
Permalink | Reply |

Gmane