Bug #95755 for Git-FastExport: Bug in git-stitch-repo?

Sun May 18 09:46:17 2014 newsScott [...] gmx.de - Ticket created

Subject:	Bug in git-stitch-repo?
Date:	Sun, 18 May 2014 15:46:05 +0200
To:	bug-Git-FastExport [...] rt.cpan.org
From:	Henning Moll <newsScott [...] gmx.de>

After doing a $ git-stitch-repo ../A:A ../B:B | git fast-import I would expect only a A directory in the root after git checkout master-A and only a B directory in the root after git checkout master-B But there is also a A directory after checkout master-B. (Sometimes it's vice versa). Here’s how to reproduce mkdir A; cd A; git init; echo "a" > x; git add x; git commit -m 'x added'; cd .. mkdir B; cd B; git init; echo "b" > x; git add x; git commit -m 'x added'; cd .. mkdir R; cd R; git init; git-stitch-repo ../A:A ../B:B | git fast-import; cd .. cd R; git checkout master-A; ls -1 | grep B; cd .. cd R; git checkout master-B; ls -1 | grep A; cd .. None of the grep commands should report anything, but one does… Note that there is also content below the 'false' directory... git: 1.7.9.5 perl: v5.14.2 git-stitch-repo version 0.104 mkdir A; cd A; git init; echo "a" > x; git add x; git commit -m 'x added'; cd .. mkdir B; cd B; git init; echo "b" > x; git add x; git commit -m 'x added'; cd .. mkdir R; cd R; git init; git-stitch-repo ../A:A ../B:B | git fast-import; cd .. cd R; git checkout master-A; ls -1 | grep B; cd .. cd R; git checkout master-B; ls -1 | grep A; cd .. Best regards Henning

Sun May 18 18:30:24 2014 book [...] cpan.org - Correspondence added

On Sun May 18 09:46:17 2014, newsScott@gmx.de wrote: Show quoted text

> After doing a > $ git-stitch-repo ../A:A ../B:B | git fast-import > I would expect > only a A directory in the root after git checkout master-A and > only a B directory in the root after git checkout master-B

That would be what you'd get if you added B as a remote in A, and then merged the two head commits. Before the merge, you'd have two disconnected histories, with one directory each. Show quoted text

> But there is also a A directory after checkout master-B.

Actually, this is exactly what should happen. The whole point of git-stich-repo is to merge the history of both repositories as if they had lived in neighbouring directories all the time. Every time there's a commit in A, only the files in the A directory are changed. And it is the same for B. There as to be a first commit, though, which will contain files from the repository in which the first commit happened (the available commits are sorted by date). So the first commit in the new history will contains commits from the first repository, because the other repository "doesn't exist" yet. Show quoted text

> (Sometimes it's vice versa).

Given that the granularity of git is one second, in your example, both repositories have their first commit happen at the same time (or one second apart). This probably explains why you sometimes see the reverse. Maybe the example in the documentation can make it clearer: https://metacpan.org/pod/Git::FastExport::Stitch#Example In all the four resulting repositories, commit A1 only contains the A directory, while B1 will contain whatever A1 added to A/, in addition to the content that B1 added in B/. Best regards, -- BooK

Sun May 18 18:30:24 2014 The RT System itself - Status changed from 'new' to 'open'

Sun May 18 22:34:25 2014 newsScott [...] gmx.de - Correspondence added

Subject:	Re: [rt.cpan.org #95755] Bug in git-stitch-repo?
Date:	Mon, 19 May 2014 04:34:14 +0200
To:	bug-Git-FastExport [...] rt.cpan.org
From:	Henning Moll <newsScott [...] gmx.de>

Am 19.05.2014 00:30, schrieb Philippe Bruhat (BooK) via RT: Show quoted text

> Maybe the example in the documentation can make it clearer: > https://metacpan.org/pod/Git::FastExport::Stitch#Example In all the > four resulting repositories, commit A1 only contains the A directory, > while B1 will contain whatever A1 added to A/, in addition to the > content that B1 added in B/.

Thank you very much. Now i am a bit more clear what git-stitch-repo is doing. ;-) But there's still something that puzzles me: In your example, it is obvious that both original masters (A6, B8) are on "the same line" in the final result. Or in other words: A6 is the "common" master in the new repository. With my tests i unfortunately observe a different behaviour (And as the history of the participating repositories is huge, i can't give a small example). So here is what i observe. Given A1 - A2 - A3 - A4 - master B1 - B2 - B3 - B4- master masters point to A4 resp. B4. There are additional branches, but none of them is merged back into master. The stitched repository looks like this: / - - - A3 - A4 / A1 - B1 - A2 - B2 - B3 - B4 What could be the reason, that it is not A1 - B1 - A2 - B2 - A3 - B3 - A4 - B4 You mentioned the requirement of a "linearhistory". What do you mean by that? Best regards Henning

Mon May 19 04:31:02 2014 book [...] cpan.org - Correspondence added

On Sun May 18 22:34:25 2014, newsScott@gmx.de wrote: Show quoted text

> > Thank you very much. Now i am a bit more clear what git-stitch-repo is > doing. ;-) > But there's still something that puzzles me: > > With my tests i unfortunately observe a different behaviour (And as the > history of the participating repositories is huge, i can't give a small > example). So here is what i observe. Given > > A1 - A2 - A3 - A4 - master > > B1 - B2 - B3 - B4- master > > masters point to A4 resp. B4. There are additional branches, but none of > them is merged back into master. > The stitched repository looks like this: > > / - - - A3 - A4 > / > A1 - B1 - A2 - B2 - B3 - B4 > > What could be the reason, that it is not > > A1 - B1 - A2 - B2 - A3 - B3 - A4 - B4 > > You mentioned the requirement of a "linearhistory". What do you mean by > that? >

Alas, I'm afraid you've encountered the issue discussed in another bug: https://rt.cpan.org/Ticket/Display.html?id=70695 which is a real issue. Basically, git-stitch-repo processes commits one at a time, and must decide where to attach them in the new history that it is building. Because it's processing the output of git-fast-export, it has very little data to work with. If repo A branches off (say between master and branch-A), git-stitch repo could mistakenly attach the next commit on the B master onto branch-A, because at this point they are both perfectly valid locations to attach this commit (based on the constraints listed in https://metacpan.org/pod/Git::FastExport::Stitch#Constraints-of-the-stitching-algorithm ). Unfortunately, at this point, master-A and master-B have effectively branched, and the constraints prevent them to ever be joined again. There are ways to be smarter with the attachement algorithm, at a heavy startup cost (proportional to the total number of commits).I'm still experimenting with it, and haven't a proper fix yet. This kind of issue doesn't show up with linear repositories (i.e. without branches). When I wrote git-stich-repo, the use case was for merging conversions of cvs or svn repositories, which are mostly linear. Now that git is widely used and understood, branches are much more likely to happen, and this issues shows up more. -- BooK

Mon May 19 06:14:36 2014 newsScott [...] gmx.de - Correspondence added

Subject:	Re: [rt.cpan.org #95755] Bug in git-stitch-repo?
Date:	Mon, 19 May 2014 12:14:26 +0200
To:	bug-Git-FastExport [...] rt.cpan.org
From:	Henning Moll <newsScott [...] gmx.de>

Thank you very much for your detailed answer! Best regards Henning