Re: [boost] Git Modularization Review no vote heads-up

newer
[Atomic] Rationale for preventing...

older
Re: [boost] [gsoc] Getting started...

"Jürgen Hunold"

23 May 2013 23 May '13

7:04 a.m.

Hi Dave,

...

Gesendet: Donnerstag, 23. Mai 2013 um 07:28 Uhr Von: "Dave Abrahams" An: boost@lists.boost.org Betreff: Re: [boost] Git Modularization Review no vote heads-up

on Wed May 22 2013, Jürgen Hunold wrote:

...
But this can easily be done after the conversion.

I think we already lost too much time with failed rewrite attempts, so let us get a working repository first. Then Volodya can test git mv v2/* . and report results. Afterwards, we can always try rewriting in a separate clone.

Done this myself yesterday. Works like a charm, git blame shows undisrupted history after git commit shows "rename" with 100% on each file moved.

...

Rewriting _published_ history is most strongly discouraged by the Git people, for several good reasons. *If* there is to be any rearrangement, it should happen before the switchover, so it doesn't bork people who are doing work based on the history originally published.

Yes, the basic rule of git.

...

However, I am loath to do any rearrangement that doesn't (reasonably) faithfully reflect how things were set up in the past. Otherwise, someone will check out an old state of the super-module and find that things have the wrong path relationships. Of course, path relationships will not match SVN anyway (because we don't have a modular layout in SVN), but people on this list made it quite clear that modularizing history was important to them, so I presume they want the Git history to reflect reality with maximal fidelity.

Correct.

...

That said, if the consensus is that things should be rearranged in the build repository, we can do that. We just need clear and explicit instructions that cover what to do with *all* the paths that have been used in Boost.Build (including branches and tags) throughout time. As you can see from https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261 and following lines, just preserving the existing structure was nontrivial.

That one starts with repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" : "build"; "tools/jam/" : "jam"; } right? So, this now creates "tools/build/build" with the contents of "tools/build" and "tools/build/jam" with the contents of "tools/jam" which is long gone. The former is wrong, as it does not preserve the current structure despite the thing you wrote above. I think that a one-to-one mapping like: repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; } is the way to go now. If something goes wrong, please put "tools/jam/" somewhere else. Btw. would repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; } be the right way to move v2 one directory up? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! juergen.hunold@gmx.eu * voice: 0049 4257 300 ! Fährstraße 1 * fax : 0049 4257 300 ! 31609 Balge/Sebbenhausen * mobil: 0049 178 186 1566 ! Germany

Show replies by date

Dave Abrahams

23 May 23 May

1:44 p.m.

New subject: Git Modularization Review no vote heads-up

on Thu May 23 2013, "Jürgen Hunold" wrote:

...

Dave wrote:

...
if the consensus is that things should be rearranged in the build repository, we can do that. We just need clear and explicit instructions that cover what to do with *all* the paths that have been used in Boost.Build (including branches and tags) throughout time. As you can see from https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261 and following lines, just preserving the existing structure was nontrivial.

That one starts with

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" : "build"; "tools/jam/" : "jam"; }

right?

Yes, but note that there are five or so "repository" declarations for the build repo.

...

So, this now creates "tools/build/build" with the contents of "tools/build" and "tools/build/jam" with the contents of "tools/jam" which is long gone. The former is wrong, as it does not preserve the current structure despite the thing you wrote above.

It preserves *a fragment of* the current structure *within the build repo*. That's presumably the important one. I expect the submodules themselves to have a future as independent repositories, but not necessarily as submodules.

...

I think that a one-to-one mapping like:

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; }

is the way to go now. If something goes wrong,

It will, because we'd no longer be catching tools/jam throughout history. That will cause modularization to fail.

...

please put "tools/jam/" somewhere else.

Yeah, that's not specific enough. What is "somewhere else?" We could put it in the "engine" /v2/engine/ subdirectory of the build repository, but that would be utterly inaccurate for older history. We can also have it move into /v2/engine from somewhere else (as it actually does in SVN), but again, what is "somewhere else?" ...and—just checking—do you speak for the Boost.Build project, or is Volodya likely to countermand your requests?

...

Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

...

Yours,

Jürgen

-- Dave Abrahams

Rene Rivera

24 May 24 May

5 a.m.

New subject: Git Modularization Review no vote heads-up

On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams wrote:

...

on Thu May 23 2013, "Jürgen Hunold" wrote:

...
Dave wrote:

I think that a one-to-one mapping like:

...
repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; }

is the way to go now. If something goes wrong,

It will, because we'd no longer be catching tools/jam throughout history. That will cause modularization to fail.

...
please put "tools/jam/" somewhere else.

Yeah, that's not specific enough. What is "somewhere else?"

branch = historical path = "jam/" We could put it in the "engine" /v2/engine/ subdirectory of the build

...

repository, but that would be utterly inaccurate for older history. We can also have it move into /v2/engine from somewhere else (as it actually does in SVN), but again, what is "somewhere else?"

...and—just checking—do you speak for the Boost.Build project, or is Volodya likely to countermand your requests?

...
Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

Anything before v2 existed should go to: branch = historical path = "build/" Or more descriptive.. 1. Put anything that ever existed in the current build/v2/* files at the root of the new build repo. 2. Put anything else in some form of "historical" branch using the path from "boost-root/" as the path in the new repo. (I don't care that much about the actual branch names other than to tell that they are there only as history). But since I don't know git sufficiently.. I don't know if that's practically possible. Specifically I don't know if one can follow history back across branches. if it's not possible then I would say change #2 above to: 2. Put anything else at historical/jam, and historical/build in whatever branches you have now. I'll let Volodya and Jürgen comment and concur as they see fit :-) -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Dave Abrahams

5:33 a.m.

New subject: Git Modularization Review no vote heads-up

on Thu May 23 2013, Rene Rivera wrote:

...

On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams wrote:

...
on Thu May 23 2013, "Jürgen Hunold" wrote:

...
Dave wrote:

I think that a one-to-one mapping like:

...
repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; }

is the way to go now. If something goes wrong,

It will, because we'd no longer be catching tools/jam throughout history. That will cause modularization to fail.

...
please put "tools/jam/" somewhere else.

Yeah, that's not specific enough. What is "somewhere else?"

branch = historical path = "jam/"

AFAICT, that doesn't work. tools/jam (or its equivalent) has appeared in many different SVN branches, including trunk. You can't just squash them all into one branch and come out with anything meaningful.

...

...
We could put it in the "engine" /v2/engine/ subdirectory of the build repository, but that would be utterly inaccurate for older history. We can also have it move into /v2/engine from somewhere else (as it actually does in SVN), but again, what is "somewhere else?"

...and—just checking—do you speak for the Boost.Build project, or is Volodya likely to countermand your requests?

...
Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

Anything before v2 existed should go to:

branch = historical path = "build/"

As before, you can't meaningfully squash everything into one branch.

...

Or more descriptive..

1. Put anything that ever existed in the current build/v2/* files at the root of the new build repo.

Easy enough

...

2. Put anything else in some form of "historical" branch using the path from "boost-root/"

? There's no "boost-root/" in SVN AFAIK

...

as the path in the new repo. (I don't care that much about the actual branch names other than to tell that they are there only as history).

But since I don't know git sufficiently.. I don't know if that's practically possible. Specifically I don't know if one can follow history back across branches.

Branches in Git are merely (reference-counted) labels for commits, each of which is the root of a history DAG. When you merge branch A into branch B, you can follow history from B to the last commit on A when it was merged, and thence to all of that commit's ancestors. If no further commits are made to A, it looks like this: /--> A --> A~1 ... B --> B~1 --> B~2 ... B~N < \--> B~(N+1) --> ... If you mean something else by "follow history," I guess you'd better explain.

...

if it's not possible then I would say change #2 above to:

2. Put anything else at historical/jam, and historical/build in whatever branches you have now.

Are those (historical/jam and historical/build) supposed to be branch names or paths? -- Dave Abrahams

Rene Rivera

10:25 p.m.

New subject: Git Modularization Review no vote heads-up

On Fri, May 24, 2013 at 12:33 AM, Dave Abrahams wrote:

...

on Thu May 23 2013, Rene Rivera wrote:

...
On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams wrote:

...
on Thu May 23 2013, "Jürgen Hunold" wrote:

Or more descriptive..

1. Put anything that ever existed in the current build/v2/* files at the root of the new build repo.

Easy enough

...
2. Put anything else in some form of "historical" branch using the path from "boost-root/"

? There's no "boost-root/" in SVN AFAIK

I did not mean it literally.. I was trying to refer to the root of the boost tree in svn (of which there are various). I.e. to just use the subpath from the boost root (for example tools/jam, or tools/build).

...

...
as the path in the new repo. (I don't care that much about the actual branch names other than to tell that they are there only as history).

But since I don't know git sufficiently.. I don't know if that's practically possible. Specifically I don't know if one can follow history back across branches.

Branches in Git are merely (reference-counted) labels for commits, each of which is the root of a history DAG.

When you merge branch A into branch B, you can follow history from B to the last commit on A when it was merged, and thence to all of that commit's ancestors. If no further commits are made to A, it looks like this:

/--> A --> A~1 ... B --> B~1 --> B~2 ... B~N < \--> B~(N+1) --> ...

If you mean something else by "follow history," I guess you'd better explain.

I mean whatever command I would use to do a diff between any two arbitrary versions of a single file. I.e. I only care about seeing *all* the diffs for say the "jam.h" file regardless of what branch the history happened in. In svn terms it would be following the history across copies (which is what the branches are in svn). Hence I would see all the history here < https://github.com/boostorg/build/commits/master/build/v2/engine/jam.h> event if the commit did not occur in the master branch (because the file didn't exist in the branch and was copied from another branch). Which I guess means that.. Yes that's what I mean by history :-)

...

if it's not possible then I would say change #2 above to:

...
2. Put anything else at historical/jam, and historical/build in whatever branches you have now.

Are those (historical/jam and historical/build) supposed to be branch names or paths?

Paths. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Daniel Pfeifer

25 May 25 May

12:35 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/25 Rene Rivera

...

On Fri, May 24, 2013 at 12:33 AM, Dave Abrahams wrote:

...
on Thu May 23 2013, Rene Rivera wrote:

...
On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams wrote:

...
on Thu May 23 2013, "Jürgen Hunold" wrote:

Or more descriptive..

1. Put anything that ever existed in the current build/v2/* files at

the

...
...
root of the new build repo.

Easy enough

...
2. Put anything else in some form of "historical" branch using the path from "boost-root/"

? There's no "boost-root/" in SVN AFAIK

I did not mean it literally.. I was trying to refer to the root of the boost tree in svn (of which there are various). I.e. to just use the subpath from the boost root (for example tools/jam, or tools/build).

...
...
as the path in the new repo. (I don't care that much about the actual branch names other than to tell that they are there only as history).

But since I don't know git sufficiently.. I don't know if that's practically possible. Specifically I don't know if one can follow

history

...
...
back across branches.

Branches in Git are merely (reference-counted) labels for commits, each of which is the root of a history DAG.

When you merge branch A into branch B, you can follow history from B to the last commit on A when it was merged, and thence to all of that commit's ancestors. If no further commits are made to A, it looks like this:

/--> A --> A~1 ... B --> B~1 --> B~2 ... B~N < \--> B~(N+1) --> ...

If you mean something else by "follow history," I guess you'd better explain.

I mean whatever command I would use to do a diff between any two arbitrary versions of a single file. I.e. I only care about seeing *all* the diffs for say the "jam.h" file regardless of what branch the history happened in. In svn terms it would be following the history across copies (which is what the branches are in svn). Hence I would see all the history here < https://github.com/boostorg/build/commits/master/build/v2/engine/jam.h> event if the commit did not occur in the master branch (because the file didn't exist in the branch and was copied from another branch).

Which I guess means that.. Yes that's what I mean by history :-)

...
if it's not possible then I would say change #2 above to:

...
2. Put anything else at historical/jam, and historical/build in

whatever

...
branches you have now.

Are those (historical/jam and historical/build) supposed to be branch names or paths?

Paths.

Jürgen, Volodya, all, Would Rene's approach be acceptable? - Send tools/build/v2/* to the top level of the build repository. - Send the rest of tools/build to historic/build in the build repo. - Send tools/jam to historic/jam in the build repo. Pro: - The change is quick and easy. - I did it, so you can review it. Contra: - It does not reflect the history. You might not succeed to check out and build a very old version of boost. cheers, Daniel

Jürgen Hunold

1:23 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Daniel, On Saturday, 25. May 2013 14:35:23 Daniel Pfeifer wrote:

...

Jürgen, Volodya, all,

Would Rene's approach be acceptable?

- Send tools/build/v2/* to the top level of the build repository. - Send the rest of tools/build to historic/build in the build repo. - Send tools/jam to historic/jam in the build repo.

This is hopefully what we all wanted. Yes. Perfect.

...

Pro: - The change is quick and easy. - I did it, so you can review it.

Thanks a lot. I've only tested building HEAD for now, but the layout seems to be correct. I'll poke around the history the next days. And try to get my patches applying again.

...

Contra: - It does not reflect the history. You might not succeed to check out and build a very old version of boost.

For really ancient version we should have the old repository in read-only mode somewhere. This is sufficient at least for me. The removal of "v2" requires some patching. I've attached to necessary patch to bootstrap.{sh,bat}. Can this be applied automatically right after the conversion so that the scripts are working on checkout? Otherwise the patch(es) should be added to the wiki page. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Jürgen Hunold

1:38 p.m.

New subject: Git Modularization Review no vote heads-up

Hi, On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote:

...

The removal of "v2" requires some patching. I've attached to necessary patch to bootstrap.{sh,bat}. Can this be applied automatically right after the conversion so that the scripts are working on checkout? Otherwise the patch(es) should be added to the wiki page.

Forgot to patch boost-build.jam. New patch attached. Next task is generating forward headers. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Daniel Pfeifer

9 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/25 Jürgen Hunold

...

Hi,

...
The removal of "v2" requires some patching. I've attached to necessary

On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote: patch

...
to bootstrap.{sh,bat}. Can this be applied automatically right after the conversion so that the scripts are working on checkout? Otherwise the patch(es) should be added to the wiki page.

Forgot to patch boost-build.jam. New patch attached. Next task is generating forward headers.

Hi Jürgen, This patch does not apply to the complete history. I added the following command instead: $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;" cheers, Daniel

Bjørn Roald

26 May 26 May

11:01 a.m.

New subject: Git Modularization Review no vote heads-up

On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:

...

2013/5/25 Jürgen Hunold

...
Hi,

...
The removal of "v2" requires some patching. I've attached to necessary

On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote: patch

...
to bootstrap.{sh,bat}. Can this be applied automatically right after the conversion so that the scripts are working on checkout? Otherwise the patch(es) should be added to the wiki page.

Forgot to patch boost-build.jam. New patch attached. Next task is generating forward headers.

Hi Jürgen,

This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Nice - composing with tools the unix way certainly has it's strengths. Given that the proposed changes to tools/build structure are made in git history with the conversion, I think it make a lot of sense to change these scripts in history as well, as Daniel suggest here. That said, I am not much fan of changing history when not strictly needed. I am not convinced all these changes in tools/build history is needed, but I am not making that call. For the record, here is another alternative: - Send history of tools/jam to historic/jam in the build repo. - After transition to git is complete, move tools/build/v2/* to the top level of the build repository and change top level scrips as a single git commit in build and boost. Pro: - Sending history of tools/jam to historic/jam to is quick and easy as part of conversion to modularized boost. - Reflect real history for build v1 and v2 Contra: - It does not reflect the history for jam. Jam is moved in history to get all into the build repository. This may cause problems following legacy instructions for building historic boost. However jam was normally built separately as a first manual step, so it may be feasible and sufficient to have simple wiki or other howto instructions on building historic boost from modularized git repositories. How to deal with the staging of header files for the build in the boost directory does also need to be described the same place if such descriptions are created or needed. -- Bjørn

Jürgen Hunold

1:15 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Bjørn, On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:

...

On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:

...
This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Nice - composing with tools the unix way certainly has it's strengths.

Definetely ;-)

...

Given that the proposed changes to tools/build structure are made in git history with the conversion, I think it make a lot of sense to change these scripts in history as well, as Daniel suggest here.

Yes, I didn't realise this to be necessary.

...

That said, I am not much fan of changing history when not strictly needed. I am not convinced all these changes in tools/build history is needed, but I am not making that call.

For the record, here is another alternative:

- Send history of tools/jam to historic/jam in the build repo. - After transition to git is complete, move tools/build/v2/* to the top level of the build repository and change top level scrips as a single git commit in build and boost.

Pro: - Sending history of tools/jam to historic/jam to is quick and easy as part of conversion to modularized boost. - Reflect real history for build v1 and v2

Complete history is not needed.

...

Contra: - It does not reflect the history for jam.

And jam was moved into v2/engine several years ago.

...

Jam is moved in history to get all into the build repository. This may cause problems following legacy instructions for building historic boost.

I think that support for building "historic" versions from the modular repository was never the goal. You can always built them from read-only subversion.

...

However jam was normally built separately as a first manual step, so it may be feasible and sufficient to have simple wiki or other howto instructions on building historic boost from modularized git repositories. How to deal with the staging of header files for the build in the boost directory does also need to be described the same place if such descriptions are created or needed.

I even fear that building historic boost is from git is not even possible. The source tree rewriting needed for the modularisation itself is quite heavy. This is the main reason why Dave proposed dropping the history altogether at the first conversion attempt I think. The only "real" reason (at least for me) for keeping history is the ability to use "git diff" and "git blame" and follow edits and reading commit messages. Those are my personal opinions, but I guees Rene and Volodya are on the same track here. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Rene Rivera

3:15 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, May 26, 2013 at 8:15 AM, Jürgen Hunold wrote:

...

On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:

...
Contra: - It does not reflect the history for jam.

And jam was moved into v2/engine several years ago.

And it makes it harder to transition the testing infrastructure. Those are my personal opinions, but I guees Rene and Volodya are on the same

...

track here.

Yes, I am. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Daniel Pfeifer

7:25 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/26 Jürgen Hunold

...

Hi Bjørn,

On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:

...
On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:

...
This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Nice - composing with tools the unix way certainly has it's strengths.

Definetely ;-)

It is also pretty slow. Rewriting Boost's history that way takes about 4 hours. I changed the mapping accoring to Bjørn's suggestion. tools/build/v2 will end up in tools/build/v2. I think that makes most sense. cheers, Daniel

Bjørn Roald

7:42 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, 2013-05-26 at 21:25 +0200, Daniel Pfeifer wrote:

...

2013/5/26 Jürgen Hunold

...
Hi Bjørn,

On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:

...
On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:

...
This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Nice - composing with tools the unix way certainly has it's strengths.

Definetely ;-)

It is also pretty slow. Rewriting Boost's history that way takes about 4 hours.

I changed the mapping accoring to Bjørn's suggestion.

tools/build/v2 will end up in tools/build/v2. I think that makes most sense.

Err -- although I still support my suggestion, boost build maintainers seems to differ and they have more of a say here than I do and I willingly yield. I feel like I am just getting in the way of progress here. -- Bjørn

Rene Rivera

7:49 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, May 26, 2013 at 2:42 PM, Bjørn Roald wrote:

...

On Sun, 2013-05-26 at 21:25 +0200, Daniel Pfeifer wrote:

...
2013/5/26 Jürgen Hunold

...
Hi Bjørn,

On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:

...
On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:

...
This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Nice - composing with tools the unix way certainly has it's strengths.

Definetely ;-)

It is also pretty slow. Rewriting Boost's history that way takes about 4 hours.

I changed the mapping accoring to Bjørn's suggestion.

tools/build/v2 will end up in tools/build/v2. I think that makes most sense.

Err -- although I still support my suggestion, boost build maintainers seems to differ and they have more of a say here than I do and I willingly yield. I feel like I am just getting in the way of progress here.

I was about to mention something.. So thanks for the comment :-) But if life is such a pain with squashing the v2 subtree.. I guess the b2 maintainers will just have to squash the v2 at the source.. I.e. make the current Boost svn structure be tools/build/* moving all files and changing the scripts.. and especially painful adjusting the testing regression scripts. Which may impact the upcoming release schedules. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo

Bjørn Roald

8:39 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, 2013-05-26 at 14:49 -0500, Rene Rivera wrote:

...

I was about to mention something.. So thanks for the comment :-)

But if life is such a pain with squashing the v2 subtree.. I guess the b2 maintainers will just have to squash the v2 at the source.. I.e. make the current Boost svn structure be tools/build/* moving all files and changing the scripts.. and especially painful adjusting the testing regression scripts. Which may impact the upcoming release schedules.

I may be misunderstanding you, but it was _not_ my suggestion for you to change anything at all in svn. Nor do I think you need to move anything in git from tools/build/v2 to tools/build until you feel ready for it. E.g after next release. I am concerned we are have different approaches in mind and think we are talking of the same approach. Basically, what I have in mind is keeping everything that is in active use as-is. Then when you are ready to restructure, at some time convenient after the move to git, you simply do it in git. History is traced across a commit where you do git mv v2/* . I think, at least if done properly this should work fine, I tried it - had to move the duplicated css file out of the way first. To some degree git tracing file history across moves depend on the tool used for tracing history. There is not an explicit notion of a moved file in git, just derived from commit changes. So to some degree I think it may be safer to get the conversion to do it if the result is all tools/build/v2 files history are in tools/build. But I would have taken that risk and used git mv. It is up to you though. Off-cause you may do it in svn if you like, but then you are on a tight schedule as you point out, and I would be checking with Daniel if the conversion scripts can actually maintain your file history for the files moved. I would not suggest to fix history later in git after the boost repository commits are public. The last option, is to ignore fixing the history with $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;" all together, or just do it for the final conversion. We do not need it for getting all the other stuff right. And it is questionable how useful it is at all as special instructions or scripts are needed anyway to build historic boost in modularized boost. Maybe simply have a wiki page with a bootstrap-historic.sh script along the lines of: find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \; ... make-header-symlinks and other stuff ... ./bootstrap.sh -- Bjørn

...

--

Vladimir Prus

30 May 30 May

5:44 a.m.

New subject: Git Modularization Review no vote heads-up

On 26.05.2013 17:15, Jürgen Hunold wrote:

...

I even fear that building historic boost is from git is not even possible. The source tree rewriting needed for the modularisation itself is quite heavy. This is the main reason why Dave proposed dropping the history altogether at the first conversion attempt I think. The only "real" reason (at least for me) for keeping history is the ability to use "git diff" and "git blame" and follow edits and reading commit messages.

Those are my personal opinions, but I guees Rene and Volodya are on the same track here.

Yep. I primarily care that in the new git repository, I can go "git blame" and "git log" and "git show" without hassle. I *suspect* that "git mv" after conversion won't be quite as smooth, e.g. if I print a previous commit it will have "v2/" prefix in all paths, which will slightly complicate things. And practically speaking, the history is only important from the point where jam moved into build/v2/engine. Top-level scripts should not be a problem; we can code them to detect which layout is used (well, I can do that), and I don't think that building previous Boost releases from git repository is a use case were need to support -- even the structure of the checkout will be different, so checking out from git, rearranging, and then building is not 100% reproduction of past builds anyway. I.e. we probably need not bother with rewriting history of top-level scripts. So where are we now? Daniel previously said: - Send tools/build/v2/* to the top level of the build repository. - Send the rest of tools/build to historic/build in the build repo. - Send tools/jam to historic/jam in the build repo. and it looks like it was actually done? I don't know whether 'historic' is a branch or directory name, but it does not matter much. However, it looks like further tweak was made to disable this? Thanks, Volodya

Beman Dawes

2:40 p.m.

New subject: Git Modularization Review no vote heads-up

On Thu, May 30, 2013 at 1:44 AM, Vladimir Prus wrote:

...

... ... I don't think that building previous Boost releases from git repository is a use case were need to support -- even the structure of the checkout will be different, so checking out from git, rearranging, and then building is not 100% reproduction of past builds anyway.

Strongly agree. For those not comfortable with going back to the old svn repo for archeological research involving rebuilding an old release, there is always SourceForge. They've got past releases back to 1.20.0 (February 2003) available. --Beman

Jürgen Hunold

4:46 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Volodya On Thursday, 30. May 2013 09:44:00 Vladimir Prus wrote:

...

On 26.05.2013 17:15, Jürgen Hunold wrote:

...
I even fear that building historic boost is from git is not even possible. The source tree rewriting needed for the modularisation itself is quite heavy. This is the main reason why Dave proposed dropping the history altogether at the first conversion attempt I think. The only "real" reason (at least for me) for keeping history is the ability to use "git diff" and "git blame" and follow edits and reading commit messages.

Those are my personal opinions, but I guees Rene and Volodya are on the same track here.

Yep. I primarily care that in the new git repository, I can go "git blame" and "git log" and "git show" without hassle.

I am glad to hear that I am not too far off track here ;-)

...

I *suspect* that "git mv" after conversion won't be quite as smooth, e.g. if I print a previous commit it will have "v2/" prefix in all paths, which will slightly complicate things.

Yes, it will.

...

And practically speaking, the history is only important from the point where jam moved into build/v2/engine.

Good to hear.

...

Top-level scripts should not be a problem; we can code them to detect which layout is used (well, I can do that), and I don't think that building previous Boost releases from git repository is a use case were need to support -- even the structure of the checkout will be different, so checking out from git, rearranging, and then building is not 100% reproduction of past builds anyway. I.e. we probably need not bother with rewriting history of top-level scripts.

That what I meant by saying "...building historic boost is from git is not even possible."

...

So where are we now? Daniel previously said:

- Send tools/build/v2/* to the top level of the build repository. - Send the rest of tools/build to historic/build in the build repo. - Send tools/jam to historic/jam in the build repo.

and it looks like it was actually done?

Yes, there were at least two conversions with path rewriting.

...

I don't know whether 'historic' is a branch or directory name, but it does not matter much. However, it looks like further tweak was made to disable this?

Yes, it looks like running the regular expression sed over each commit blew up conversion time to 4 hours. That seems to be the current bottleneck. But I still cannot get my adapded patch from Steven for symlinking to work with the regression tests and at least 5 submodules are still missing. My 2 cents: Finalise the conversion and get the libraries etc. up and running. Then enable the v2/ removal again and test if everything is still working. Done. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Vladimir Prus

8:01 p.m.

New subject: Git Modularization Review no vote heads-up

On 30.05.2013 20:46, Jürgen Hunold wrote:

...

My 2 cents:

Finalise the conversion and get the libraries etc. up and running. Then enable the v2/ removal again and test if everything is still working. Done.

As longs as long parties agree to get back to v2/ removal before the final switch to git, I think that's very reasonable and practical suggestion. If we have neither content in place, or infrastructure updates ready, then details of history is of secondary importance.

Jürgen Hunold

26 May 26 May

1:01 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Daniel, On Saturday, 25. May 2013 23:00:48 Daniel Pfeifer wrote:

...

This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Okay, that "fixes" all the history. Now bootstrap.sh works out-of-the-box. But running b2 first time results in: ~/src/devel/modular-boost$ ./b2 -j4 -q /home/hunold/src/devel/modular-boost/tools/build/build/project.jam:287: in load-jamfile from module project error: Multiple Jamfiles found at '/home/hunold/src/devel/modular-boost' error: Filenames are: Jamroot Jamroot.jam

...

From the contents of Jamroot.jam I guess this comes from one the sandboxes:

git log Jamroot.jam commit 689e7d3d89c109a9ca1c59f0897071d0b35cf7ac Author: John Maddock Date: Mon Jun 21 08:36:14 2010 +0000 Add use-project rule to suppress some BB issues. svn path=/sandbox/Jamroot; revision=63182 Can you please remove this one from the toplevel repository? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Daniel Pfeifer

7:47 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/26 Jürgen Hunold

...

Hi Daniel,

On Saturday, 25. May 2013 23:00:48 Daniel Pfeifer wrote:

...
This patch does not apply to the complete history. I added the following command instead:

$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {} \;"

Okay, that "fixes" all the history. Now bootstrap.sh works out-of-the-box. But running b2 first time results in:

~/src/devel/modular-boost$ ./b2 -j4 -q /home/hunold/src/devel/modular-boost/tools/build/build/project.jam:287: in load-jamfile from module project error: Multiple Jamfiles found at '/home/hunold/src/devel/modular-boost' error: Filenames are: Jamroot Jamroot.jam

From the contents of Jamroot.jam I guess this comes from one the sandboxes:

git log Jamroot.jam commit 689e7d3d89c109a9ca1c59f0897071d0b35cf7ac Author: John Maddock Date: Mon Jun 21 08:36:14 2010 +0000

Add use-project rule to suppress some BB issues.

svn path=/sandbox/Jamroot; revision=63182

Can you please remove this one from the toplevel repository?

I see there are multiple files in the root directory that don't belong there. Investigating...

Jürgen Hunold

6:05 p.m.

New subject: Git Modularization Review no vote heads-up

Hi, On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:

...

Next task is generating forward headers.

I got the patches applying and Boost.Build getting started. Patches for Jamroot and tools/build repository attached. Caveats: - Dependency analysis is screwed, symlinks are created _after_ compilation. Just run compilation twice for the time being. Repository layout: Most targets fail with: fatal error: boost/detail/workaround.hpp: No such file or directory We have 7 workaround files after the conversion: libs/math/include/boost/math/tools/workaround.hpp libs/interprocess/include/boost/interprocess/detail/workaround.hpp libs/test/include/boost/test/detail/workaround.hpp libs/intrusive/include/boost/intrusive/detail/workaround.hpp libs/container/include/boost/container/detail/workaround.hpp libs/mpl/include/boost/mpl/aux_/config/workaround.hpp but the missing one from boost/details seems to belong to mpl. Any ideas? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Daniel Pfeifer

7:43 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/26 Jürgen Hunold

...

Hi,

On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:

...
Next task is generating forward headers.

I got the patches applying and Boost.Build getting started. Patches for Jamroot and tools/build repository attached.

Caveats: - Dependency analysis is screwed, symlinks are created _after_ compilation. Just run compilation twice for the time being.

Repository layout:

Most targets fail with:

fatal error: boost/detail/workaround.hpp: No such file or directory

We have 7 workaround files after the conversion:

libs/math/include/boost/math/tools/workaround.hpp libs/interprocess/include/boost/interprocess/detail/workaround.hpp libs/test/include/boost/test/detail/workaround.hpp libs/intrusive/include/boost/intrusive/detail/workaround.hpp libs/container/include/boost/container/detail/workaround.hpp libs/mpl/include/boost/mpl/aux_/config/workaround.hpp

but the missing one from boost/details seems to belong to mpl.

Any ideas?

Many files from "boost/detail" were sent to a repository "core" that was not mapped as a submodule. I renamed the repository to "detail" and mapped it to "libs/detail".

Beman Dawes

10:30 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, May 26, 2013 at 3:43 PM, Daniel Pfeifer wrote: ...>

...

Many files from "boost/detail" were sent to a repository "core" that was not mapped as a submodule. I renamed the repository to "detail" and mapped it to "libs/detail".

+1 That should be instantly recognized by those familiar with Boost. --Beman

Jürgen Hunold

27 May 27 May

2:52 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Daniel, On Sunday, 26. May 2013 21:43:32 Daniel Pfeifer wrote:

...

Many files from "boost/detail" were sent to a repository "core" that was not mapped as a submodule. I renamed the repository to "detail" and mapped it to "libs/detail".

Thanks. But now I get boost/array.hpp:52:41: fatal error: boost/functional/hash_fwd.hpp: No such file or directory It seems that somehow libs/functional is missing the "hash" subdirectory too. Another missing submodule? As Steven has already committed most of the symlink logic, here two new patches activating symlinks. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Daniel Pfeifer

3:38 p.m.

New subject: Git Modularization Review no vote heads-up

2013/5/27 Jürgen Hunold

...

Hi Daniel,

On Sunday, 26. May 2013 21:43:32 Daniel Pfeifer wrote:

...
Many files from "boost/detail" were sent to a repository "core" that was not mapped as a submodule. I renamed the repository to "detail" and mapped it to "libs/detail".

Thanks. But now I get

boost/array.hpp:52:41: fatal error: boost/functional/hash_fwd.hpp: No such file or directory

It seems that somehow libs/functional is missing the "hash" subdirectory too.

Another missing submodule?

Oh, "functional_hash" is not mapped as a submodule yet! That is a tricky one. I'd prefer the boost metarepository to be the only repository with submodules. But since "libs/functional" is a submodule, "libs/functional/hash" could only be a submodule of that one. Would it be acceptable to keep Boost.Functional/Hash in the repository of Boost.Functional? -- Daniel

Daniel James

4:32 p.m.

New subject: Git Modularization Review no vote heads-up

On 27 May 2013 16:38, Daniel Pfeifer wrote:

...

Oh, "functional_hash" is not mapped as a submodule yet!

That is a tricky one. I'd prefer the boost metarepository to be the only repository with submodules. But since "libs/functional" is a submodule, "libs/functional/hash" could only be a submodule of that one.

Would it be acceptable to keep Boost.Functional/Hash in the repository of Boost.Functional?

Either way is fine with me (I'm the maintainer). If it's in a different location it will break the documentation build, but that isn't much of a problem.

Jürgen Hunold

8:48 p.m.

New subject: Git Modularization Review no vote heads-up

Hi Daniel, On Monday, 27. May 2013 17:38:00 Daniel Pfeifer wrote:

...

2013/5/27 Jürgen Hunold

...
It seems that somehow libs/functional is missing the "hash" subdirectory too.

Another missing submodule?

Oh, "functional_hash" is not mapped as a submodule yet!

I found some more missing submodules . dynamic_bitset function_types logic statechart tuple unordered Found while trying to run the regression tests. See attached patch for the commented out tests. And assign is registered, but can't be checked out. Any progress on this one? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Michael Marcin

8:52 p.m.

New subject: Git Modularization Review no vote heads-up

On 5/27/2013 3:48 PM, Jürgen Hunold wrote:

...

And assign is registered, but can't be checked out. Any progress on this one?

fwiw if you git reset --hard in libs/assign you get it. HEAD is now at e36bf9c Resolve C++0x ambiguities. Fixes #3073

Bjørn Roald

26 May 26 May

7:44 p.m.

New subject: Git Modularization Review no vote heads-up

On Sun, 2013-05-26 at 20:05 +0200, Jürgen Hunold wrote:

...

Hi,

On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:

...
Next task is generating forward headers.

I got the patches applying and Boost.Build getting started. Patches for Jamroot and tools/build repository attached.

Great, I tried it and I get: bjorn@frodo2:~/repo/boost (master)$ ./b2 link.jam: No such file or directory Jamroot:144: in modules.load ERROR: rule "link-directory" unknown in module "Jamfile". /home/bjorn/repo/boost/tools/build/build/project.jam:311: in load-jamfile /home/bjorn/repo/boost/tools/build/build/project.jam:64: in load /home/bjorn/repo/boost/tools/build/build/project.jam:145: in project.find /home/bjorn/repo/boost/tools/build/build-system.jam:535: in load /home/bjorn/repo/boost/../boost/tools/build/kernel/modules.jam:289: in import /home/bjorn/repo/boost/../boost/tools/build/kernel/bootstrap.jam:139: in boost-build /home/bjorn/repo/boost/boost-build.jam:67: in module scope Looks to me like link-directory rule is missing. If you have added new file or changes to git index, then you need to do git diff HEAD. Without HEAD, git diff compare working dir to the index. -- Bjørn

Jürgen Hunold

27 May 27 May

7:05 a.m.

New subject: Git Modularization Review no vote heads-up

Hi Bjørn, On Sunday, 26. May 2013 21:44:26 Bjørn Roald wrote:

...

Looks to me like link-directory rule is missing. If you have added new file or changes to git index, then you need to do git diff HEAD. Without HEAD, git diff compare working dir to the index.

Yes, git ignores link.jam. Adding it and using git diff HEAD seems to work. Complete patch attached. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany

Bjørn Roald

24 May 24 May

5:21 a.m.

New subject: Git Modularization Review no vote heads-up

On Thu, 2013-05-23 at 06:44 -0700, Dave Abrahams wrote:

...

on Thu May 23 2013, "Jürgen Hunold" wrote:

...

...
I think that a one-to-one mapping like:

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; }

is the way to go now. If something goes wrong,

It will, because we'd no longer be catching tools/jam throughout history. That will cause modularization to fail.

...
please put "tools/jam/" somewhere else.

Yeah, that's not specific enough. What is "somewhere else?"

Why not in a separate repository called jam that are referenced only in relevant past commits of boost meta repository? Boost.Build would not be the only submodule with dependencies to other submodules, would it... This build --> jam dependency is also gone in current boost, so I don't understand the need to bundle all in one repository. It complicates the conversion and changes file structure in history. It is better to try as far as feasible to make the conversion an accurate reflection of history while getting a sensible set of repositories representing modularized libraries. Build is no a library, and as a tool the modularization is good enough with two historic repositories and a build --> jam dependency in the past. Current build repository is self contained if I understand this correctly.

...

...
Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

Agree that this is not smart or needed to be done in the conversion. This step could be done with git mv at a convenient time after the conversion. No real need to do that as part of the conversion - it complicates and obfuscates. Other build related files are in their respective repositories, this could (or should) also be the case for all top level scripts that is part of boost meta repository. Is it really a pure meta repository anyway? only my $0.02 -- Bjørn

Dave Abrahams

6:06 a.m.

New subject: Git Modularization Review no vote heads-up

on Thu May 23 2013, Bjørn Roald wrote:

...

On Thu, 2013-05-23 at 06:44 -0700, Dave Abrahams wrote:

...
on Thu May 23 2013, "Jürgen Hunold" wrote:

...
...
I think that a one-to-one mapping like:

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/" ; }

is the way to go now. If something goes wrong,

It will, because we'd no longer be catching tools/jam throughout history. That will cause modularization to fail.

...
please put "tools/jam/" somewhere else.

Yeah, that's not specific enough. What is "somewhere else?"

Why not in a separate repository called jam that are referenced only in relevant past commits of boost meta repository?

We can do that.

...

Boost.Build would not be the only submodule with dependencies to other submodules, would it...

Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's hard to justify separating them.

...

This build --> jam dependency is also gone in current boost,

I don't think so: http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine

...

so I don't understand the need to bundle all in one repository. It complicates the conversion and changes file structure in history.

I don't know what you mean by "changes file structure in history" or how you conclude that it complicates the conversion.

...

It is better to try as far as feasible to make the conversion an accurate reflection of history while getting a sensible set of repositories representing modularized libraries. Build is no a library, and as a tool the modularization is good enough with two historic repositories and a build --> jam dependency in the past. Current build repository is self contained if I understand this correctly.

It's only self-contained because it contains jam in its "engine" directory. If Boost.Jam is going to be there in the present, its revision history should be there in the past, so you can follow it.

...

...
...
Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

Agree that this is not smart or needed to be done in the conversion. This step could be done with git mv at a convenient time after the conversion. No real need to do that as part of the conversion - it complicates and obfuscates.

Other build related files are in their respective repositories, this could (or should) also be the case for all top level scripts that is part of boost meta repository.

I don't know what you mean here either. Specifics, please.

...

Is it really a pure meta repository anyway?

I'm afraid it's not pure at all. -- Dave Abrahams

Bjørn Roald

10:04 p.m.

New subject: Git Modularization Review no vote heads-up

On Thu, 2013-05-23 at 23:06 -0700, Dave Abrahams wrote:

...

on Thu May 23 2013, Bjørn Roald wrote:

...

...
Boost.Build would not be the only submodule with dependencies to other submodules, would it...

Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's hard to justify separating them.

I understand that point of view, but if historic Jam could be used by itself, why not have them as separate modules?

...

...
This build --> jam dependency is also gone in current boost,

I don't think so: http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine

I was not aware there is a single line of history here - in case it is it may be worth trying to re-join the histories if that is done by the conversion. but it seems odd to re-introduce tools/jam in tools/build/jam, a new place where it has never been -- and move tools/build to tools/build/build, where it has never been.

...

...
so I don't understand the need to bundle all in one repository. It complicates the conversion and changes file structure in history.

I don't know what you mean by "changes file structure in history" or how you conclude that it complicates the conversion.

Well, are you saying I can check out any boost historic version from http://github.com/boostorg/boost and run the instructions of that release to successfully build? Those instructions will fail as structure of those historic commits are changed beyond moving the headers. The changes in tools/build is a prime example.

...

...
It is better to try as far as feasible to make the conversion an accurate reflection of history while getting a sensible set of repositories representing modularized libraries. Build is no a library, and as a tool the modularization is good enough with two historic repositories and a build --> jam dependency in the past. Current build repository is self contained if I understand this correctly.

It's only self-contained because it contains jam in its "engine" directory. If Boost.Jam is going to be there in the present, its revision history should be there in the past, so you can follow it.

so are you are saying you will have some commit the build repository history show how files are moved from tools/build/jam where they never have been to tools/build/build/v2/engine where they never went? Even if you manage to make such a commit, how is that preserving file structure in the history?

...

...
...
...
Btw. would

repository build : common_branches { submodule of "boost" : "tools/build"; content { "tools/build/v2" ; }

be the right way to move v2 one directory up?

That would move the contents of v2 to the top level of the build repo, but it would also drop all the commits that historically went directly into tools/build/ (Boost.Build v1 mostly). Like I said, we need decisions about where *everything* that was in Boost.Build—throughout history—belongs.

Agree that this is not smart or needed to be done in the conversion. This step could be done with git mv at a convenient time after the conversion. No real need to do that as part of the conversion - it complicates and obfuscates.

Other build related files are in their respective repositories, this could (or should) also be the case for all top level scripts that is part of boost meta repository.

I don't know what you mean here either. Specifics, please.

Ok, it is just the obvious I guess I am trying to confirm, sorry for not being clear here and having to explain these things you know much better than me - feels very odd. But I will try to explayn what I mean. There are jam files all over the place -- to find the files that have jam in name. The command @ find | grep -i jam | grep -v tools/build does not provide complete list of all build related scripts outside of tools/build, but it will list a lot of build stuff that will not be part of build repository I presume, some of these are in the top level directory ./project-config.jam ./Jamfile.v2 ./Jamroot.jam ./boostcpp.jam and they may end up staying there, i.e. not become part of any of the new submodules, candidates are : in addition bootstrap.(sh|bat) are part of the build system. With file structure changes in history for tools/build and tools/jam, these and their past siblings are surly broken. In addition files in lib/*/build/ directories that belong to the respective lib/* submodules does not go into build repository, right? Nevertheless I struggle with understanding the intension of some of the entries for the build repository in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt so there may be intention to do more than the current converted repositories reflect and I expect here. -- Bjørn

Dave Abrahams

30 May 30 May

10:34 p.m.

New subject: Git Modularization Review no vote heads-up

on Fri May 24 2013, Bjørn Roald wrote:

...

On Thu, 2013-05-23 at 23:06 -0700, Dave Abrahams wrote:

...
on Thu May 23 2013, Bjørn Roald wrote:

...
...
Boost.Build would not be the only submodule with dependencies to other submodules, would it...

Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's hard to justify separating them.

I understand that point of view, but if historic Jam could be used by itself, why not have them as separate modules?

Because it makes everything harder. If the Boost.Build maintainers would prefer to have them separated, we can do that, but I advise against it.

...

...
...
This build --> jam dependency is also gone in current boost,

I don't think so: http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine

I was not aware there is a single line of history here - in case it is it may be worth trying to re-join the histories if that is done by the conversion. but it seems odd to re-introduce tools/jam in tools/build/jam, a new place where it has never been -- and move tools/build to tools/build/build, where it has never been.

You have to let go of the idea of the Boost super-repository having the actual historical paths to anything.

...

...
...
so I don't understand the need to bundle all in one repository. It complicates the conversion and changes file structure in history.

I don't know what you mean by "changes file structure in history" or how you conclude that it complicates the conversion.

Well, are you saying I can check out any boost historic version from http://github.com/boostorg/boost and run the instructions of that release to successfully build?

No. That *will not* be the case. For that, use SVN.

...

Those instructions will fail as structure of those historic commits are changed beyond moving the headers. The changes in tools/build is a prime example.

...
...
It is better to try as far as feasible to make the conversion an accurate reflection of history while getting a sensible set of repositories representing modularized libraries. Build is no a library, and as a tool the modularization is good enough with two historic repositories and a build --> jam dependency in the past. Current build repository is self contained if I understand this correctly.

It's only self-contained because it contains jam in its "engine" directory. If Boost.Jam is going to be there in the present, its revision history should be there in the past, so you can follow it.

so are you are saying you will have some commit the build repository history show how files are moved from tools/build/jam where they never have been to tools/build/build/v2/engine where they never went? Even if you manage to make such a commit, how is that preserving file structure in the history?

I didn't say anything about preserving file structure in the history. What I am saying is that you should be able to follow the changes to any file that is part of Boost.Jam through its history, and if those files are in the Build repository in the present but jump there from a different repository some time in the past, you won't be able to do that.

...

...
...
Agree that this is not smart or needed to be done in the conversion. This step could be done with git mv at a convenient time after the conversion. No real need to do that as part of the conversion - it complicates and obfuscates.

Other build related files are in their respective repositories, this could (or should) also be the case for all top level scripts that is part of boost meta repository.

I don't know what you mean here either. Specifics, please.

Ok, it is just the obvious I guess I am trying to confirm, sorry for not being clear here and having to explain these things you know much better than me - feels very odd. But I will try to explayn what I mean.

There are jam files all over the place -- to find the files that have jam in name. The command

@ find | grep -i jam | grep -v tools/build

does not provide complete list of all build related scripts outside of tools/build, but it will list a lot of build stuff that will not be part of build repository I presume, some of these are in the top level directory

./project-config.jam ./Jamfile.v2 ./Jamroot.jam ./boostcpp.jam

and they may end up staying there, i.e. not become part of any of the new submodules, candidates are :

in addition bootstrap.(sh|bat) are part of the build system. With file structure changes in history for tools/build and tools/jam, these and their past siblings are surly broken.

In addition files in lib/*/build/ directories that belong to the respective lib/* submodules does not go into build repository, right?

Nevertheless I struggle with understanding the intension of some of the entries for the build repository in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt so there may be intention to do more than the current converted repositories reflect and I expect here.

Sorry, I just don't know what to do with this input. I'm a bit overwhelmed with everything going on in my life at the moment, so I apologize. -- Dave Abrahams

Bjørn Roald

11:49 p.m.

New subject: Git Modularization Review no vote heads-up

On Thu, 2013-05-30 at 15:34 -0700, Dave Abrahams wrote:

...

on Fri May 24 2013, Bjørn Roald wrote:

...
On Thu, 2013-05-23 at 23:06 -0700, Dave Abrahams wrote:

...
on Thu May 23 2013, Bjørn Roald wrote:

...
...
Boost.Build would not be the only submodule with dependencies to other submodules, would it...

Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's hard to justify separating them.

I understand that point of view, but if historic Jam could be used by itself, why not have them as separate modules?

Because it makes everything harder. If the Boost.Build maintainers would prefer to have them separated, we can do that, but I advise against it.

Agree, its their choice. Especially in the future. My issue is more about changing the past.

...

...
...
...
This build --> jam dependency is also gone in current boost,

I don't think so: http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine

I was not aware there is a single line of history here - in case it is it may be worth trying to re-join the histories if that is done by the conversion. but it seems odd to re-introduce tools/jam in tools/build/jam, a new place where it has never been -- and move tools/build to tools/build/build, where it has never been.

You have to let go of the idea of the Boost super-repository having the actual historical paths to anything.

Ok, I can accept that, if that is how it is. It just escapes me what the super-repository's purpose is then. But that is no big deal, I guess it acts a holder of references to other boost code.

...

...
...
...
so I don't understand the need to bundle all in one repository. It complicates the conversion and changes file structure in history.

I don't know what you mean by "changes file structure in history" or how you conclude that it complicates the conversion.

Well, are you saying I can check out any boost historic version from http://github.com/boostorg/boost and run the instructions of that release to successfully build?

No. That *will not* be the case. For that, use SVN.

Or tar-balls I guess. This make sense as you say no global state of history is kept or attempted to be kept, so that means no need to mangle the history of the top level scripts in the modularized boost repo.

...

...
Those instructions will fail as structure of those historic commits are changed beyond moving the headers. The changes in tools/build is a prime example.

...
...
It is better to try as far as feasible to make the conversion an accurate reflection of history while getting a sensible set of repositories representing modularized libraries. Build is no a library, and as a tool the modularization is good enough with two historic repositories and a build --> jam dependency in the past. Current build repository is self contained if I understand this correctly.

It's only self-contained because it contains jam in its "engine" directory. If Boost.Jam is going to be there in the present, its revision history should be there in the past, so you can follow it.

so are you are saying you will have some commit the build repository history show how files are moved from tools/build/jam where they never have been to tools/build/build/v2/engine where they never went? Even if you manage to make such a commit, how is that preserving file structure in the history?

I didn't say anything about preserving file structure in the history. What I am saying is that you should be able to follow the changes to any file that is part of Boost.Jam through its history,

Ok, In that case I follow the logic.

...

and if those files are in the Build repository in the present but jump there from a different repository some time in the past, you won't be able to do that.

fair enough. I guess I expected more than was feasible when you had added history to the modularized boost repositories. I understand the need to restructure some, I guess it surprised me how many adaptations has been done beyond the header file reorganization. ... snip...

...

Sorry, I just don't know what to do with this input. I'm a bit overwhelmed with everything going on in my life at the moment, so I apologize.

no worries, I am happy with the replies given above and understands the direction this is moving. I have mo major issues with the choices that has been made. Best regards, -- Bjørn

4200

Age (days ago)

4207

Last active (days ago)

List overview

Download

36 comments

10 participants

participants (10)

"Jürgen Hunold"
Beman Dawes
Bjørn Roald
Daniel James
Daniel Pfeifer
Dave Abrahams
Jürgen Hunold
Michael Marcin
Rene Rivera
Vladimir Prus