git reset and force push
Hi all, Is there any policy wrt. reseting a branch master/develop and force pushing? The side question is to know if the boost git tools (the bot for submodules) can cope with those operations. Thanks, Raffi
On Tue, Oct 6, 2015 at 7:56 PM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Hi all,
Is there any policy wrt. reseting a branch master/develop and force pushing? The side question is to know if the boost git tools (the bot for submodules) can cope with those operations.
Thanks, Raffi
Are you trying to remove/hide something from the history? Unless it was a largish binary file added by mistake, why would you need to do that? If you want to remove commit(s) that have been published "git revert" should do it. The interim changes will be shown in the history, so force pushing won't be necessary. Lee
On 07.10.2015 02:56, Raffi Enficiaud wrote:
Hi all,
Is there any policy wrt. reseting a branch master/develop and force pushing? The side question is to know if the boost git tools (the bot for submodules) can cope with those operations.
Please, don't do this. It breaks pulling changes from github.
Le 07/10/15 09:20, Andrey Semashev a écrit :
On 07.10.2015 02:56, Raffi Enficiaud wrote:
Hi all,
Is there any policy wrt. reseting a branch master/develop and force pushing? The side question is to know if the boost git tools (the bot for submodules) can cope with those operations.
Please, don't do this. It breaks pulling changes from github.
Ok, this is what I wanted to know. Basically I recreated a develop
branch for boost.test release to develop (then to master) on a clean
history. I wanted to know if the git bot of the modular boost (the one
that commits under the name "Automated Commit
On 07.10.2015 10:25, Raffi Enficiaud wrote:
Le 07/10/15 09:20, Andrey Semashev a écrit :
On 07.10.2015 02:56, Raffi Enficiaud wrote:
Hi all,
Is there any policy wrt. reseting a branch master/develop and force pushing? The side question is to know if the boost git tools (the bot for submodules) can cope with those operations.
Please, don't do this. It breaks pulling changes from github.
Ok, this is what I wanted to know. Basically I recreated a develop branch for boost.test release to develop (then to master) on a clean history. I wanted to know if the git bot of the modular boost (the one that commits under the name "Automated Commit
") is able to take just the sha of the develop branch, or if its relying on some linear history. You're saying this is not supported. Is that correct? Why not supporting this? Is it by design?
No, I'm not saying that the commit bot can't handle that (it probably can because someone already tried to rewrite history in the past and it kept working). I don't know for sure since I'm not maintaining the bot (or any testers, which may also be affected). I'm saying that rewriting history breaks pulling, which is what developers do when they update local git repos. It is a major nuisance when the history breaks as everyone has to repair their local branches.
Le 07/10/15 09:58, Andrey Semashev a écrit :
You're saying this is not supported. Is that correct? Why not supporting this? Is it by design?
No, I'm not saying that the commit bot can't handle that (it probably can because someone already tried to rewrite history in the past and it kept working). I don't know for sure since I'm not maintaining the bot (or any testers, which may also be affected).
I'm saying that rewriting history breaks pulling, which is what developers do when they update local git repos. It is a major nuisance when the history breaks as everyone has to repair their local branches.
Ok, but this occurs only if develop is checked out locally, which does not happen when we just do git submodule update --recursive So for people that have checked out boost.test develop branch, they would need to reset, for the others, it should work, isn't it? git submodule update --force libs/test should do the trick for those who have checked out boost.test[develop]. What do you think? Sorry for insisting a bit on this path, but this is the shortest and cleanest path now for fixing the C++11 vs. C++03 issues of boost.test. In the future, I think I will maintain a "develop-release" branch for being able to force push the topics on it (and have our own testing infrastructure on that). Raffi
On 07.10.2015 11:43, Raffi Enficiaud wrote:
Le 07/10/15 09:58, Andrey Semashev a écrit :
You're saying this is not supported. Is that correct? Why not supporting this? Is it by design?
No, I'm not saying that the commit bot can't handle that (it probably can because someone already tried to rewrite history in the past and it kept working). I don't know for sure since I'm not maintaining the bot (or any testers, which may also be affected).
I'm saying that rewriting history breaks pulling, which is what developers do when they update local git repos. It is a major nuisance when the history breaks as everyone has to repair their local branches.
Ok, but this occurs only if develop is checked out locally, which does not happen when we just do
git submodule update --recursive
So for people that have checked out boost.test develop branch, they would need to reset, for the others, it should work, isn't it?
The command above leaves submodules in a detached state. To get a working state of submodules, with history and ability to create local branches (e.g. for development or pull requests), you have to checkout a branch. Because of that I always keep all submodules checked out on some branch (typically, develop). Personally, I update Boost with the following command: git pull --recurse-submodules && git submodule foreach git merge --ff-only I use git submodule update --recursive --init git submodule foreach git checkout develop only on rare occasions, when I know there are new submodules that I want to download. Even barring the pull breakage, IMO tampering with history is not something you should be doing to revert changes in the first place. History exists for the very reason of reflecting changes made to the code along with the comments why those changes were made.
git submodule update --force libs/test
should do the trick for those who have checked out boost.test[develop]. What do you think?
I'm not sure what --force means in this command, but I'd rather avoid it if possible, because in general --force is not safe.
Sorry for insisting a bit on this path, but this is the shortest and cleanest path now for fixing the C++11 vs. C++03 issues of boost.test. In the future, I think I will maintain a "develop-release" branch for being able to force push the topics on it (and have our own testing infrastructure on that).
I'm not sure what kind of changes there are that require rewriting history and cannot be fixed with new commits reverting the older ones. The only reason to rewrite history I can imagine is to drop commits to reduce the size of the git database. I don't think this is your case though.
Le 07/10/15 12:01, Andrey Semashev a écrit :
On 07.10.2015 11:43, Raffi Enficiaud wrote:
Le 07/10/15 09:58, Andrey Semashev a écrit :
You're saying this is not supported. Is that correct? Why not supporting this? Is it by design?
No, I'm not saying that the commit bot can't handle that (it probably can because someone already tried to rewrite history in the past and it kept working). I don't know for sure since I'm not maintaining the bot (or any testers, which may also be affected).
I'm saying that rewriting history breaks pulling, which is what developers do when they update local git repos. It is a major nuisance when the history breaks as everyone has to repair their local branches.
Ok, but this occurs only if develop is checked out locally, which does not happen when we just do
git submodule update --recursive
So for people that have checked out boost.test develop branch, they would need to reset, for the others, it should work, isn't it?
The command above leaves submodules in a detached state. To get a working state of submodules, with history and ability to create local branches (e.g. for development or pull requests), you have to checkout a branch. Because of that I always keep all submodules checked out on some branch (typically, develop). Personally, I update Boost with the following command:
git pull --recurse-submodules && git submodule foreach git merge --ff-only
Personally I never checkout a submodule unless I have to work on it, since I do not need them (!), and a submodule develop/master might not be part of any version of the superproject itself (and hence not being able to test it). Also with your merge you might create new commits that are not on the remote. If I were to work on a fork, I just add a remote, and check out against what I am doing. That is to say, I am perfectly fine with a detached state. I also think that for the reasons I exposed, the way you update your copy is not a good practice in general.
I use
git submodule update --recursive --init git submodule foreach git checkout develop
only on rare occasions, when I know there are new submodules that I want to download.
Even barring the pull breakage, IMO tampering with history is not something you should be doing to revert changes in the first place. History exists for the very reason of reflecting changes made to the code along with the comments why those changes were made.
This is not something I would like to do always. But since our boost.test develop branch has this so special value for the other libraries in there, and many commits were performed after the point I want to revert to, that I want to organize the things a bit differently. OTOH, the features git provides enable to maintain a much cleaner and sharper history, which helps a lot over time. Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
git submodule update --force libs/test
should do the trick for those who have checked out boost.test[develop]. What do you think?
I'm not sure what --force means in this command, but I'd rather avoid it if possible, because in general --force is not safe.
Right, so maybe without the force it would work as well. I believe this is not a problem in general, since the submodule command checks out a specific commit. But for your case, if you fastforward, some problems might happen.
Sorry for insisting a bit on this path, but this is the shortest and cleanest path now for fixing the C++11 vs. C++03 issues of boost.test. In the future, I think I will maintain a "develop-release" branch for being able to force push the topics on it (and have our own testing infrastructure on that).
I'm not sure what kind of changes there are that require rewriting history and cannot be fixed with new commits reverting the older ones. The only reason to rewrite history I can imagine is to drop commits to reduce the size of the git database. I don't think this is your case though.
I really do not care about the size. I would like to have a clean history, with consistent commits organized in topics, being able to revert topics as a whole, make topics orthogonal with appropriate rebasing etc. Resetting the develop branch - would ensure that we go back to a state where we know everything was ok, - is a fast way of fixing things - is the least error prone (in terms of commits, not in terms of git manipulations). The purpose of this thread is to have a better idea of the practices of boost, also wrt. a potential nondestructive hard reset (non destructive in the sense that the commits have been cherry picked on appropriate topic branches). So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test). Best, Raffi
On 10/7/2015 10:27 AM, Raffi Enficiaud wrote:
The purpose of this thread is to have a better idea of the practices of boost, also wrt. a potential nondestructive hard reset (non destructive in the sense that the commits have been cherry picked on appropriate topic branches).
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
Since you are counting, make that 2. A force push has already happen once in the past, unintentionally, and it was a disruptive process not only for developers but also for external test runners. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
Le 07/10/15 15:54, Agustín K-ballo Bergé a écrit :
On 10/7/2015 10:27 AM, Raffi Enficiaud wrote:
The purpose of this thread is to have a better idea of the practices of boost, also wrt. a potential nondestructive hard reset (non destructive in the sense that the commits have been cherry picked on appropriate topic branches).
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
Since you are counting, make that 2. A force push has already happen once in the past, unintentionally, and it was a disruptive process not only for developers but also for external test runners.
I am curious how external test runners are impacted. Do you have more details on this? Raffi
On 10/7/2015 12:26 PM, Raffi Enficiaud wrote:
Le 07/10/15 15:54, Agustín K-ballo Bergé a écrit :
On 10/7/2015 10:27 AM, Raffi Enficiaud wrote:
The purpose of this thread is to have a better idea of the practices of boost, also wrt. a potential nondestructive hard reset (non destructive in the sense that the commits have been cherry picked on appropriate topic branches).
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
Since you are counting, make that 2. A force push has already happen once in the past, unintentionally, and it was a disruptive process not only for developers but also for external test runners.
I am curious how external test runners are impacted. Do you have more details on this?
Test runners are not necessarily prepared to deal with history rewrites, since those are a frowned upon practice in "public" branches (I'm surprised it's even being considered here). A force push would cause spurious failures in those runners, which forces (pun intended) someone to look at it, notice that someone misbehaved, decide on a fix, pester the sysadmin for weeks until the fix is applied, etc. At least, that's how it went last time. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
Le 07/10/15 17:44, Agustín K-ballo Bergé a écrit :
On 10/7/2015 12:26 PM, Raffi Enficiaud wrote:
Test runners are not necessarily prepared to deal with history rewrites, since those are a frowned upon practice in "public" branches (I'm surprised it's even being considered here).
That is also a surprise: why so? the runner API is running the run.py script that is cloning/pulling things: it does not check out any branch, so it does not care if develop changed by history rewrite. I am very curious to see what type of errors, and what are the exact causes, we had at that time: do you have an idea? because to me - see below - this should not be a problem for a bot. Maybe you can point me to the corresponding thread (if you have time)?
A force push would cause spurious failures in those runners,
The /only/ (not to say it is a minor) inconvenience I can see so far is when someone checks out a public branch locally, and a remote history rewrite messes up completely the local branch, so "git reset --hard" is needed. History rewrite causes no problem with "git fetch". The problems created by history rewrite are human-to-human, not human-to-robot: 1- a human checks out develop of a submodule, for eg. working on the submodule 2- history rewrite happens remotely on a submodule 3- problems to be compared with: 1- a robot checks out the superproject 2- history rewrite happens remotely on a submodule 3- robot can continue updating the superproject
which forces (pun intended) someone to look at it, notice that someone misbehaved, decide on a fix, pester the sysadmin for weeks until the fix is applied, etc.
There is no such as thing like "misbehaviour". The topic of this thread is to discuss about policies of boost wrt. to operations /permitted/ by git rewriting the history. So far, I heard about common or less common practice, but nothing written in black on white.
At least, that's how it went last time.
Can you point me to the thread? Thanks, Raffi
Le 07/10/15 22:44, Raffi Enficiaud a écrit :
That is also a surprise: why so? the runner API is running the run.py script that is cloning/pulling things: it does not check out any branch, so it does not care if develop changed by history rewrite.
I am referring to a submodule there.
Raffi Enficiaud wrote:
The problems created by history rewrite are human-to-human, not human-to-robot:
This is not true. Today the boost supermodule points to a particular sha of the submodule you want to rewrite the history of. If you rewrite the history of the submodule, the commit referenced by the sha will be garbage collected and no one in the future will be able to make a checkout of the boost supermodule as it is today. You affect usefulness of the supermodule by rewriting history of a submodule. Just don't do it. You could push a branch with the current state of develop to a branch named develop-in-early-October-2015 which would prevent the shas referenced by the supermodule being garbage collected. How many times do you want to push branches with names like that? How do you defend against a zealous maintainer 'cleaning up useless branches' and breaking the supermodule history? Thanks, Steve.
Le 07/10/15 22:56, Stephen Kelly a écrit :
Raffi Enficiaud wrote:
The problems created by history rewrite are human-to-human, not human-to-robot:
This is not true.
Today the boost supermodule points to a particular sha of the submodule you want to rewrite the history of.
If you rewrite the history of the submodule, the commit referenced by the sha will be garbage collected and no one in the future will be able to make a checkout of the boost supermodule as it is today.
This is indeed (and sorry to say for this thread, "finally") a very good point: the problem then comes from the fact that we want to go back to time at the superproject level, which would be not possible if blobs are garbage collected. You affect usefulness of
the supermodule by rewriting history of a submodule. Just don't do it.
I understood that it is a source of annoyance for everyone, I am not a psychopath, I will not do it. I wanted to be convinced by a *good reason*, not just a potential bad implementation ("robot" or exotic "submodule update" command) or some /fear of the unknown/. What you just said is indeed a /good reason/.
You could push a branch with the current state of develop to a branch named develop-in-early-October-2015 which would prevent the shas referenced by the supermodule being garbage collected. How many times do you want to push branches with names like that? How do you defend against a zealous maintainer 'cleaning up useless branches' and breaking the supermodule history?
I which I were zealous. The reason I was bringing this topic is that this is the shortest and cleanest path to some breaking changes (and not because I do not like the shape of the history), where bad and good things happened at the same time. BTW, this point in time "develop-in-early-October-2015" is already in place... (just in case of an accident). Thanks for your insights, Raffi
On 10/7/2015 5:44 PM, Raffi Enficiaud wrote:
Le 07/10/15 17:44, Agustín K-ballo Bergé a écrit :
On 10/7/2015 12:26 PM, Raffi Enficiaud wrote:
Test runners are not necessarily prepared to deal with history rewrites, since those are a frowned upon practice in "public" branches (I'm surprised it's even being considered here).
That is also a surprise: why so? the runner API is running the run.py script that is cloning/pulling things: it does not check out any branch, so it does not care if develop changed by history rewrite.
You seem to be assuming that external test runners run the Boost regression test suite, that's not necessarily the case. It's not uncommon for libraries or applications that rely on Boost to proactively build against what will be the next version to catch errors ahead of time.
I am very curious to see what type of errors, and what are the exact causes, we had at that time: do you have an idea? because to me - see below - this should not be a problem for a bot. Maybe you can point me to the corresponding thread (if you have time)?
Others have addressed this already, and there's plenty of material on the webs too. I've only felt the need to jump into this thread when positions started to be counted, up until then I did not think it was even worth the time discussing it. The case presented is just but one example of how rewriting history can be disruptive; as a person who frequently uses Boost from git, my position would stay the same even if every bot in existence would be modified to tolerate history rewrites.
which forces (pun intended) someone to look at it, notice that someone misbehaved, decide on a fix, pester the sysadmin for weeks until the fix is applied, etc.
There is no such as thing like "misbehaviour". The topic of this thread is to discuss about policies of boost wrt. to operations /permitted/ by git rewriting the history.
That's of course subjective, the operation exists and the tools allow you to use it. My opinion is that using it on a public branch is misbehaving. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
Le 07/10/15 23:27, Agustín K-ballo Bergé a écrit :
On 10/7/2015 5:44 PM, Raffi Enficiaud wrote:
Le 07/10/15 17:44, Agustín K-ballo Bergé a écrit :
On 10/7/2015 12:26 PM, Raffi Enficiaud wrote:
Test runners are not necessarily prepared to deal with history rewrites, since those are a frowned upon practice in "public" branches (I'm surprised it's even being considered here).
That is also a surprise: why so? the runner API is running the run.py script that is cloning/pulling things: it does not check out any branch, so it does not care if develop changed by history rewrite.
You seem to be assuming that external test runners run the Boost regression test suite, that's not necessarily the case. It's not uncommon for libraries or applications that rely on Boost to proactively build against what will be the next version to catch errors ahead of time.
Still they do not need to know what other submodules are doing, they need to know what their own submodule is doing. Your example does not justify anything (see below).
I am very curious to see what type of errors, and what are the exact causes, we had at that time: do you have an idea? because to me - see below - this should not be a problem for a bot. Maybe you can point me to the corresponding thread (if you have time)?
Others have addressed this already, and there's plenty of material on the webs too.
So you're just claiming that there are many /unknown and non-cited/ resources, providing /evidence/ impossible to quote and /material/ impossible to reproduce: what you are claiming exactly again? I am sorry, but a "just don't do it" is something I cannot buy, nor the "others have said", nor "problems occurred" without at any point of the discussion being able to name or reference what we are talking about.
I've only felt the need to jump into this thread when positions started to be counted, up until then I did not think it was even worth the time discussing it.
Yes, quantifying is good, thank you for having taken part of the count. Quantifying is as important as good reference supporting claims though. "Others" gave a precise example where operations fail based on what I suggested and the history of the discussion (again to be understood: not from the developers of the submodule in question checking out a specific version, but from users of the superproject). The case presented is just but one example of how
rewriting history can be disruptive; as a person who frequently uses Boost from git, my position would stay the same even if every bot in existence would be modified to tolerate history rewrites.
Again your position sounds rather religious to me, and your claims are hardly convincing. I've **never** said rewriting history is not disruptive BTW. I said that **in the current superproject setup**, since the superproject is only interested in SHA1, rewriting history should work for most of us (mortal or robots) because we are not checking out particular branches of a submodule (but checking out SHA1). And in fact it does: if you do not checkout any submodule branch and if you always go forward in the superproject history, it just works. Only two "ifs", but important ones.
which forces (pun intended) someone to look at it, notice that someone misbehaved, decide on a fix, pester the sysadmin for weeks until the fix is applied, etc.
There is no such as thing like "misbehaviour". The topic of this thread is to discuss about policies of boost wrt. to operations /permitted/ by git rewriting the history.
That's of course subjective, the operation exists and the tools allow you to use it.
This is the exact opposite of subjective: this is just factual. - is there any policy wrt. to this? no - git allows me to do that? yes - what if I do it in the current settings of the superproject? convinced by a clear case of superproject disruption from the answer of Stephen Kelly (hey, thanks again Stephen!): going back in time may not be possible if blobs are garbage collected.
My opinion is that using it on a public branch is misbehaving.
There we are: thanks for underlining the following two important words this time: /"My opinion"/. Kind regards, Raffi
On 10/7/2015 7:18 PM, Raffi Enficiaud wrote:
Le 07/10/15 23:27, Agustín K-ballo Bergé a écrit :
On 10/7/2015 5:44 PM, Raffi Enficiaud wrote:
[snipped a lot of text] It seems to me the general gist of what you are saying is that if everyone follows the steps you describe, rewriting history should not be a problem (please correct me if I'm wrong). I'm not challenging that, I'm saying that this is not particularly the way people and projects I know of work, me included at a certain point in time. Until everyone and every tool using the git repository is reached and adjusted to your needs I don't see how you can consider it a viable option, unless your point of view is that disrupting the work of others than don't do as you think should be done is acceptable.
There is no such as thing like "misbehaviour". The topic of this thread is to discuss about policies of boost wrt. to operations /permitted/ by git rewriting the history.
That's of course subjective, the operation exists and the tools allow you to use it.
This is the exact opposite of subjective: this is just factual.
My opinion on whether doing something constitutes misbehavior is entirely subjective.
My opinion is that using it on a public branch is misbehaving.
There we are: thanks for underlining the following two important words this time: /"My opinion"/.
Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
On Wed, Oct 7, 2015 at 3:27 PM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Le 07/10/15 12:01, Andrey Semashev a écrit :
On 07.10.2015 11:43, Raffi Enficiaud wrote:
Le 07/10/15 09:58, Andrey Semashev a écrit :
You're saying this is not supported. Is that correct? Why not
supporting this? Is it by design?
No, I'm not saying that the commit bot can't handle that (it probably can because someone already tried to rewrite history in the past and it kept working). I don't know for sure since I'm not maintaining the bot (or any testers, which may also be affected).
I'm saying that rewriting history breaks pulling, which is what developers do when they update local git repos. It is a major nuisance when the history breaks as everyone has to repair their local branches.
Ok, but this occurs only if develop is checked out locally, which does not happen when we just do
git submodule update --recursive
So for people that have checked out boost.test develop branch, they would need to reset, for the others, it should work, isn't it?
The command above leaves submodules in a detached state. To get a working state of submodules, with history and ability to create local branches (e.g. for development or pull requests), you have to checkout a branch. Because of that I always keep all submodules checked out on some branch (typically, develop). Personally, I update Boost with the following command:
git pull --recurse-submodules && git submodule foreach git merge --ff-only
Personally I never checkout a submodule unless I have to work on it, since I do not need them (!), and a submodule develop/master might not be part of any version of the superproject itself (and hence not being able to test it). Also with your merge you might create new commits that are not on the remote.
If I were to work on a fork, I just add a remote, and check out against what I am doing.
That is to say, I am perfectly fine with a detached state. I also think that for the reasons I exposed, the way you update your copy is not a good practice in general.
I use
git submodule update --recursive --init git submodule foreach git checkout develop
only on rare occasions, when I know there are new submodules that I want to download.
Even barring the pull breakage, IMO tampering with history is not something you should be doing to revert changes in the first place. History exists for the very reason of reflecting changes made to the code along with the comments why those changes were made.
This is not something I would like to do always. But since our boost.test develop branch has this so special value for the other libraries in there, and many commits were performed after the point I want to revert to, that I want to organize the things a bit differently.
OTOH, the features git provides enable to maintain a much cleaner and sharper history, which helps a lot over time.
Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
git submodule update --force libs/test
should do the trick for those who have checked out boost.test[develop]. What do you think?
I'm not sure what --force means in this command, but I'd rather avoid it if possible, because in general --force is not safe.
Right, so maybe without the force it would work as well. I believe this is not a problem in general, since the submodule command checks out a specific commit. But for your case, if you fastforward, some problems might happen.
Sorry for insisting a bit on this path, but this is the shortest and
cleanest path now for fixing the C++11 vs. C++03 issues of boost.test. In the future, I think I will maintain a "develop-release" branch for being able to force push the topics on it (and have our own testing infrastructure on that).
I'm not sure what kind of changes there are that require rewriting history and cannot be fixed with new commits reverting the older ones. The only reason to rewrite history I can imagine is to drop commits to reduce the size of the git database. I don't think this is your case though.
I really do not care about the size. I would like to have a clean history, with consistent commits organized in topics, being able to revert topics as a whole, make topics orthogonal with appropriate rebasing etc.
Resetting the develop branch - would ensure that we go back to a state where we know everything was ok, - is a fast way of fixing things - is the least error prone (in terms of commits, not in terms of git manipulations).
The purpose of this thread is to have a better idea of the practices of boost, also wrt. a potential nondestructive hard reset (non destructive in the sense that the commits have been cherry picked on appropriate topic branches).
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
Best, Raffi
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Please don't force push. According to Github there are 42 people who have forked boostorg/test. At least that many people can be affected in very surprising ways and many more who're developing locally without a forked Github repository. To have a nice clean history all the time you have to always work on non-public branches and merge to public branches (like develop) only once you're sure your change is ready to be public. Until you make it public, you do whatever you want with the history. (btw, non-public branches can be branches published on Github, but not master or develop in case of boost) Please don't force push. Thanks, Andras
Le 07/10/15 15:56, András Kucsma a écrit :
[snip]
Please don't force push. According to Github there are 42 people who have forked boostorg/test. At least that many people can be affected in very surprising ways and many more who're developing locally without a forked Github repository.
To have a nice clean history all the time you have to always work on non-public branches and merge to public branches (like develop) only once you're sure your change is ready to be public. Until you make it public, you do whatever you want with the history.
(btw, non-public branches can be branches published on Github, but not master or develop in case of boost)
Please don't force push.
Thanks, Andras
Ok, ok :) so 42 against. Such a nice number BTW.
On 7 October 2015 at 15:21, Raffi Enficiaud wrote: Le 07/10/15 15:56, András Kucsma a écrit : [snip] Please don't force push. According to Github there are 42 people who have
forked
boostorg/test. At least that many people can be affected in very
surprising
ways
and many more who're developing locally without a forked Github
repository. To have a nice clean history all the time you have to always work on
non-public
branches and merge to public branches (like develop) only once you're sure
your
change is ready to be public. Until you make it public, you do whatever
you
want
with the history. (btw, non-public branches can be branches published on Github, but not
master or
develop in case of boost) Please don't force push. Thanks,
Andras Ok, ok :) so 42 against. Such a nice number BTW. out of interest, what's wrong with just reverting the offending commits?
On 7 October 2015 at 16:35, Raffi Enficiaud wrote: Le 07/10/15 16:44, Sam Kellett a écrit : out of interest, what's wrong with just reverting the offending commits? There is a pile of them, coupled with commits I want to keep. revert does
just not work, I have to rewind to that commit. sorry i'm not sure follow... what's wrong with not just reverting each
commit that was bad (how many are there that doing a force reset sounds
more appealing?). what do you mean coupled, also? as in there's bit in your
commits that want to keep but revert the majority of it? 'git rebase -i'
will allow you to edit individual commits where you can pull out the stuff
you want to keep and revert the reset of it.
nuking the history seems like such a last resort i'm struggling to see how
it could be beneficial over spending a bit more time sorting out the tree..
On 7 October 2015 at 16:45, Sam Kellett
On 7 October 2015 at 16:35, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Le 07/10/15 16:44, Sam Kellett a écrit :
out of interest, what's wrong with just reverting the offending commits?
There is a pile of them, coupled with commits I want to keep. revert does just not work, I have to rewind to that commit.
sorry i'm not sure follow... what's wrong with not just reverting each commit that was bad (how many are there that doing a force reset sounds more appealing?). what do you mean coupled, also? as in there's bit in your commits that want to keep but revert the majority of it? 'git rebase -i' will allow you to edit individual commits where you can pull out the stuff you want to keep and revert the reset of it.
nuking the history seems like such a last resort i'm struggling to see how it could be beneficial over spending a bit more time sorting out the tree..
hang on, blonde moment... git rebase will of course change the history so ignore that! of course you could take a diff of said commit, revert it, apply the diff to your working directory and remove the bad bits of the commit before commiting again what you want
Hi Raffi, Am Mittwoch, 7. Oktober 2015, 17:35:25 schrieb Raffi Enficiaud:
Le 07/10/15 16:44, Sam Kellett a écrit :
out of interest, what's wrong with just reverting the offending commits?
There is a pile of them, coupled with commits I want to keep. revert does just not work, I have to rewind to that commit.
I would not spent extra time (yours and others) for additional work. Just go forward and fix the existing issues. I'll take a look into the command line parser. I hope it is possible to select new/old behaviour depending on some preprocessor #define during compilation. This would allow slow upgrades. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
On 07.10.2015 16:27, Raffi Enficiaud wrote:
Le 07/10/15 12:01, Andrey Semashev a écrit :
git pull --recurse-submodules && git submodule foreach git merge --ff-only
Personally I never checkout a submodule unless I have to work on it, since I do not need them (!),
Well, I'm using quite a few libraries. Selecting ones I don't use and excluding them from the checkout is just a waste of time for me - it's easier to check out everything. In any case, I don't think Boost.Test qualifies as a library that is rarely used.
and a submodule develop/master might not be part of any version of the superproject itself (and hence not being able to test it).
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
Also with your merge you might create new commits that are not on the remote.
No, it can't. Not with the --ff-only argument. And that's the reason why it fails when the history is rewritten.
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
I would mind, that's why I objected in the first place.
Le 07/10/15 16:52, Andrey Semashev a écrit :
Well, I'm using quite a few libraries. Selecting ones I don't use and excluding them from the checkout is just a waste of time for me - it's easier to check out everything. In any case, I don't think Boost.Test qualifies as a library that is rarely used.
I am not saying to not checkout the full superproject with all submodules, I am saying not checking out a particular **branch** of a submodule. This works just fine, and this is what the superproject is supposed to do: having a consistent state with a revision across all components, the revision of the superproject referring to specific revisions of the submodules. git submodule update --recursive checks out everything incl. submodules, submodules are without HEAD since they are SHA1/revisions. This is why the previous command *should* work when history is rewritten (refers to SHA1, does not check out a specific branch).
and a submodule develop/master might not be part of any version of the superproject itself (and hence not being able to test it).
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
You're counting on the fact that things get updated fast, etc etc. What counts is the "VC" part in DVCS: the revision of the superproject and the associated submodules. What you are doing does might lead to an inconsistent state (plus the fact that I do not see any added value in having the branches). But that is another topic.
Also with your merge you might create new commits that are not on the remote.
No, it can't. Not with the --ff-only argument. And that's the reason why it fails when the history is rewritten.
Right.
So far, 1 is against :) (but since you are an experienced git user, you would not mind neither to reset boost.test).
I would mind, that's why I objected in the first place.
I know, and I respect and heard your points. Raffi
On 07.10.2015 18:26, Raffi Enficiaud wrote:
Le 07/10/15 16:52, Andrey Semashev a écrit :
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
You're counting on the fact that things get updated fast, etc etc.
I'm not. I don't use superproject references to commits.
What counts is the "VC" part in DVCS: the revision of the superproject and the associated submodules. What you are doing does might lead to an inconsistent state (plus the fact that I do not see any added value in having the branches). But that is another topic.
If such inconsistent state is possible at a given point then you are not protected from it by using the superproject references to submodules. There's no consistency control in the superproject updates.
Le 07/10/15 17:35, Andrey Semashev a écrit :
On 07.10.2015 18:26, Raffi Enficiaud wrote:
Le 07/10/15 16:52, Andrey Semashev a écrit :
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
You're counting on the fact that things get updated fast, etc etc.
I'm not. I don't use superproject references to commits.
Sorry for asking this question again, but why? I do not see any added value in doing that, on the contrary I see only shortcomings.
What counts is the "VC" part in DVCS: the revision of the superproject and the associated submodules. What you are doing does might lead to an inconsistent state (plus the fact that I do not see any added value in having the branches). But that is another topic.
If such inconsistent state is possible at a given point then you are not protected from it by using the superproject references to submodules. There's no consistency control in the superproject updates.
Yet, what is released, tagged, tested *is* a revision of the superproject.
On 07.10.2015 18:38, Raffi Enficiaud wrote:
Le 07/10/15 17:35, Andrey Semashev a écrit :
On 07.10.2015 18:26, Raffi Enficiaud wrote:
Le 07/10/15 16:52, Andrey Semashev a écrit :
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
You're counting on the fact that things get updated fast, etc etc.
I'm not. I don't use superproject references to commits.
Sorry for asking this question again, but why? I do not see any added value in doing that, on the contrary I see only shortcomings.
As I said, I want my checked out copy of Boost to be ready for development. That's what I use it for, right? I have no use in having a particular revision checked out because if I want to test my code, I want to test it against the actual state of Boost. If I want to modify some code (mine or not) I want to be able to commit and not remember whether I'm in a detached state or not.
Le 07/10/15 17:46, Andrey Semashev a écrit :
On 07.10.2015 18:38, Raffi Enficiaud wrote:
Le 07/10/15 17:35, Andrey Semashev a écrit :
On 07.10.2015 18:26, Raffi Enficiaud wrote:
Le 07/10/15 16:52, Andrey Semashev a écrit :
The superproject is automatically updated to refer to the latest commits in develop and master branches. I think it may skip a few pushed commits if pushes are done in quick succession, but really, I don't care. For all practical purposes you can assume that the superproject always refers to the latest commit.
You're counting on the fact that things get updated fast, etc etc.
I'm not. I don't use superproject references to commits.
Sorry for asking this question again, but why? I do not see any added value in doing that, on the contrary I see only shortcomings.
As I said, I want my checked out copy of Boost to be ready for development. That's what I use it for, right?
Right, git submodule update --recursive git submodule update --remote libs/the_library_on_which_I_am_working is ready for development of the_library_on_which_I_am_working. Since your changes are not spanning the full set of submodules (and there are a lot of warnings in Git when you try to commit in a detached HEAD), this way looks cleaner to me. I have no use in having a
particular revision checked out because if I want to test my code, I want to test it against the actual state of Boost. If I want to modify some code (mine or not) I want to be able to commit and not remember whether I'm in a detached state or not.
My opinion is that once you checkout a develop or master branch of a specific module, you are departing from the master/develop branch of the superproject, and you are promoting yourself as a developer of the submodule. In your case and the commands you are using, you become a developer of all submodules. /Developer/ is not exactly a /user/: in the case of boost.test, you are /not/ a developer, you do not care 99.9% of the time what commit boost.test is pointing to, it is just there and pointing to a revision that you use. This is also the same at the superproject level: let's say I am not "rewriting" the history this time, but pointing the superproject (the .git/modules/test/HEAD) boost.test submodule to the commit before things started to break (this is by design of the git submodule machinery, like svn externals). This is currently impossible because of the bot looking for "develop" and bringing it to the superproject, but this is a plausible scheme for managing submodules and the overall stability of the superproject. In this scheme, your approach does not work and what I was saying previously
You're counting on the fact that things get updated fast, etc etc.
has a bit of truth. Yet this is not the way how boost works today. Anyway, thank you for your time! Kind regards, Raffi
On Thursday 08 October 2015 00:33:22 Raffi Enficiaud wrote:
Le 07/10/15 17:46, Andrey Semashev a écrit :
On 07.10.2015 18:38, Raffi Enficiaud wrote:
Sorry for asking this question again, but why? I do not see any added value in doing that, on the contrary I see only shortcomings.
As I said, I want my checked out copy of Boost to be ready for development. That's what I use it for, right?
Right,
git submodule update --recursive git submodule update --remote libs/the_library_on_which_I_am_working
is ready for development of the_library_on_which_I_am_working. Since your changes are not spanning the full set of submodules (and there are a lot of warnings in Git when you try to commit in a detached HEAD), this way looks cleaner to me.
It doesn't look cleaner to me. I don't commit to every submodule, that's true, but I do make PRs to many. I don't want to type the commands for each and every submodule I happen to edit. Instead I run two commands I mentioned before (which are in update.sh, conveniently lying in my local directory) and get what I want for all submodules.
I have no use in having a particular revision checked out because if I want to test my code, I want to test it against the actual state of Boost. If I want to modify some code (mine or not) I want to be able to commit and not remember whether I'm in a detached state or not.
My opinion is that once you checkout a develop or master branch of a specific module, you are departing from the master/develop branch of the superproject, and you are promoting yourself as a developer of the submodule.
That's your interpretation, but I have no problem with that. I am a developer.
In your case and the commands you are using, you become a developer of all submodules. /Developer/ is not exactly a /user/: in the case of boost.test, you are /not/ a developer, you do not care 99.9% of the time what commit boost.test is pointing to, it is just there and pointing to a revision that you use.
I see what you mean, but I don't consider myself as a blind consumer of other Boost libraries of which I'm not a formal maintainer. If I see a bug in Boost.Test I will probably try to fix it myself and create a PR, if I have time. But we're getting philosophical here.
This is also the same at the superproject level: let's say I am not "rewriting" the history this time, but pointing the superproject (the .git/modules/test/HEAD) boost.test submodule to the commit before things started to break (this is by design of the git submodule machinery, like svn externals).
This is currently impossible because of the bot looking for "develop" and bringing it to the superproject, but this is a plausible scheme for managing submodules and the overall stability of the superproject.
Right. And this is why submodules are badly suited for active development. The design of submodules is good as long as you want a fixed external dependency, which is perfect for snapshots, tags, releases, dependency management and so forth. Though even then I would have second thoughts about using git as a tool for dependency management. As soon as you start developing multiple submodules and try to be on the bleeding edge and keep things working the fixed dependencies only get in the way. This is closer to what is happening in Boost (in my view, at least). The automatic superproject updates are one indication of that, people committing to multiple submodules and doing synchronized changes is another. You may say it's lack of modularization and I'll call it collaboration and historical practice. It doesn't matter. In the end of the day, the way Boost works currently, I don't care if my code worked with 10-commits-old revision of Boost.Test or not - if it doesn't work with HEAD then I have a problem (or you have, if it turns out to be a regression in Boost.Test).
On 07.10.2015 16:27, Raffi Enficiaud wrote:
Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
There's one problem with this approach, if you intend to accept patches or pull requests. What branch are they supposed to be against? So far the general rule was to create PRs against develop. Others have mentioned problems with forks in case when history is rewritten, so maybe PRs will become unusable.
Le 07/10/15 17:02, Andrey Semashev a écrit :
On 07.10.2015 16:27, Raffi Enficiaud wrote:
Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
There's one problem with this approach, if you intend to accept patches or pull requests. What branch are they supposed to be against? So far the general rule was to create PRs against develop. Others have mentioned problems with forks in case when history is rewritten, so maybe PRs will become unusable.
The patch can be rebased manually before being commited. This is what they do in eg. CMake and it is working quite well.
Raffi Enficiaud wrote:
Le 07/10/15 17:02, Andrey Semashev a écrit :
On 07.10.2015 16:27, Raffi Enficiaud wrote:
Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
There's one problem with this approach, if you intend to accept patches or pull requests. What branch are they supposed to be against? So far the general rule was to create PRs against develop. Others have mentioned problems with forks in case when history is rewritten, so maybe PRs will become unusable.
The patch can be rebased manually before being commited. This is what they do in eg. CMake and it is working quite well.
Speaking as an upstream CMake maintainer, this statement is not true. Once a commit is reachable from the master branch it is definitely not rebased. The master branch does not get rewritten. Rewriting long-lived branches is wrong. Temporary branches under review can get rebased. That is very different. Then the temporary branch gets merged to master and the temporary branch is removed. Thanks, Steve.
Le 07/10/15 18:49, Stephen Kelly a écrit :
Raffi Enficiaud wrote:
Le 07/10/15 17:02, Andrey Semashev a écrit :
On 07.10.2015 16:27, Raffi Enficiaud wrote:
Since the organization of boost.test is biased by the runners, our develop branch will become a kind of release branch (like master for other less coupled modules). Our internal develop branch (not called develop but something else) will accept force pushes, but this is fine since it is not meant to be checked out by anyone but boost.test developers.
There's one problem with this approach, if you intend to accept patches or pull requests. What branch are they supposed to be against? So far the general rule was to create PRs against develop. Others have mentioned problems with forks in case when history is rewritten, so maybe PRs will become unusable.
The patch can be rebased manually before being commited. This is what they do in eg. CMake and it is working quite well.
Speaking as an upstream CMake maintainer, this statement is not true. Once a commit is reachable from the master branch it is definitely not rebased. The master branch does not get rewritten. Rewriting long-lived branches is wrong.
I was specifically mentioning the case of PR and patches, hence not yet included in the history: they can be rebased as we want quite easily, although manually (a PR can be downloaded as a patch or pulled into a branch).
Temporary branches under review can get rebased. That is very different. Then the temporary branch gets merged to master and the temporary branch is removed.
Correct, in our case, develop is not a temporary branch and has the same public value as master. I believe this is the case for "next" in CMake as well, which serves a bit as our develop. As soon as there is a merge point from the topic to master or next, it is impossible to rewrite the history of the topic (rebase +/- interactive). Best, Raffi
Raffi Enficiaud wrote:
Le 07/10/15 18:49, Stephen Kelly a écrit :
Raffi Enficiaud wrote:
The patch can be rebased manually before being commited. This is what they do in eg. CMake and it is working quite well.
Speaking as an upstream CMake maintainer, this statement is not true. Once a commit is reachable from the master branch it is definitely not rebased. The master branch does not get rewritten. Rewriting long-lived branches is wrong.
I was specifically mentioning the case of PR and patches, hence not yet included in the history: they can be rebased as we want quite easily, although manually (a PR can be downloaded as a patch or pulled into a branch).
I see. I misread the 'before being committed' part of what you wrote.
Temporary branches under review can get rebased. That is very different. Then the temporary branch gets merged to master and the temporary branch is removed.
Correct, in our case, develop is not a temporary branch and has the same public value as master.
You appear to want to rewrite the history of the develop branch of a git repo. You definitely should not do that. Thanks, Steve.
On 7 October 2015 at 08:25, Raffi Enficiaud
Ok, this is what I wanted to know. Basically I recreated a develop branch for boost.test release to develop (then to master) on a clean history. I wanted to know if the git bot of the modular boost (the one that commits under the name "Automated Commit
") is able to take just the sha of the develop branch, or if its relying on some linear history.
I'm the maintainer of the bot, so I'll add a quick reply. It just reads new commit hashes from the github api, and pays no attention to module history so it has no problems with force pushes. But that doesn't mean they're okay, they can cause all sorts of problems for people who have branches, for the history of the super-project, for scripts which assume a linear history, and for developers who aren't fully versed in git techniques. I haven't read all of the thread, but it looks like this has all been discussed. Also, if you have any further questions about the bot, please make it clear in the subject, so that it will get my attention. I don't read most threads.
Le 07/10/15 23:35, Daniel James a écrit :
On 7 October 2015 at 08:25, Raffi Enficiaud
wrote: Ok, this is what I wanted to know. Basically I recreated a develop branch for boost.test release to develop (then to master) on a clean history. I wanted to know if the git bot of the modular boost (the one that commits under the name "Automated Commit
") is able to take just the sha of the develop branch, or if its relying on some linear history. I'm the maintainer of the bot, so I'll add a quick reply. It just reads new commit hashes from the github api, and pays no attention to module history so it has no problems with force pushes.
This was also my intuition, now I am sure.
But that doesn't mean they're okay, they can cause all sorts of problems for people who have branches, for the history of the super-project, for scripts which assume a linear history, and for developers who aren't fully versed in git techniques. I haven't read all of the thread, but it looks like this has all been discussed.
Indeed, but at least the piece of information you provide on this bot promotes the bot as stable wrt. history rewrite, which was not obvious.
Also, if you have any further questions about the bot, please make it clear in the subject, so that it will get my attention. I don't read most threads.
Sure thanks! Raffi
participants (9)
-
Agustín K-ballo Bergé
-
Andrey Semashev
-
András Kucsma
-
Daniel James
-
Jürgen Hunold
-
Lee Clagett
-
Raffi Enficiaud
-
Sam Kellett
-
Stephen Kelly