[gmx-developers] creating a new branch in the main repository?

Szilárd Páll pall.szilard at gmail.com
Wed Feb 11 00:40:56 CET 2015 

--
Szilárd


On Tue, Feb 10, 2015 at 11:58 PM, Mark Abraham <mark.j.abraham at gmail.com> wrote:
>
>
> On Tue, Feb 10, 2015 at 7:55 PM, Szilárd Páll <pall.szilard at gmail.com>
> wrote:
>>
>> On Tue, Feb 10, 2015 at 6:52 PM, Mark Abraham <mark.j.abraham at gmail.com>
>> wrote:
>> >
>> >
>> > On Tue, Feb 10, 2015 at 5:03 PM, Szilárd Páll <pall.szilard at gmail.com>
>> > wrote:
>> >>
>> >> On Tue, Feb 10, 2015 at 4:27 PM, Mark Abraham
>> >> <mark.j.abraham at gmail.com>
>> >> wrote:
>> >> >
>> >> >
>> >> > On Sun, Feb 8, 2015 at 9:28 PM, Szilárd Páll <pall.szilard at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> I thought, those branches on the main git.gromacs.org are supposed
>> >> >> to
>> >> >> be long-term feature branches that are aimed at merging in the
>> >> >> future,
>> >> >> but I could be wrong.
>> >> >
>> >> >
>> >> > I don't think it matters too much, either way. If people want to
>> >> > clone
>> >> > and
>> >> > don't want others' junk, they can clone from gerrit.gromacs.org, or
>> >> > use
>> >> > git
>> >> > clone --single-branch. Being able to share some gromacs code and have
>> >> > it
>> >> > backed up is just plain useful :-)
>> >>
>> >> Sure, it does not matter us sitting right next to the git server, but
>> >> increasing the size of the repo will matter for devs across the
>> >> continent and will result in lots of !@$#%^ :) - especially as most
>> >> are likely used to simply do
>> >> git clone git://git.gromacs.org/gromacs.
>> >>
>> >> I'm not saying that using clone --single-branch is too difficult, just
>> >> noting the drawbacks. A workaround would likely require setting up a
>> >> different git server.
>> >
>> >
>> > There is only a handful of non-release branches being used in that repo,
>> > and
>> > while I'd love to see that change, I can't see it on the horizon yet. If
>> > download volume becomes a problem someone actually complains about, them
>> > we
>> > can start by making a backup and pruning all the old development
>> > branches.
>>
>> It has been an issue, and people have complained.
>
>
> OK. As already described, there are other solutions besides exacting a
> non-binding promise that any proposed branch in gromacs.git will lead to
> code added to the main repo. I just want people to write code and be happy
> :-)
>
>> (Off topic, but: I don't believe in the "[it] becomes a problem
>> someone actually complains about" -- IMO it does not work, especially
>
>
> If something's a problem for someone, but not so much of a problem that they
> can't write a polite request for something better, then it's not a problem
> for them that we need to solve in advance.

Many won't even know why does their git clone take 10 minutes. Or why
does their mdrun run slower. And many won't dare to try shouting up to
the ivory tower. Enough said, enough times.


>>
>> with the increasingly internalized/private dev channels.)
>
>
> :-) Letting people have branches somewhere promotes collaboration.
>
>> >> >> However, hosting non-gerrit feature branches on gromacs.org is
>> >> >> something I support too!
>> >> >>
>> >> >> At some point in the past we discussed the question of keeping
>> >> >> medium
>> >> >> to long-term feature branches in some centralized location. IIRC the
>> >> >> answer was that there's no need for a secondary server (like
>> >> >> gitolite)
>> >> >> and instead we sohuld use private branches in gerrit. Not sure if
>> >> >> this
>> >> >> ever worked, AFAIR I tried once and failed - the gerrit management
>> >> >> interface is quite unfriendly (and that's an understatement).
>> >> >
>> >> >
>> >> > I used some kind of per-user sandbox section to share a commit with
>> >> > Michael,
>> >> > one time, but I forget the details.
>> >>
>> >> Does anybody know how does that work - if it works at all? We need to
>> >> set up asap a couple of feature branches to do close collaborative
>> >> development and I'd like to decide weather it'll be bitbucket or
>> >> there's a chance to have them hosted in-house.
>> >>
>> >> >>
>> >> >> I would very much like to have reasonably easy way to keep such
>> >> >> development branches maintained by (core?) devs on a central server.
>> >> >> I
>> >> >> can count at least a half dozen feature branches I worked with
>> >> >> recently that live offline, in gerrit drafts, or in 100 PS long
>> >> >> gerrit
>> >> >> "review" changes (which isn't optimal either because history is
>> >> >> impossible to follow). All these could have started out in such
>> >> >> feature branches shared with a few devs with a lifetime of weeks to
>> >> >> months.
>> >> >> I myself keep ending up with dangling drafts and jungle of parents
>> >> >> and
>> >> >> dependent commits. Gerrit review and draft changes are clearly not a
>> >> >> solution for such cases and keeping WIP branches offline is an
>> >> >> obvious
>> >> >> barrier to collaboration.
>> >> >
>> >> >
>> >> > Gerrit's not great for collaborative development, as distinct from
>> >> > code
>> >> > review. However, I'm not aware of an alternative that would also
>> >> > permit
>> >> > using Jenkins. The knowledge that all the commits in the main
>> >> > branches
>> >> > passed the tests of the day is very valuable, so even if we add some
>> >> > merge-style development to our workflow, I think we should strive to
>> >> > keep
>> >> > per-commit testing of the branch to be merged.
>> >>
>> >> I don't think I need per-commit testing of my feature branches and I
>> >> think most others don't need it either. If we work with a couple of
>> >> others closely on a feature, we just want to have a branch where we
>> >> can push regularly without having to ditch commit history. Testing on
>> >> Jenkins is easy, I can simply upload a squashed commit of the branch
>> >> to gerrit as draft. As the feature branch is nearing its EOL, the same
>> >> way it can be pushed up as a single/few commits to gerrit.
>> >
>> >
>> > Sounds like you actually want a distributed version-control system,
>> > rather
>> > than a centrally-hosted feature branch. We can certainly do/set up
>> > something
>> > like
>>
>> More or less, but I personally only want the which is share-able part.
>> I don't care where is it hosted, I just considered a benefits to have
>> it stored centrally by gmx servers.
>
>
> Roland's /refs/privates suggestion does that, and nobody but the author and
> their friends need to know where to go get the code or how to set up a
> branch (I think), so that sounds perfect.

Hopefully. Gerrit as a git server is slow, but hopefully that will be
the only pain.

>> > git push gerrit local-feature-branch:user/sandbox/feature-branch
>> >
>> > so that people can just get the code. They can't browse it or comment on
>> > it
>> > in the style of code review, but it serves the need of a publishing a
>> > branch
>> > where someone can get it.
>>
>> Sure, but last time I tried I failed miserably at setting up such a
>> private branch. Perhaps it's only challenging to me, but honestly, I
>> have not heard many people use it. :)
>>
>> > Two people can collaborate in pull-request
>> > workflow quite happily... that's what git was designed for, of course.
>>
>> I don't think pull requests are ideal for close collaboration, pull
>> requests seems to be tailored for the high latency very distributed
>> collaborative development. There it's often the receiver's problem if
>> e.g. the branch has already moved on and extra conflict resolution is
>
>
> High latency is just a problem for merge resolution, e.g. the many rebases
> of the work to remove md-vv iterated constraints. The choice of merge vs
> rebase vs whatever workflow doesn't matter.

To me it does, but that doesn't matter.

>> needed while merge issues in close collaboration can typically be
>> resolved easier/quicker.
>
>
> I'm confused. If the collaboration is close, more than one person will
> commit, merging will somehow be a pain, and you don't want to rebase patch
> series gerrit-style, how are you actually planning to collaborate? Shared
> code development is intrinsically difficult; a plan for how to do it can
> reduce the pain, but the lunch just cannot be free.

Just need to plan and not throw code around - especially if you well
split responsibilities/tasks. Worked before, should work in the future
too. Even rebase is no big deal, just need to sync up about it.

Gerrit-style is not just rebase, but also squash and forget the
history of the changes to your change-set!

>> >> > We can pick some upcoming feature and have a feature branch in gerrit
>> >> > and
>> >> > see how it works, if you want.
>> >>
>> >> I'm not insisting on having it in gerrit, but I would very much like
>> >> to use feature branches. Using gerrit drafts for collaborative feature
>> >> development has proved to be awful and I really feel that it would be
>> >> much more productive to use branches and when the feature stabilizes
>> >> move the change to gerrit.
>> >
>> >
>> > Sure - but from the point of view of the blessed central repo, that
>> > branch
>> > is a non-entity unless it is merged into master. So the work-in-progress
>> > branch can exist anywhere its collaborators would like.
>>
>> Exactly. I as a "collaborator" asked if we could host such a repo on
>> the gmx infrastructure.
>
>
> Sure, we have them already in gerrit's /refs/private, now that Roland's
> prompted my memory.
>
>> >> > Probably we would want to prohibit any
>> >> > rebasing of non-tip commits on that branch (else there's the
>> >> > ripple-and-re-verify effect). But would that add value vs using a
>> >> > gerrit
>> >> > topic branch? The latter is almost zero work...
>> >>
>> >> I'm not sure I get this last part, I am admittedly a bit lacking in
>> >> advanced git knowledge.
>> >
>> >
>> > It's a gerrit feature, not git. For example, someone (probably me) put a
>> > bunch of your recent CUDA work onto the cuda topic, e.g.
>> > https://gerrit.gromacs.org/#/q/topic:cuda which is potentially useful
>> > for
>> > someone finding related work that is past. All you do is fill in the box
>> > below project and branch on gerrit after you upload (or use the right
>> > secret
>> > sauce when uploading). On the main page you can see "master (cuda)" for
>> > the
>> > "branch" for the remaining commit from that group.
>>
>> I knew about "topic"s but not sure what's "branch" about them. I guess
>> I'll chew on this and get back later.
>
>
> As you know, a branch is a basic git feature - you have a series of
> parent-child commits and you can choose to give it a name. A gerrit topic is
> a set of patches that have a common theme. Often, both will go together,
> because later things in the theme depend on other things in the theme, thus
> "gerrit topic branch." For example, I used one called g-tune-pme-reform, but
> the low of enthusiasm for reviewing the code changes meant it went nowhere.

Sure, I did get that part. I got lost when you wrote:
"we would want to prohibit any rebasing of non-tip commits on that
branch (else there's the ripple-and-re-verify effect)"

>> Re: "value vs using a gerrit topic branch"
>> In gerrit I see branches as inherently single-commit length (you can't
>> post a fix without squashing it) with any additional dependent changes
>> chained exponentially increasing the pain of maintaining things - at
>> least this is my experience.
>
>
> OK, but if you don't want to use pull requests or gerrit-style patch
> updates, how are you going to work collaboratively?

Those are the only two ways? <scratch head/> As explained before: I
want a branch where fixing a minor screwup does not require squashing
the fix, where a fix does not vanish but a record of it is kept (as a
commit), where I don't have to pus into a repo where unstable things
will hurt others, etc.

> Mark
>
>> >
>> >> To conclude, I'm not trying to force anyone to change their work
>> >> pattern, but there are a few smaller ongoing and about to start
>> >> projects that I'd like do differently - and there seems to be interest
>> >> from others in employing similar workflow.
>> >>
>> >> The barrier to this would perhaps be the branch management and access
>> >> control, and AFAIR that's why it was the rather awkward gerrit-based
>> >> solution that some proposed.
>> >>
>> >> Does anybody know of a reasonably simple alternative? One thing that
>> >> comes to my mind is that if a gitolite (web?) management interface
>> >> does exists, we could have that on top of git.gromacs.org.
>> >
>> >
>> > Seems like the thing we want is a lightweight way to share a branch, and
>> > perhaps to be able to comment on it on some website. github makes this
>> > easy,
>> > but I'm not sure what the best solution is for us.
>>
>> Commenting is not a deal-breaker feature - at least not to me. Sure,
>> github work, so does bitbucket.
>> --
>> Szilárd
>>
>> > Cheers,
>> >
>> > Mark
>> >
>> >>
>> >> Cheers,
>> >> --
>> >> Szilárd
>> >>
>> >> > Mark
>> >> >
>> >> >> Cheers,
>> >> >> --
>> >> >> Szilárd
>> >> >>
>> >> >>
>> >> >> On Sat, Feb 7, 2015 at 3:47 AM, Mark Abraham
>> >> >> <mark.j.abraham at gmail.com>
>> >> >> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> > Rossen is the guru here, but we administer git.gromacs.org via a
>> >> >> > gitolite
>> >> >> > server. There are various private branches in gromacs.git to which
>> >> >> > people
>> >> >> > have direct push access; only Gerrit can push to the main
>> >> >> > branches.
>> >> >> > So
>> >> >> > if
>> >> >> > you let Rossen know what branch name would be good to create, and
>> >> >> > public
>> >> >> > keys for whoever should have push access, that'll be fine.
>> >> >> >
>> >> >> > Cheers,
>> >> >> >
>> >> >> > Mark
>> >> >> >
>> >> >> > On Sat, Feb 7, 2015 at 5:20 AM, Shirts, Michael R. (mrs5pt)
>> >> >> > <mrs5pt at eservices.virginia.edu> wrote:
>> >> >> >>
>> >> >> >> Apologies if this should have been obvious and I could figure it
>> >> >> >> out.
>> >> >> >>
>> >> >> >> I want to create a new branch in the main repository so I can
>> >> >> >> share
>> >> >> >> a
>> >> >> >> particular functionality that will never make it into the main
>> >> >> >> repository
>> >> >> >> (too much complication, only useable in some very restrictive
>> >> >> >> cases,
>> >> >> >> long
>> >> >> >> story).  I tried googling around, but couldn't find something
>> >> >> >> that
>> >> >> >> made
>> >> >> >> sense. git push origin NEWBRANCH gave me a 'fatal: remote error:
>> >> >> >> access
>> >> >> >> denied or repository not exported: /gromacs.git' error - was this
>> >> >> >> because
>> >> >> >> my permissions weren't set up right?  Should I just set up a
>> >> >> >> private
>> >> >> >> branch?
>> >> >> >>
>> >> >> >> Thanks for any advice!
>> >> >> >>
>> >> >> >> Best,
>> >> >> >> ~~~~~~~~~~~~
>> >> >> >> Michael Shirts
>> >> >> >> Associate Professor
>> >> >> >> Department of Chemical Engineering
>> >> >> >> University of Virginia
>> >> >> >> michael.shirts at virginia.edu
>> >> >> >> (434) 243-1821
>> >> >> >>
>> >> >> >> --
>> >> >> >> Gromacs Developers mailing list
>> >> >> >>
>> >> >> >> * Please search the archive at
>> >> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>> >> >> >> before
>> >> >> >> posting!
>> >> >> >>
>> >> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >> >>
>> >> >> >> * For (un)subscribe requests visit
>> >> >> >>
>> >> >> >>
>> >> >> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >> >> >> or
>> >> >> >> send a mail to gmx-developers-request at gromacs.org.
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Gromacs Developers mailing list
>> >> >> >
>> >> >> > * Please search the archive at
>> >> >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>> >> >> > before
>> >> >> > posting!
>> >> >> >
>> >> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >> >
>> >> >> > * For (un)subscribe requests visit
>> >> >> >
>> >> >> >
>> >> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >> >> > or
>> >> >> > send a mail to gmx-developers-request at gromacs.org.
>> >> >> --
>> >> >> Gromacs Developers mailing list
>> >> >>
>> >> >> * Please search the archive at
>> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>> >> >> before
>> >> >> posting!
>> >> >>
>> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >>
>> >> >> * For (un)subscribe requests visit
>> >> >>
>> >> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >> >> or
>> >> >> send a mail to gmx-developers-request at gromacs.org.
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Gromacs Developers mailing list
>> >> >
>> >> > * Please search the archive at
>> >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>> >> > before
>> >> > posting!
>> >> >
>> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >> >
>> >> > * For (un)subscribe requests visit
>> >> >
>> >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >> > or
>> >> > send a mail to gmx-developers-request at gromacs.org.
>> >> --
>> >> Gromacs Developers mailing list
>> >>
>> >> * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> >> posting!
>> >>
>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >>
>> >> * For (un)subscribe requests visit
>> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> >> or
>> >> send a mail to gmx-developers-request at gromacs.org.
>> >
>> >
>> >
>> > --
>> > Gromacs Developers mailing list
>> >
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> > posting!
>> >
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>> > * For (un)subscribe requests visit
>> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>> > or
>> > send a mail to gmx-developers-request at gromacs.org.
>> --
>> Gromacs Developers mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or
>> send a mail to gmx-developers-request at gromacs.org.
>
>
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or
> send a mail to gmx-developers-request at gromacs.org.

More information about the gromacs.org_gmx-developers mailing list