[gmx-developers] creating a new branch in the main repository?

[Mark Abraham]

On Tue, Feb 10, 2015 at 7:55 PM, Szilárd Páll <pall.szilard at gmail.com>
wrote:

> On Tue, Feb 10, 2015 at 6:52 PM, Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
> >
> >
> > On Tue, Feb 10, 2015 at 5:03 PM, Szilárd Páll <pall.szilard at gmail.com>
> > wrote:
> >>
> >> On Tue, Feb 10, 2015 at 4:27 PM, Mark Abraham <mark.j.abraham at gmail.com
> >
> >> wrote:
> >> >
> >> >
> >> > On Sun, Feb 8, 2015 at 9:28 PM, Szilárd Páll <pall.szilard at gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I thought, those branches on the main git.gromacs.org are supposed
> to
> >> >> be long-term feature branches that are aimed at merging in the
> future,
> >> >> but I could be wrong.
> >> >
> >> >
> >> > I don't think it matters too much, either way. If people want to clone
> >> > and
> >> > don't want others' junk, they can clone from gerrit.gromacs.org, or
> use
> >> > git
> >> > clone --single-branch. Being able to share some gromacs code and have
> it
> >> > backed up is just plain useful :-)
> >>
> >> Sure, it does not matter us sitting right next to the git server, but
> >> increasing the size of the repo will matter for devs across the
> >> continent and will result in lots of !@$#%^ :) - especially as most
> >> are likely used to simply do
> >> git clone git://git.gromacs.org/gromacs.
> >>
> >> I'm not saying that using clone --single-branch is too difficult, just
> >> noting the drawbacks. A workaround would likely require setting up a
> >> different git server.
> >
> >
> > There is only a handful of non-release branches being used in that repo,
> and
> > while I'd love to see that change, I can't see it on the horizon yet. If
> > download volume becomes a problem someone actually complains about, them
> we
> > can start by making a backup and pruning all the old development
> branches.
>
> It has been an issue, and people have complained.
>

OK. As already described, there are other solutions besides exacting a
non-binding promise that any proposed branch in gromacs.git will lead to
code added to the main repo. I just want people to write code and be happy
:-)

(Off topic, but: I don't believe in the "[it] becomes a problem
> someone actually complains about" -- IMO it does not work, especially
>

If something's a problem for someone, but not so much of a problem that
they can't write a polite request for something better, then it's not a
problem for them that we need to solve in advance.


> with the increasingly internalized/private dev channels.)


:-) Letting people have branches somewhere promotes collaboration.

>> >> However, hosting non-gerrit feature branches on gromacs.org is
> >> >> something I support too!
> >> >>
> >> >> At some point in the past we discussed the question of keeping medium
> >> >> to long-term feature branches in some centralized location. IIRC the
> >> >> answer was that there's no need for a secondary server (like
> gitolite)
> >> >> and instead we sohuld use private branches in gerrit. Not sure if
> this
> >> >> ever worked, AFAIR I tried once and failed - the gerrit management
> >> >> interface is quite unfriendly (and that's an understatement).
> >> >
> >> >
> >> > I used some kind of per-user sandbox section to share a commit with
> >> > Michael,
> >> > one time, but I forget the details.
> >>
> >> Does anybody know how does that work - if it works at all? We need to
> >> set up asap a couple of feature branches to do close collaborative
> >> development and I'd like to decide weather it'll be bitbucket or
> >> there's a chance to have them hosted in-house.
> >>
> >> >>
> >> >> I would very much like to have reasonably easy way to keep such
> >> >> development branches maintained by (core?) devs on a central server.
> I
> >> >> can count at least a half dozen feature branches I worked with
> >> >> recently that live offline, in gerrit drafts, or in 100 PS long
> gerrit
> >> >> "review" changes (which isn't optimal either because history is
> >> >> impossible to follow). All these could have started out in such
> >> >> feature branches shared with a few devs with a lifetime of weeks to
> >> >> months.
> >> >> I myself keep ending up with dangling drafts and jungle of parents
> and
> >> >> dependent commits. Gerrit review and draft changes are clearly not a
> >> >> solution for such cases and keeping WIP branches offline is an
> obvious
> >> >> barrier to collaboration.
> >> >
> >> >
> >> > Gerrit's not great for collaborative development, as distinct from
> code
> >> > review. However, I'm not aware of an alternative that would also
> permit
> >> > using Jenkins. The knowledge that all the commits in the main branches
> >> > passed the tests of the day is very valuable, so even if we add some
> >> > merge-style development to our workflow, I think we should strive to
> >> > keep
> >> > per-commit testing of the branch to be merged.
> >>
> >> I don't think I need per-commit testing of my feature branches and I
> >> think most others don't need it either. If we work with a couple of
> >> others closely on a feature, we just want to have a branch where we
> >> can push regularly without having to ditch commit history. Testing on
> >> Jenkins is easy, I can simply upload a squashed commit of the branch
> >> to gerrit as draft. As the feature branch is nearing its EOL, the same
> >> way it can be pushed up as a single/few commits to gerrit.
> >
> >
> > Sounds like you actually want a distributed version-control system,
> rather
> > than a centrally-hosted feature branch. We can certainly do/set up
> something
> > like
>
> More or less, but I personally only want the which is share-able part.
> I don't care where is it hosted, I just considered a benefits to have
> it stored centrally by gmx servers.


Roland's /refs/privates suggestion does that, and nobody but the author and
their friends need to know where to go get the code or how to set up a
branch (I think), so that sounds perfect.

> git push gerrit local-feature-branch:user/sandbox/feature-branch
> >
> > so that people can just get the code. They can't browse it or comment on
> it
> > in the style of code review, but it serves the need of a publishing a
> branch
> > where someone can get it.
>
> Sure, but last time I tried I failed miserably at setting up such a
> private branch. Perhaps it's only challenging to me, but honestly, I
> have not heard many people use it. :)
>
> > Two people can collaborate in pull-request
> > workflow quite happily... that's what git was designed for, of course.
>
> I don't think pull requests are ideal for close collaboration, pull
> requests seems to be tailored for the high latency very distributed
> collaborative development. There it's often the receiver's problem if
> e.g. the branch has already moved on and extra conflict resolution is
>

High latency is just a problem for merge resolution, e.g. the many rebases
of the work to remove md-vv iterated constraints. The choice of merge vs
rebase vs whatever workflow doesn't matter.

needed while merge issues in close collaboration can typically be
> resolved easier/quicker.


I'm confused. If the collaboration is close, more than one person will
commit, merging will somehow be a pain, and you don't want to rebase patch
series gerrit-style, how are you actually planning to collaborate? Shared
code development is intrinsically difficult; a plan for how to do it can
reduce the pain, but the lunch just cannot be free.

>> > We can pick some upcoming feature and have a feature branch in gerrit
> >> > and
> >> > see how it works, if you want.
> >>
> >> I'm not insisting on having it in gerrit, but I would very much like
> >> to use feature branches. Using gerrit drafts for collaborative feature
> >> development has proved to be awful and I really feel that it would be
> >> much more productive to use branches and when the feature stabilizes
> >> move the change to gerrit.
> >
> >
> > Sure - but from the point of view of the blessed central repo, that
> branch
> > is a non-entity unless it is merged into master. So the work-in-progress
> > branch can exist anywhere its collaborators would like.
>
> Exactly. I as a "collaborator" asked if we could host such a repo on
> the gmx infrastructure.


Sure, we have them already in gerrit's /refs/private, now that Roland's
prompted my memory.

>> > Probably we would want to prohibit any
> >> > rebasing of non-tip commits on that branch (else there's the
> >> > ripple-and-re-verify effect). But would that add value vs using a
> gerrit
> >> > topic branch? The latter is almost zero work...
> >>
> >> I'm not sure I get this last part, I am admittedly a bit lacking in
> >> advanced git knowledge.
> >
> >
> > It's a gerrit feature, not git. For example, someone (probably me) put a
> > bunch of your recent CUDA work onto the cuda topic, e.g.
> > https://gerrit.gromacs.org/#/q/topic:cuda which is potentially useful
> for
> > someone finding related work that is past. All you do is fill in the box
> > below project and branch on gerrit after you upload (or use the right
> secret
> > sauce when uploading). On the main page you can see "master (cuda)" for
> the
> > "branch" for the remaining commit from that group.
>
> I knew about "topic"s but not sure what's "branch" about them. I guess
> I'll chew on this and get back later.
>

As you know, a branch is a basic git feature - you have a series of
parent-child commits and you can choose to give it a name. A gerrit topic
is a set of patches that have a common theme. Often, both will go together,
because later things in the theme depend on other things in the theme, thus
"gerrit topic branch." For example, I used one called g-tune-pme-reform,
but the low of enthusiasm for reviewing the code changes meant it went
nowhere.

Re: "value vs using a gerrit topic branch"
> In gerrit I see branches as inherently single-commit length (you can't
> post a fix without squashing it) with any additional dependent changes
> chained exponentially increasing the pain of maintaining things - at
> least this is my experience.


OK, but if you don't want to use pull requests or gerrit-style patch
updates, how are you going to work collaboratively?

Mark

>
> >> To conclude, I'm not trying to force anyone to change their work
> >> pattern, but there are a few smaller ongoing and about to start
> >> projects that I'd like do differently - and there seems to be interest
> >> from others in employing similar workflow.
> >>
> >> The barrier to this would perhaps be the branch management and access
> >> control, and AFAIR that's why it was the rather awkward gerrit-based
> >> solution that some proposed.
> >>
> >> Does anybody know of a reasonably simple alternative? One thing that
> >> comes to my mind is that if a gitolite (web?) management interface
> >> does exists, we could have that on top of git.gromacs.org.
> >
> >
> > Seems like the thing we want is a lightweight way to share a branch, and
> > perhaps to be able to comment on it on some website. github makes this
> easy,
> > but I'm not sure what the best solution is for us.
>
> Commenting is not a deal-breaker feature - at least not to me. Sure,
> github work, so does bitbucket.
> --
> Szilárd
>
> > Cheers,
> >
> > Mark
> >
> >>
> >> Cheers,
> >> --
> >> Szilárd
> >>
> >> > Mark
> >> >
> >> >> Cheers,
> >> >> --
> >> >> Szilárd
> >> >>
> >> >>
> >> >> On Sat, Feb 7, 2015 at 3:47 AM, Mark Abraham <
> mark.j.abraham at gmail.com>
> >> >> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > Rossen is the guru here, but we administer git.gromacs.org via a
> >> >> > gitolite
> >> >> > server. There are various private branches in gromacs.git to which
> >> >> > people
> >> >> > have direct push access; only Gerrit can push to the main branches.
> >> >> > So
> >> >> > if
> >> >> > you let Rossen know what branch name would be good to create, and
> >> >> > public
> >> >> > keys for whoever should have push access, that'll be fine.
> >> >> >
> >> >> > Cheers,
> >> >> >
> >> >> > Mark
> >> >> >
> >> >> > On Sat, Feb 7, 2015 at 5:20 AM, Shirts, Michael R. (mrs5pt)
> >> >> > <mrs5pt at eservices.virginia.edu> wrote:
> >> >> >>
> >> >> >> Apologies if this should have been obvious and I could figure it
> >> >> >> out.
> >> >> >>
> >> >> >> I want to create a new branch in the main repository so I can
> share
> >> >> >> a
> >> >> >> particular functionality that will never make it into the main
> >> >> >> repository
> >> >> >> (too much complication, only useable in some very restrictive
> cases,
> >> >> >> long
> >> >> >> story).  I tried googling around, but couldn't find something that
> >> >> >> made
> >> >> >> sense. git push origin NEWBRANCH gave me a 'fatal: remote error:
> >> >> >> access
> >> >> >> denied or repository not exported: /gromacs.git' error - was this
> >> >> >> because
> >> >> >> my permissions weren't set up right?  Should I just set up a
> private
> >> >> >> branch?
> >> >> >>
> >> >> >> Thanks for any advice!
> >> >> >>
> >> >> >> Best,
> >> >> >> ~~~~~~~~~~~~
> >> >> >> Michael Shirts
> >> >> >> Associate Professor
> >> >> >> Department of Chemical Engineering
> >> >> >> University of Virginia
> >> >> >> michael.shirts at virginia.edu
> >> >> >> (434) 243-1821
> >> >> >>
> >> >> >> --
> >> >> >> Gromacs Developers mailing list
> >> >> >>
> >> >> >> * Please search the archive at
> >> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> >> >> >> before
> >> >> >> posting!
> >> >> >>
> >> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >> >>
> >> >> >> * For (un)subscribe requests visit
> >> >> >>
> >> >> >>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >> >> or
> >> >> >> send a mail to gmx-developers-request at gromacs.org.
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Gromacs Developers mailing list
> >> >> >
> >> >> > * Please search the archive at
> >> >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> >> >> > before
> >> >> > posting!
> >> >> >
> >> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >> >
> >> >> > * For (un)subscribe requests visit
> >> >> >
> >> >> >
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >> > or
> >> >> > send a mail to gmx-developers-request at gromacs.org.
> >> >> --
> >> >> Gromacs Developers mailing list
> >> >>
> >> >> * Please search the archive at
> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> before
> >> >> posting!
> >> >>
> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >>
> >> >> * For (un)subscribe requests visit
> >> >>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >> or
> >> >> send a mail to gmx-developers-request at gromacs.org.
> >> >
> >> >
> >> >
> >> > --
> >> > Gromacs Developers mailing list
> >> >
> >> > * Please search the archive at
> >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> before
> >> > posting!
> >> >
> >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >
> >> > * For (un)subscribe requests visit
> >> >
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> > or
> >> > send a mail to gmx-developers-request at gromacs.org.
> >> --
> >> Gromacs Developers mailing list
> >>
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> >> posting!
> >>
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> >> * For (un)subscribe requests visit
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or
> >> send a mail to gmx-developers-request at gromacs.org.
> >
> >
> >
> > --
> > Gromacs Developers mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or
> > send a mail to gmx-developers-request at gromacs.org.
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20150210/8b62df8b/attachment-0001.html>