[gmx-developers] slightly odd-looking code
Mark Abraham
mark.j.abraham at gmail.com
Wed Mar 25 10:45:20 CET 2020
Hi,
github is just a mirror, and it looks like something stopped pushing the
tags to it about a year ago. https://gitlab.com/gromacs/gromacs is the
(brand new!) primary, so use that.
mark
On Wed, 25 Mar 2020 at 10:41, jan <rtm443x at googlemail.com> wrote:
> Hi,
> there is no tag I can find that has '2020' in it.
> I saw the v20xx.y pattern, with 'v2019.1' being the most recent
> looking, but otherwise no. This is the code cloned from github BTW,
> just in case there's another git repo.
> Could I ask that someone just check and see if I've missed it.
>
> Regarding the WSL, thanks I'll look into that!
>
> cheers
>
> jan
>
>
>
> On 24/03/2020, Szilárd Páll <pall.szilard at gmail.com> wrote:
> > Hi,
> >
> > On Tue, Mar 24, 2020 at 11:24 PM jan <rtm443x at googlemail.com> wrote:
> >
> >> Hi all,
> >> I got the git repo from <https://github.com/gromacs/gromacs>, used
> >> HEAD and started compiling.
> >>
> >> I guess I should use the right tag, bit I could not see a tag that
> >> looked like the release the tarball says it is, which is 2020.1 so I
> >> just compiled it (with cygwin) even though it's had commits more
> >> recently.
> >>
> >
> > Release maintenance branches are named "release-VERSION", so the
> > release-2020 branch is a good stable place to start from.
> > Alternatively, releases are tagged with tags prefixed by "v", so v2020.1
> > for the 2020.1 release.
> >
> >
> >> Anyway I hit this:
> >>
> >>
> >>
> /cygdrive/c/Users/jan/Desktop/gromacs_git/gromacs/src/external/thread_mpi/src/tmpi_init.cpp:476:42:
> >> error: ‘strdup’ was not declared in this scope; did you mean ‘strcmp’?
> >> 476 | threads[i].argv[j] = strdup( (*argv)[j] );
> >>
> >> which comes from
> >>
> >> #if !(defined( _WIN32 ) || defined( _WIN64 ) )
> >> threads[i].argv[j] = strdup( (*argv)[j] ); // <--- this line
> >> #else
> >> threads[i].argv[j] = _strdup( (*argv)[j] );
> >> #endif
> >>
> >> so it looks like it didn't pick up I'm running in windows.
> >> I'm not so familiar with make (or these build tools generally), but I
> >> poked around in the makefile (generated by cmake I assume) but could
> >> not see where to add the symbol for _WIN64
> >>
> >
> > AFAICT the _WIN64 macro should be defined only if code uses 64-bit
> Windows
> > API, see
> > https://www.cygwin.com/faq.html#faq.programming.64bitporting-cygwin64
> >
> > As a side-note, there are a few reports/tutorials on compiling GROMACS on
> > cygwin I know of, but may be a bit dated:
> > http://tmacchant3.starfree.jp/gromacs/win/notes_cygwin.html
> > http://cdlc.cau.ac.kr/Gromacs/966
> >
> > Since I understand it has been compiled and tested under MSVC2017
> >> I'll try that tomorrow.
> >>
> >
> > Native MSVC should be easier (IIRC fftw might require a separate
> download),
> > but a possibly even easier option is to just use WSL. After installing
> the
> > Linux subsystem and built toolchain, you should be able to just follow
> the
> > regular user guide installation procedure.
> >
> > Cheers,
> > --
> > Szilárd
> >
> >
> >> cheers
> >>
> >> jan
> >>
> >>
> >>
> >> On 24/03/2020, Paul Bauer <paul.bauer.q at gmail.com> wrote:
> >> > Hello,
> >> >
> >> > I agree with Mark that the webinars should be a good start to have an
> >> idea
> >> > a out the code.
> >> >
> >> > Concerning the error you are getting, this shouldn't happen if you
> work
> >> and
> >> > build from a git repository. But it is still something I think should
> >> > be
> >> > fixed (especially because I have been the one pushing for it against
> >> Mark's
> >> > objections)
> >> >
> >> > Cheers
> >> >
> >> > Paul
> >> >
> >> > On Tue, 24 Mar 2020, 19:50 Mark Abraham, <mark.j.abraham at gmail.com>
> >> wrote:
> >> >
> >> >> Hi gmx developers!
> >> >>
> >> >>
> >> >> On Tue, 24 Mar 2020 at 15:29, jan <rtm443x at googlemail.com> wrote:
> >> >>
> >> >>> Hi all,
> >> >>> given what's said below I need to be clear about where I am.
> >> >>> I'm a back-end dev specialising somewhat in SQL/RDBMSs + data
> >> >>> management (but not data analysis of any note), with plenty of
> >> >>> experience of other languages etc. however I have never done any
> >> >>> x86/x64 as such, I normally use high-level languages, I have no
> >> >>> experience at all with GPUs and (as already mentioned) my C/C++ is
> >> >>> neolithic, and I've no experience with the modern c/c++ build tools.
> >> >>> My linux is not great. And the nearest I can get to your kind of
> >> >>> mathematics is a background in classical logic ie. not very close at
> >> >>> all.
> >> >>>
> >> >>> None of these bother me; all are fixable, but it will take time.
> I'll
> >> >>> need a minimal amount of guidance to get gowing - "read this" is
> fine
> >> >>> but I'll need no handholding, and you don't have time for that
> >> >>> anyway.
> >> >>>
> >> >>
> >> >> Great - you're coming at the questions from a totally different
> >> >> perspective, which is healthy for everyone, but that's going to give
> >> >> you
> >> >> a
> >> >> steep learning curve. There's some useful recorded webinars from
> >> BioExcel
> >> >> given by particularly Paul, Szilard, Carsten, and I over recent years
> >> >> that
> >> >> are a good starting point for understanding how the code operates at
> >> >> run
> >> >> time, but you should look for something else for "intro to molecular
> >> >> dynamics for non-scientists." There's a bunch of material online -
> has
> >> >> anybody got suggestions?
> >> >>
> >> >> First things first, I need to compile this stuff up.
> >> >>> I'm using windows + cygwin - is this the best environment or do you
> >> >>> recommend working entirely in Linux?
> >> >>>
> >> >>
> >> >> We test in CI on Windows + MSVC, so that works fine, but there has
> >> >> been
> >> >> no
> >> >> love on cygwin for about a decade. It's almost certainly fixable, but
> >> >> never
> >> >> been a priority. So go native one way or the other :-)
> >> >>
> >> >> I've followed the instructions to get it working on cygwin and got
> >> >>> stuck. Compilation fails with
> >> >>>
> >> >>> C:\Program Files\Python38\python.exe: can't open file
> >> >>>
> >> >>>
> >>
> '/cygdrive/c/Users/jan/Desktop/gromacs/gromacs-2020.1/admin/createFileHash.py':
> >> >>> [Errno 2] No such file or directory
> >> >>> CMake Error at cmake/gmxGenerateVersionInfoRelease.cmake:115 (file):
> >> >>> file failed to open for reading (No such file or directory):
> >> >>>
> >> >>>
> /cygdrive/c/Users/jan/Desktop/gromacs/gromacs-2020.1/computed_checksum
> >> >>>
> >> >>> computed_checksum indeed doesn't exist. What now?
> >> >>>
> >> >>
> >> >> Sigh, that's broken implementation of a new feature that I never
> >> >> thought
> >> >> was worth its cost. Don't know how to fix it.
> >> >>
> >> >> The above is going by
> >> >>> <
> >>
> http://manual.gromacs.org/documentation/current/install-guide/index.html
> >> >>> >
> >> >>>
> >> >>> Other problem is that untarring the tarball works, however on
> windows
> >> >>> it's natural to use something like 7-zip. This doesn't work and I
> >> >>> lost
> >> >>> a couple of hours to that (certainly not the fault of the
> >> >>> instructions
> >> >>> but FYI anyway).
> >> >>>
> >> >>> Comments below too
> >> >>>
> >> >>>
> >> >>> On 24/03/2020, Mark Abraham <mark.j.abraham at gmail.com> wrote:
> >> >>> > Hi,
> >> >>> >
> >> >>> > Jan, the biggest bang-for-buck optimizations relevant to
> >> >>> > Folding at Home
> >> >>> are
> >> >>> > to
> >> >>> >
> >> >>> > a) offer to build them an OpenCL-enabled GROMACS "core" (ie the
> >> >>> > version
> >> >>> of
> >> >>> > GROMACS that they run, when they run GROMACS). Currently they seem
> >> >>> > to
> >> >>> run
> >> >>> > all GPU jobs with OpenCL and OpenMM, which is nice but leaves a
> lot
> >> of
> >> >>> > throughput on the table. The GROMACS OpenCL port is mature and
> >> stable,
> >> >>> runs
> >> >>> > on AMD/NVIDIA/Intel current GPUs, and should present no more
> >> >>> > driver/user
> >> >>> > problems than their OpenMM one. Their concept of a GPU slot is a
> >> >>> > single
> >> >>> GPU
> >> >>> > accompanied by a single CPU thread/, whereas the GROMACS OpenCL
> >> >>> > port
> >> >>> would
> >> >>> > prefer multiple dedicated cores. That's still better than leaving
> >> GPUs
> >> >>> > empty if there's not enough OpenMM jobs in the queue, though the
> >> >>> > actual
> >> >>> > performance will be woeful compared to GROMACS when you give it a
> >> >>> healthy
> >> >>> > chunk of CPU cores also. Could even be better than OpenMM's GPU
> >> >>> > core,
> >> >>> > depending how modern that one is, too ;-) The GROMACS CUDA port is
> >> >>> better
> >> >>> > still (and in 2020 can do a decent job even with only a single CPU
> >> >>> core),
> >> >>> > but they have made a philosophical choice to use OpenCL only.
> >> >>>
> >> >>> That has to come later when I get up to speed, but carefully noted,
> >> >>> thanks.
> >> >>>
> >> >>> > b) update the GROMACS CPU core in F at H because the one used in F at H
> >> >>> > is
> >> >>> > several years behind and losing the benefit of the hard
> >> >>> > optimization
> >> >>> work
> >> >>> > that we've done in the meantime.
> >> >>>
> >> >>> Why would f at h not do this already??
> >> >>>
> >> >>
> >> >> Limited resources and priorities for it. It's a science-driven
> >> >> project,
> >> >> and if the people prepared to do the work want to use not-GROMACS for
> >> >> their
> >> >> science then that is that...
> >> >>
> >> >> But again, noted.
> >> >>>
> >> >>> > c) demonstrate that they can maintainably and usefully offer more
> >> than
> >> >>> two
> >> >>> > x86 builds of that GROMACS CPU core (GROMACS has lots of SIMD
> >> >>> specialized
> >> >>> > flavours, but F at H only offers SSE4.1 and basic AVX from those
> >> >>> > flavours,
> >> >>>
> >> >>> Yes, thought this might be the case. Definitely worth it for newer
> >> >>> chips.
> >> >>> However, please note that SIMD performance for later chips do not
> >> >>> always mix well with non-SIMD code and can overall *cost*
> performance
> >> >>> <
> >> https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/
> >
> >> >>>
> >> >>
> >> >> Yes thanks, most of us know ;-) Just updating to add AVX2 would give
> a
> >> >> clear win.
> >> >>
> >> >>
> >> >>> > which leaves a lot of performance on the table on recent x86 CPUs.
> >> >>> > We
> >> >>> > already have all the logic needed to work out which pre-built
> >> >>> > GROMACS
> >> >>> > to
> >> >>> > download and run, because we use it in containerized GROMACS
> builds
> >> >>> also.)
> >> >>> >
> >> >>> > Unfortunately they've never open-sourced any of that, so finding
> >> >>> > out
> >> >>> where
> >> >>> > to start is the first challenge. But that way you'll have a lot
> >> >>> > more
> >> >>> impact
> >> >>> > sooner than you will from profiling GROMACS runs after 30 years of
> >> >>> > optimization. ;-)
> >> >>>
> >> >>> I dunno yet. Model tuning is beyond me obviously, but I've seen some
> >> >>> stuff in the code that I question WRT optimality. However if it's
> >> >>> cold
> >> >>> code or all memory bound then I'll be fixing the wrong thing. Time
> to
> >> >>> profile, but need to get it to compile first.
> >> >>>
> >> >>
> >> >> Memory? What's that? :-D GROMACS memory usage is typically measured
> in
> >> >> megabytes, with sophisticated data-parallelism to keep the working
> set
> >> >> for
> >> >> each core down around cache sizes. Obviously you can scale up the
> >> problem
> >> >> to get out of cache, but the problem sizes that suit interesting
> >> >> science
> >> >> are comparable with the amount of L3 cache you get on a socket these
> >> >> days.
> >> >>
> >> >> There's a big pile of code in the repo that warrants exhaustive
> >> >> optimization, and a lot that is used by only a handful of people,
> >> >> which
> >> >> generally doesn't. It's hard to make a valuable impact in either kind
> >> >> of
> >> >> place, for different reasons.
> >> >>
> >> >> Mark
> >> >>
> >> >> Happy to take this offline and reduce mailing list clutter.
> >> >>>
> >> >>> cheers
> >> >>>
> >> >>> jan
> >> >>>
> >> >>> >
> >> >>> > Mark
> >> >>> >
> >> >>> > On Mon, 23 Mar 2020 at 14:59, jan <rtm443x at googlemail.com> wrote:
> >> >>> >
> >> >>> >> Hi,
> >> >>> >> I'm a general back-end dev. Given the situation, and
> folding at home
> >> >>> >> using gromacs, I thought I'd poke through the code. I noticed
> >> >>> >> something unexpected, and was advised to email it here. in
> >> edsam.cpp,
> >> >>> >> this:
> >> >>> >>
> >> >>> >>
> >> >>> >> void do_linacc(rvec* xcoll, t_edpar* edi)
> >> >>> >> {
> >> >>> >> /* loop over linacc vectors */
> >> >>> >> for (int i = 0; i < edi->vecs.linacc.neig; i++)
> >> >>> >> {
> >> >>> >> /* calculate the projection */
> >> >>> >> real proj = projectx(*edi, xcoll,
> >> >>> >> edi->vecs.linacc.vec[i]);
> >> >>> >>
> >> >>> >>
> >> >>> >> /* calculate the correction */
> >> >>> >> real preFactor = 0.0;
> >> >>> >> if (edi->vecs.linacc.stpsz[i] > 0.0)
> >> >>> >> {
> >> >>> >> if ((proj - edi->vecs.linacc.refproj[i]) < 0.0)
> >> >>> >> {
> >> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj;
> >> >>> >> }
> >> >>> >> }
> >> >>> >> if (edi->vecs.linacc.stpsz[i] < 0.0)
> >> >>> >> {
> >> >>> >> if ((proj - edi->vecs.linacc.refproj[i]) > 0.0)
> >> >>> >> {
> >> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj;
> >> >>> >> }
> >> >>> >> }
> >> >>> >> [...]
> >> >>> >>
> >> >>> >>
> >> >>> >> In both cases it reaches the same code
> >> >>> >>
> >> >>> >> preFactor = edi->vecs.linacc.refproj[i] - proj
> >> >>> >>
> >> >>> >> That surprised me a bit, is it deliberate? If so it may be the
> >> >>> >> code
> >> >>> >> can be simplified anyway.
> >> >>> >>
> >> >>> >> That aside, if you're looking for performance I might be able to
> >> >>> >> help.
> >> >>> >> I don't know the high level stuff *at this point* and my C++ is
> so
> >> >>> >> rusty it creaks, but I can brush that up, do profiling and
> >> >>> >> whatnot.
> >> >>> >> I'm pretty experience, just not in this area. Speeding things up
> >> >>> >> is
> >> >>> >> something I've got a track record of (though I usually have a
> good
> >> >>> >> feel for the problem domain first, which I don't here)
> >> >>> >>
> >> >>> >> Would it be of some value for me to try getting more speed? If
> so,
> >> >>> >> first thing I'd need is to get this running under cygwin, which
> >> >>> >> I'm
> >> >>> >> struggling with.
> >> >>> >>
> >> >>> >> regards
> >> >>> >>
> >> >>> >> jan
> >> >>> >> --
> >> >>> >> Gromacs Developers mailing list
> >> >>> >>
> >> >>> >> * Please search the archive at
> >> >>> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> >> >>> before
> >> >>> >> posting!
> >> >>> >>
> >> >>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >>> >>
> >> >>> >> * For (un)subscribe requests visit
> >> >>> >>
> >> >>>
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >>> >> or send a mail to gmx-developers-request at gromacs.org.
> >> >>> >>
> >> >>> >
> >> >>> --
> >> >>> Gromacs Developers mailing list
> >> >>>
> >> >>> * Please search the archive at
> >> >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> >> before
> >> >>> posting!
> >> >>>
> >> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >>>
> >> >>> * For (un)subscribe requests visit
> >> >>>
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >>> or send a mail to gmx-developers-request at gromacs.org.
> >> >>>
> >> >> --
> >> >> Gromacs Developers mailing list
> >> >>
> >> >> * Please search the archive at
> >> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
> >> >> before
> >> >> posting!
> >> >>
> >> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >> >>
> >> >> * For (un)subscribe requests visit
> >> >>
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> >> or send a mail to gmx-developers-request at gromacs.org.
> >> >
> >> --
> >> Gromacs Developers mailing list
> >>
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> >> posting!
> >>
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> >> * For (un)subscribe requests visit
> >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> >> or send a mail to gmx-developers-request at gromacs.org.
> >
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20200325/8dca5607/attachment-0001.html>
More information about the gromacs.org_gmx-developers
mailing list