module load gromacs/version

The default version is the well scaling 4.X version. As it is a new version, we suggest the users to check that the chosen run options work properly. The new code uses automatically the domain decomposition algorithm, separate PME nodes and dynamic load balancing even without explicit flags to mdrun_mpi. If you want to specify them manually, you can use -dd and -npme flags.  A starting point would be to use roughly 1/3 of the cores for pme only, but test different proportions as this can have a huge impact on performance! Usually it is best to leave the division of cores between dd and pme to mdrun. The log file also suggests better values. Note also that with the new version you don't need to specify the number of processors for grompp.

The old version 3.3.3 is available only on Murska where you can test for similarity of results.

Batch jobs

To run jobs in the compute nodes, use a PBS-script of the following type (you can also load the module before submitting the job and then omit it from the script):

#!/bin/csh
#PBS -l mppwidth=64
#PBS -N gmx
#PBS -r n
#PBS -l walltime=01:10:00

module load gromacs
cd $PBS_O_WORKDIR

aprun -n 64 mdrun_mpi -s topol.tpr

The script requests 64 processing elements (PEs) and by default uses all cores in a node. Older versions are not recommended.

aprun -n 8 mdrun_mpi -s topol.tpr

All parallel computations must be done in the compute nodes.

Optimising for speed

Gromacs is a very efficient code. There are a number of things that can be tuned to get the most out of the used computing resources. This is particularly important when running jobs with many cores (like 64 - 2048). Gromacs can choose many parameters automatically, but with very large number of cores it is good to check things yourself. Below are just some examples, but please check the Gromacs home page and tool specific helps for more parameters.

How many cores for pme?

Gromacs has a utiliti tool g_tune_pme which will run test jobs to search for the optimal choice. Below is one example on Louhi. Note that Louhi requires a special version of the tool, namely g_tune_pme_louhi because of the running syntax.

#!/bin/csh
#PBS -l mppwidth=64
#PBS -N gmx-tune
#PBS -r n
#PBS -l walltime=00:30:00

module load gromacs/4.5.2
cd $PBS_O_WORKDIR

setenv MDRUN mdrun_mpi

# change this to match the number of cores!
setenv MPIRUN "aprun -n 64 "

g_tune_pme_louhi -s topol.tpr

Other topics include using virtual sites (instead of explicit hydrogens with bond constraints), -nosum with mdrun, Ewald parameter balance (grid spacing / cutoff ratio)...

Additional details can be asked through helpdesk@csc.fi