The default Fortran and C/C++ compilers used on Louhi are the Portland Group (PGI) compilers v. 9.0.2 for Fortran 77 and 90/95, C and C++, the GNU Compiler Collection (GCC) compilers v. 4.2.0.quadcore (a special Cray XT4/XT5 version) for Fortran 90/95 (gfortran) , C and C++, and the PathScale compiler suite  compilers v. 3.2 for Fortran  90/95, C and C++.

In the case of the GCC and PathScale compilers the Fortran 90/95 compilers can be used also with FOTRAN 77 programs. Compilers are accessed through the Cray drivers (wrapper scripts) ftn, cc and CC. All these compiler versions contain special optimization options for the quad-core Opteron Barcelona processor architecture, which make possible to access quad-core specic capabilities. See the compiler suite sub-sections for these options.

There are also older and/or newer versions of these which are taken into use with the module command (see below). By the command module avail you can see what these are.  You may select them by loading/swapping the appropriate modules, if needed. Please, note that GCC 3 versions are not any more supported by any means in CNL and thus in quad-core compute nodes. In login nodes (in SuSE Linux) they can still be used.

More information on the compilers can be found on the following sub-sections:

• PGI compiler suite
• GNU Compiler Collection
• PathScale Compiler Suite
• Mixed-language usage
• Further information on linking

This page contains information on using modules, wrapper scripts and general compiler options.

Modules

When you log in a set of modules is loaded automatically. The loaded modules are shown by the command module list. The upgrade from dual-core to quad-core processors  required certain quad-core specific modules, which are loaded by default when you log in:  xtpe-barcelona (xtpe-quadcore is not used anymore) and xt-asyncpe. xtpe-barcelona adds references to quad-core-built libraries for the purpose of building quadcore applications and sets automatically the following compiler-appropriate Opteron quad-core architecture, that is, Barcelona options:

xt-asyncpe contains compiler driver scripts (wrappers) f77, ftn, cc and CC and internally used driver scripts linux-*, which replaced mpi* and linux-* (qk-* in Catamount) scripts used for dual-core Catamount and CNL applications. Please, note that in present CLE 2.2 and its CNL operating system  f77 is supported for quad-core compute nodes only in the PGI compiler suite. For the PathScale and GCC compiler suites you must use ftn instead.

The Barcelona or Shanghai options are set into the following environment variables for PGI, GCC (GNU) and PathScale as follows:

PGI_POST_COMPILE_OPTS= -tp barcelona-64 
   or
PGI_POST_COMPILE_OPTS= -tp shanghai-64
PATHSCALE_POST_COMPILE_OPTS= -march=barcelona
GNU_POST_COMPILE_OPTS= -march=barcelona

If it is absolutely necessary to use such older versions of compiler which do not have the Barcelona options, it is necessary remove, replace or redefine the options -tp, -march .  Follow error messages you may get. You need to redefine the environment variable only for the compiler suite you are using.

For MPI programs the Cray Message Passing Toolkit (MPT) module xt-mpt is also loaded automatically. It contains SHMEM and MPI-2 (derives from MPICH-2) include files and libraries. The version MPT 3 was installed when quad-core upgrade was done. It contains many improvements as compared to MPT 2. 

The different programming and compiler environments and versions are taken into use as modules, too. When you log onto Louhi, the PGI environment, i.e., the PGI module PrgEnv-pgi is loaded by default. Thus, you don't need to load this module to use PGI compilers, unless you have changed the environment earlier to the GCC module PrgEnv-gnu or the PathScale module PrgEnv-pathscale. At present, there is only one PrgEnv-<compiler> module for each 3 compiler set. These load the default versions of compilers mentioned above.

For example, to use the GCC compilers you need to load the GCC environment module. Before this you should first unload the PGI environment:

module unload PrgEnv-pgi
module load PrgEnv-gnu

Alternatively, you can switch from the PGI environment to the GCC environment directly:

module swap PrgEnv-pgi PrgEnv-gnu

Similarly, you can load the module PrgEnv-pathscale or switch to it, or load back  PrgEnv-pgi.

The default version of GCC is currently 4.2.0.quadcore. You cannot use anymore  v. 3 GCC compilers for compute node applications (these were the last GCC compilers containing g77 for FORTRAN 77).  You may posibly use the GCC versions 4.1.x and other 4.2 versiosn, but then you must redefine the environment variable GNU_POST_COMPILE_OPTS so that barcelona option is removed. In addition you have to issue, e.g., the following commands after logging in:

module swap PrgEnv-pgi PrgEnv-gnu
module swap gcc gcc/4.1.2

However, it is possible that various problems may arise when other version than the quad-core specific version 4.2.0.quadcore (or possibly versions 4.3, which also support Barcelona processors) is used.

For more information about modules see Chapter Using the Louhi system.

Login nodes, wrapper scripts and libraries

Programs are compiled on the login (or front) nodes and executed on the compute nodes. Since the latter have a different operating system (SuSE based CNL) than the first ones (full SuSE), drivers or wrapper scripts are provided to include the necessary compiler and linker options. See Further information on linking on what the linker does automatically (several libraries are linked automatically, if needed) and how the user can pass options to the linker. Note that the binaries created for compute nodes may not run on the front node.

The general syntax of invoking the compiler drivers is:

compiler_command [driver_options] [PGI_options|GCC_options|PathScale_options] filename ...

The Cray XT Series compiler commands for invoking the PGI, GCC and PathScale compilers are:

Compile and link all applications, including MPI applications, using these Cray XT Series compiler commands. Do not use directly the PGI compiler commands pgf77, pgf90, pgf95, pgfortran, pgcc and pgCC or pgcpp, the GCC compiler commands g77, gfortran,  gcc and g++, or the PathScale compiler commands pathf90, pathf95, pathcc and pathCC.

If you compile and link in separate steps, use the Cray XT Series compiler commands also in the linking step and not the linker ld directly (unless you know which options it needs).

The CLN OS of compute nodes does not support shared object files, but everything needs to be linked statically to be run on the compute nodes.

For more information about the few options (driver_options above) specific for the different compiler commands given in the table see their manual pages: man ftn, for example. All PGI, GCC or PathScale compiler flags can be given directly to the drivers.

Options

The options available for the PGI, GCC and PathScale compiler suites (PGI_options, GCC_options and PathScale_options above) can be found from their man pages when the corresponding modules are loaded: man pgf90, man gcc or man pathCC, for example. You can also refer to the PGI, GCC manuals and PathScale documentation on the Web or online on Louhi.

The following table shows some options, which are common for all PGI and GCC compilers:

Compilation examples

Here are few examples of how to use the Cray compiler commands:

ftn fprog.f90

f77 -o f77prog f77proga.f f77progb.f

CC -o CCprog CCprog.C

cc -c cprog1.c

ftn -o prog1 fprog1.f95 cprog1.o

ftn -c fprog2.f90

ftn -o prog12 cprog1.o fprog2.o

Parallel MPI programs

Use the Cray compiler wrapper scripts cc, CC, f77 and ftn also when you compile MPI and/or OpenMP programs. Earlier these wrappers used internally  and automatically mpi* compiler commands/wrapper scripts (e.g., mpif90) for dual-core Opteron processors, but now when dual-core processors have replaced by quad-core processors, these scripts are not used anymore. They are replaced by the scripts  linux-*  (where * is cc, CC, f77 or f90) which are executed by the above mentioned wrapper scripts. These internally used scripts in turn execute the actual compiler command.

Louhi is dedicated for parallel jobs only, and MPI applications are compiled by invoking the same compiler commands as introduced above. For more information, see the sub-section Using MPI.

Different programming environments and their versions

Because the different programming environments and compilers and their different versions all use the same Cray compiler wrapper commands, you should always check which programming environment and which compiler and its version is loaded. There should normally be only one programming evironment loaded at a time. The command

module list

lists the loaded modules. There may also be different versions of the PGI and GCC available. You can check the version of a PGI compiler using the option -V, the version of a GCC compiler with the option --version, and the version of PathScale compiler with the option -dumpversion or -version, for example: CC -V, f77 --version or  ftn -dumpversion (depending on the loaded programming environment).

The default versions are loaded using the module names given above. For the older versions there are similar names but containing version numbers. You can list available modules including ones for programming environments by the command:

module avail

New versions of operating system, libraries, compilers and other modules

When a new version, release or update of the operating system is installed, you should at least relink your application. The same applies to libraries, compilers and other modules. Otherwise it may not work correctly or at all on compute nodes. Users are notified when new installations are done. Here is the general procedure in such cases:

1. Relink your application using the same Cray wrapper script commands and options as you used for your original application (unless you are addvised to change options).
2. If this does not work, recompile and relink your application.
3. If this does not help, try to make changes to the source code and/or compiling and linking options. It may be necessary to debug your application in this step.
4. If this does not help, contact CSC's specialists.