The Louhi User's Guide presents the mixed Cray XT4/XT5 system, located at CSC. Louhi has 1012 XT4 compute nodes, each with one quad-core AMD Opteron processor (Barcelona), and 852 XT5 compute nodes, each with two quad-core AMD Opteron Barcelona (672 nodes) or Shanghai (180 PRACE project nodes) processor. The operating system is now Cray Linux Environment (CLE) (originally UNICOS/lc) containing SuSE Linux for service nodes and Compute Node Linux (CNL) for compute nodes. Louhi is used for running large parallel codes.

This web guide introduces the computing and program development environment of Louhi. Special attention is paid to how to obtain a good performance for massively parallel scientific applications. The system was opened for CSC customers on April 2, 2007, in its original configuration, The Cray XT4 "Hood" system with 1012 compute nodes.

Louhi was updated in the beginning of February 2008 into the configuration, which contained the Compute Node Linux (CNL), as operating system for the compute nodes, the Portable Batch System (PBS Pro) version 8 or higher and the Application Level Placement Scheduler (ALPS) including the aprun job launcher command.

On June-July 2008 dual-core processors were upgraded to quad-core processors. At the same time memory as added to compute nodes so that the amount of memory stayed the same per core (1 GB / core or 2 GB / core depending on a compute node). 

Louhi expanded to its final configuration on September-October 2008. Then a Cray XT5 facility with 5376 compute cores was attached to the former Cray XT4 facility with 4048 compute cores. Cray XT5 differs from Cray XT4 in several ways. The XT5 compute nodes contain two quad-core processors (so called NUMA nodes) and thus eight cores in a node using shared memory. One 3D SeaStar2 link is connected to eight cores instead of four cores. Most of the XT5 part has a memory of 1 GB/core, but 768 cores has a 2 GB/core memory.  The operating system UNICOS/lc was upgraded to the version 2.1 with the new name Cray Linux Environment (CLE) and a new version 9.2 of the batch system PBS Pro was taken into use during the extension.  At the same time, the batch queues were changed and reconfigured.

During November 2008, two XT5 cabinets containing 180 compute nodes each consisting of two quad-core Barcelona processors (in total 1440 cores) and belonging to the PRACE (Partnership for Advanced Computing in Europe) project were added to Louhi. This extension was named to Loviatar.

Loviatar’s processors were updated  on June 2009 to AMD's quad-core, 2.7 GHz Opteron Shanghai processors (1440 cores).

The operating system of Louhi (and Loviatar) was updated from CLE 2.1 to CLE 2.2 on August 11, 2009.

This is the third edition of this guide and is updated from the Release 2.2 by Raimo Uusvuori.

The second edition of this guide and was updated for the CNL and new PBS Pro and ALPS (Release 2.0) by Sebastian von Alfthan, Juha Jäykkä, Vesa Kolhinen, Yrjö Leino, Juha Lento, Pekka Manninen, Jarmo Pirhonen,  Juha Ruokolainen, Raimo Uusvuori (Editor) and Joni Virtanen.  The same persons mostly updated the guide for quad-core and memory upgrades (Release 2.1). In additition, Jussi Enkovaara joined to the updating team. The same writers also updated the guide for the final extension of Louhi (Release 2.2).

The second edition was based on the first edition of this guide. It was written by Ville Savolainen (Editor), Sebastian von Alfthan, Jussi Heikonen, Yrjö Leino, Juha Lento,  Juha Ruokolainen, Raimo Uusvuori and Joni Virtanen for the original configuration of Louhi, which contained Catamount compute node operating system,  the login node batch job launcher command yod, and the PBS Pro version 5. 

Our sincere intention is to keep this guide upto date as system and software upgrades are made, but you are advised to use the tools described in the guide to check the latest and default versions and configurations of the system components you are using.

Comments concerning the content of this publication are most welcome.
Please send your feedback and possible corrections to the e-mail address helpdesk at csc.fi and as a Cc: to the mail address below.

Espoo August 13, 2009

Authors