Edition 3

Release 3.0 (coming)

Adding of PRACE cabinets:

During November 7–12, 2008, two XT5 cabinets containing 180 compute nodes each consisting of two quad-core Barcelona processors (in total 1440 cores) and belonging to the PRACE (Partnership for Advanced Computing in Europe) project were added to Louhi. This extension to CSC’s Louhi super computer is one of the six prototypes of the PRACE  project.

The facility was then the following features:

• 2716 AMD Opteron 2.3 GHz quad-core Barcelona processors, totalling 10864 compute cores (4048 XT4 + 6816 XT5).
• Approximately 11.7 TB RAM.
• Compute cores have mainly 1 GB/core. In XT5, 768 cores and in XT4, 384 cores have 2 GB/core.
• Theoretical computing speed 102.2 TF, in Linpack 76.5 TF.

The PRACE prototype cabinets are primarily intended for the project but CSC’s customers will be able to use them under certain restrictions.

Operating system was updated to the General Access (GA) level of UNICOS/lc 2.1, whose name was officially changed to Cray Linux Environment 2.1 (CLE 2.1). The Cray Compiling Environment 7.0 release was also installed. It is the initial release of the Cray Fortran and Cray C compilers for use on x86-based Cray XT systems running the UNICOS/lc 2.0 CNL or Cray Linux Environment (CLE) 2.1 CNL operating systems. The Cray C++ compiler for x86-based systems is deferred to a future release.

Relacing  Barcelona processors of the PRACE cabinets to Shanghai processors:

The PRACE extension called Loviatar received additional effectiveness on June 1-5, 2009. Loviatar’s processors were updated to AMD's quad-core, 2.7 GHz Opteron Shanghai processors that utilize new-generation technology. The extension consisted of a total of 1440 processor cores. The L3-level cache memory used by the processors will be increased from 2 to 6 megabytes.

The extension is primarily reserved for the PRACE project. CSC’s other users will get to enjoy the added effectiveness to the extent of free capacity left available by PRACE users. A queue system will automatically allocate tasks to available processors.

Updating CLE 2.1 to CLE 2.2:

Operating system of Louhi was updated from CLE 2.1 to CLE 2.2 on August 11, 2009. The commands xtshowcabs and xtshowmesh were replaced with the command xtnodestat.

Edition 2

Release 2.2

The third phase upgrading of the Cray XT system Louhi to its final configuration performed September - October 2008.

The maintenance break proceeded in several steps:

• The operating system was upgraded to UNICOS 2.1 level (Limited Access, LA) . All programs have to be recompiled (when available). CSC provides support and assistance in recompiling (contact helpdesk@csc.fi).
• The 11 existing XT4 cabinets were moved physically to the new site.
• The 7 new XT5 cabinets were delivered and physically installed to the new site.
• The XT4 and XT5 parts were connected.
• The system OS was installed and tested.
• Benchmarks (part of the acceptance tests) were run.
• PBS Pro 9.2 batch system was installed and configured. At the same time, batch queues were redefined. Moderately parallel jobs have more resources than before in the final system.

Many chapters of this guide required updating due to these updates and upgrades. Two new sub-chapters, Cray XT system and Compute nodes, were added to Chapter Louhi hardware and numbering of it was rearranged.

Release 2.1

Processors of Louhi compute nodes were upgraded from dual-core to quad-core on June-July 2008. At the same time memory was added to compute nodes so that the amount of memory stayed the same per core (916 nodes 1 GB / core and 96 nodes 2 GB / core). The number of nodes remained same (1012). It was necessary to update software for the quad-core system.
Most of the chapters of this guide required updating due to the processor upgrading.

In the Release 2.0 of this guide there was misunderstanding what Cray Linux Environment (CLE) really means. It is the synomym to the operating system of the whole system UNICOS/lc, which contains the operating system of the login and other service nodes, SuSe Linux Enterprise Server, and the operating system of  the compute nodes, the Compute Node Linux (CNL). Because the name CLE is used very rarely, we use mainly the name UNICOS/lc instead in this guide now. In Release 2.0 of this guide CLE was used as the name for  the operating system of  the compute nodes. In this release all occurences of CLE are changed to CNL.

In the beginning of February, 2008, the operating system of the compute nodes of Louhi, the batch job system PBS pro and job launching system were updated. Also physical memory of the system was increased. All these updates have changed the way how to use the system and the update of this guide was necessary, too.

The operating system of the compute nodes of Louhi was updated from Catamount to Cray Linux Environment (CLE), formerly known as Compute Node Linux (CNL).

The batch job system PBS Pro was updated from version 5 to version 8 or higher.  In this connection the job launching utility yod was changed to ALPS including the job launching command aprun.

At the same time the memory of 576 compute cores was upgraded from 1 GB to 2 GB. CLE and the new version of PBS Pro facilitates the utilization of the inhomogeneous memory and is also required by the quad core processors that will be installed later on.

Names of some chapters have modified or they are renamed and the order of chapters are partly rearranged. Few new chapters are added, including this one (What's new) and Open issues.

Edition 1

Release 1.0

See the revision history below and the link in it.

Revision history

Edition 1

• Release 1.0, Jun 2007, Ville Savolainen (Editor)

Edition 2

• Release 2.0, Feb 2008, Raimo Uusvuori (Editor)
• Release 2.1, Sep 2008, Raimo Uusvuori (Editor)
• Release 2.2, Oct  2008, Raimo Uusvuori (Editor)