Tehdyt toimenpiteet
Cray XT system
Louhi - Cray XT4 part
The XT4 part has 11 cabinets that include mostly 24 blades per cabinet. A blade includes four compute nodes. Each compute node has a single AMD Opteron 2.3 GHz quad-core processor. Each compute node has at least 4 GB memory (1 GB/core) and 96 nodes (1 cabinet) has 8 GB memory (2GB/core). The Cray SeaStar interconnect system directly connects all compute nodes in a 3D torus topology using Hyper-Transport links of Opteron processors. So 4 cores (one quad-core processor) share a same interconnect link.
Louhi - Cray XT5 part
The XT5 part has 9 cabinets and each of them has 24 blades per cabinet. As in XT4 a blade includes four compute nodes. Now each compute node has two AMD Opteron 2.3 GHz quad-core processors. Each compute node has at least 8 GB memory (1 GB/core) and 96 nodes (1 cabinet) has 16 GB memory (2GB/core). The interconnecting system is the same as in the XT4 part, one SeaStar interconnect per compute node. It means that 8 cores (two quad-core processor) share a same interconnect link.
Service nodes
In addition to compute nodes, Louhi has also 22 service nodes located to two XT4 cabinets (c0-0 and c0-1), running SuSE Enterprise Linux, used for login, I/O, boot and other service usage. Louhi has 70 TB of local fast
The Cray SeaStar2 interconnect system
The heart of the performance for massively parallel runs lies in the interconnection network between the processors. In Louhi, the interconnect is Cray SeaStar2 communication system. The Cray SeaStar2 interconnect system directly connects all compute nodes in a 3D torus topology using Hyper-Transport links of Opteron processors, providing a bandwidth-rich environment. The SeaStar2 interconnect system carries all message passing traffic (MPI) as well as all I/O traffic on both (XT4 and XT5) systems.
Cabinets
There are three critical differrencies between compute nodes: the node is of the XT4 or XT5 type, and the memory per core (and thus per node), and the clock cycle of the processor is 2.3 or 2.7 GHz (PRACE nodes). The different kind of compute nodes should not be mixed when jobs are sent to execution. The following table shows which cabinets contains which kind of nodes (PRACE cabinets also included):
| Node ID (NID) | Cabinet | Memory bus | Node type | Memory/core | Memory/node | Other |
|---|---|---|---|---|---|---|
| 24-95 | c0-0 | 800 MHz | XT4 | 1 GB | 4 GB | + Service Nodes |
| 148-223 | c0-1 | 800 MHz | XT4 | 1 GB | 4 GB | + Service Nodes |
| 256-351 | c1-0 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 384-479 | c1-1 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 512-607 | c2-0 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 640-735 | c2-1 | 800 MHz | XT4 | 2 GB | 8 GB | |
| 768-863 | c3-0 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 896-991 | c3-1 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 1024-1119 | c4-0 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 1152-1247 | c4-1 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 1280-1375 | c5-0 | 800 MHz | XT5 | 2 GB | 16 GB | |
| 1408-1503 | c5-1 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 1536-1631 | c6-0 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 1664-1759 | c6-1 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 1792-1887 | c7-0 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 1920-2015 | c7-1 | 800 MHz | XT5 | 1 GB | 8 GB | |
| 2048-2143 | c8-0 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 2176-2271 | c8-1 | 800 MHz | XT4 | 1 GB | 4 GB | |
| 2316-2399 | c9-0 | 800 MHz | XT5 | 1 GB | 8 GB | PRACE cabinet, + Service Nodes |
| 2432-2527 | c9-1 | 800 MHz | XT5 | 1 GB | 8 GB | PRACE cabinet |
The 20 cabinets are physically on the floor of the computer hall in two rows, 10 cabinets in each row. In the column Cabinet there is the cabinet name, which is the first part of the node name. The second number in the cabinet name is the row number (0 or 1) and the first number is the cabinet number (0-8) in that row. The full node names are explained in Chapter Job launching command: aprun.