Vuori User's Guide > Using the Vuori cluster > Storing and moving files
Tehdyt toimenpiteet

Storing and moving files

The organization of the file systems on Vuori is described here.

There are several file storage areas available for users. Usually you need not (and should not) refer to directories with full path names. Instead, use the defined environment variables. The most commonly used storage areas are $HOME, $METAWRK, $WRKDIR, $USERAPPL, $TMPDIR and $ARCHIVE. The lifetime of files in each directory is limited and isdescribed in the table. For exhaustive list of available directories and their visibility in CSC environment please check Data services guide: 4 Directories at CSC.


Environment variable
Purpose
Data lifetime
 $HOME User's home directory
Permanent
 $METAWRK  User's common work directory for CSC environment
At least 30 days
 $WRKDIR  User's local work directory
60 days
 $USERAPPL  User's applications directory
Permanent
 $TMPDIR  Local temporary directory
~1 day
 $ARCHIVE  User's archive directory
Permanent

The login nodes of Vuori see all available directories in the CSC environment. On the compute nodes, however, it is not possible access following directories: your home $HOME, the archive $ARCHIVE, the metawork $METAWRK, and your project directory. 

The most important user directories that the compute nodes do see are the work directory $WRKDIR and $USERAPPL for user's own applications and programs. These are shared between all the compute and login nodes on Vuori. Both directories are local to Vuori and are on the fast Lustre filesystem. You must install your own applications under $USERAPPL and launch your batch jobs from $WRKDIR and generally use it for your batch jobs.

$HOME directory

Your home directory $HOME on the CSC computing environment is located on a disk server. The files in your home directory are shared between the login nodes of Vuori and other computing servers at CSC. The home directory is suitable only for small initialization files and frequently used small programs. It is not intended for extensive I/O operations or for large data sets. Therefore, use of $HOME on compute nodes is not possible. Before running a job you must copy all needed files from your home directory to the local work disk due to performance reasons.

The home directory $HOME is backed up regularly. This directory is meant for permanent files, with a maximum total size of one gigabyte. It is a typical repository for source codes and small input files.

$WRKDIR directory

The local working directory $WRKDIR is common for all Vuori nodes. However, since no backups are taken, a disk crash may destroy the contents of the directory. On Vuori unmodified files will be deleted after 60 days. This directory is meant for all files which will be accessed during computational runs. Input files need to be transferred to $WRKDIR before start of the run, user's own applications should be either in $WRKDIR or $USERAPPL. Due to 60 day time limit on files in $WRKDIR, you can move or reinstall frequently used application executables under the directory $USERAPPL. Rarely used applications should be moved under your project directory (or possibly $ARCHIVE).

$USERAPPL

The local user application directory $USERAPPL is common for all Vuori nodes. Backups for this directory are taken regurlarly. This is the place where you should install your own applications, which you are using immediately after installation or which you are going to use frequently in the near future.

$METAWRK

The directory $METAWRK is shared by all the machines in the CSC computing environment. Because this directory is not local it is slower to use than the directory $WRKDIR and it is available only on the login nodes.  This directory is meant for program development and analysis of your results.

$ARCHIVE

The directory $ARCHIVE refers to the archive server. This is not a local filesystem, and it should be used only for long-term storage of compressed and infrequently used files. The access time to the archive server can be very slow because usually the data resides on tapes.

For information on using the archive please visit 4.4 Archive directory

$TMPDIR

The directory $TMPDIR is local on each Vuori node and it should be used only for short-living files. When dealing with many small files, $TMPDIR might actually be faster than the Lustre directory $WRKDIR. $TMPDIR is also automatically used by some tools, such as some compilers.

You can use the command

quota


to check your disk quota usage.