Herein we assume that the OpenFOAM environment is created and that the steps in the icoFoam tutorial example are performed.
Sequential batch job on Vuori server
Copy an example file for sequential run of the OpenFOAM cavity flow tutorial in to the run directory
cp /v/linux26_x86_64/appl/fluid/foam_instfiles/batchfiles/seqjob $FOAM_RUN/cavity
Move to the run directory and check that the memory and execution time limits etc., as well as the paths are ok in seqjob job file.
Notice that in the job file, the module add openfoam-1.X command is not used.
The job is send to the batch environment by giving the command
sbatch seqjobAnd to check job status, give the command
squeue -u $USER
Parallel batch job on Vuori server
Here we execute the cavity flow tutorial with 4 processors. First we have to increase the mesh size up to 200 x 200 cells. Edit the cavity/system/constant/polyMesh/blockMeshDict file in following way.
Change these lines to look as
convertToMeters 1.0;
hex (0 1 2 3 4 5 6 7) (200 200 1) simpleGrading (1 1 1)
Save the file. Make following copy
cp $FOAM_TUTORIALS/multiphase/interFoam/laminar/damBreak/system/decomposeParDict $FOAM_RUN/cavity/system
In decomposeParDict file you may set the parameters for the domain decomposition, i.e. for the partitioning of the fluid domain for different processors. The decomposeParDict file in damBreak tutorial case is usable as it is for this example.
Next, make a copy of the example jobfile of parallel batch job:
cp /v/linux26_x86_64/appl/fluid/foam_instfiles/batchfiles/parjob $FOAM_RUN/cavity
and edit it into proper shape.
Notice that in the jobfile, the module add openfoam command is not used.
Next, in the cavity directory, create the new mesh and decompose it to 2 x 2 domains
blockMesh
decomposePar
Four subdirectories, processor*, are created. The case is ready to be sent for processing, i.e.,
sbatch parjob
The results are written into processor* directories. For to combine the results of the different sub domains, give the command
reconstructPar
The final results are then available in 0.* directories.
Parallel batch job on Louhi (Cray XT) server
Proceed in similar way as instructed above for Vuori server. A sample batch job file for parallel batch run on Louhi can be copied with commandcp /fs/local/linux26_x86_64/appl/fluid/foam_instfiles/batchfiles/parjob $FOAM_RUN/cavity
Move to cavity directory and edit parjob file to fit your own case. The sample file is for a run of 64 processes.
The job is send to the batch environment by giving the command
qsub parjob
A ready case file set for parallel run test is also available. Use it by following way. First move to your run directory ($FOAM_RUN) and give the commands
tar xf /fs/local/linux26_x86_64/appl/fluid/foam_instfiles/batchfiles/of1000_64.tar
cd of1000_64
blockMesh
decomposePar
cp /fs/local/linux26_x86_64/appl/fluid/foam_instfiles/batchfiles/parjob .
qsub parjob
You may check your batch job status with command
qstat -u $USER