Tehdyt toimenpiteet
Parallel Batchjobs
Criteria for Parallel Simulations
Benchmark tests have been performed on murska.csc.fi. In terms of cells (for three-dimensional runs including a two-equation turbulence model, i.e., 4+2=6 variables at each node) the following conditions for parallel use of Fluent 6 could be deduced on the different platforms:
- Cases below 50,000 cells should not be considered for parallel computation (i.e., they should be run using a single CPU either on corona or murska)
- Small parallel problems (50,000 - 250,000 cells) can use up to 4 CPUs
- Medium size parallel problems (1,000,000 - 10,000,000 cells) can use up to 8 CPUs
- Large parallel problems (10,000,000 cells and larger) can use up to 12 CPUs
Preprocessing of Parallel Cases
Preprocessing of parallel Fluent runs can either be done on all platforms where Fluent is installed, i.e., murska.csc.fi and corona.csc.fi.
The mesh can alternatively be partitioned during the preprocessing phase or then later automatically by the solver. If the first of this method is applied, the user has to determine the amount of used partitions in advance, whereas the latter alternative provides more flexibility to this decision.
Batch Jon File for Parallel Run on LSF-HPC (murska.csc.fi)
The batch job file includes the necessary information for the LSF-HPC to schedule the request and assign sufficient resources to the task(s). For a four processor run with Fluent 6 on murska.csc.fi, the batch job script could have the following contents:
#!/bin/csh |
Explanation for the different lines:
| #!/bin/csh | sets the shell the batch job is executed to csh |
| #BSUB -L /bin/csh |
Initializes the execution environment |
| #BSUB -J jobname%J | name of your job, %J will show as your jobID |
| #BSUB -e output_err_%J | system error message output file |
| #BSUB -o my_output_%J | system message output file |
| #BSUB -M 1048576 |
a per-process (soft) memory limit in KB |
| #BSUB -W 00:10 |
wallclock time limit (hh:mm) |
| #BSUB -n 4 | number of processes (cores) |
| module load fluent/fluent.inc | load setup |
| fluent 3d -g -t4 -slurm -pib -mpi=hp -i journalfile.jou |
launch fluent; the options -slurm and -pib are needed |
Fluent Journal File for Parallel Run
A typical journal file for parallel runs could look like the following example:
parallel/partition/auto/use-case-file-method yes
file/read-case casefile
parallel/partition/print
solve/initialize/initialize-flow
solve/iterate 3000
parallel/timer/usage
file/confirm-overwrite no
file/write-data casefile.dat
exit
Explanation for the different lines:
| parallel/partition/auto/use-case-file-method yes |
sets the partition method of the grid to the one that has been determined in the earlier prepared case file |
||
| file/read-case casefile | reads the earlier prepared casefile (casefile.cas) | ||
| parallel/partition/print | gives statistics on the performed partition (can be checkked after the run in the output file). This is just of informative character, and consequently can be dropped |
||
| solve/initialize/initialize-flow | initializes the field variables (needed to launch the simulation |
||
| solve/iterate 3000 |
performs up to 3000 iteration (less, if converged earlier) |
||
| parallel/timer/usage | gives statistic on the run's parallel performance (can be checkked after the run in the output file). This is just of informative character, and consequently can be dropped | ||
| file/confirm-overwrite no |
will overwrite existing files without asking (which would hang the batch-job) |
||
| file/write-data casefile.dat |
writes data of result into casefile.dat |
||
| exit | properly exits Fluent |
Submitting, Monitoring and Deleting Jobs on LSF-HPC (murska.csc.fi)
Jobs are submitted using the commandbsub < jobfilename.shwhere jobfilename.sh is the file containing the batch-job script. Do not forget to insert the symbol <, as else the job will not launch properly.
Thereafter, the job (as a matterof fact, all currently running jobs) can be monitored using the command
bjobs
Jobs can be killed using the command
bkill job-id
where job-id has to be replaced by the job ID obtained applying the bjobs-command.
