@@@@@ @ @ @ @ @ @ @@@ @ @@ @@@@ @ @ @ @@@@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @@ @@@@@ @ @ @ @ @ @ @ @ @ @ @ @ @@ @ @ @ @@@@@ @@@ @ @@@@ @ @ @ @@@@ @ @ @ @ @ @ @@@@ @@@@ @ @@ @@@@ @ @ @@@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @@ @ @ @ @ @@@@@ @ @@@@ @ @ @ @ @ @ @ @ @ @ @@@@ @ @@@@ @@ @@ @@@ @@ ************************************************************************** * * * # # ##### * * # # # * * # # # # * * # # # ##### * * # # # # * * # ## # # # computing cluster * * # # # ##### * * * * Author: juergen.oehlschlaeger@kit.edu 27 Oct. 2015 * ************************************************************************** Short instruction, how to run parallel CORSIKA on KIT-CN IKP computing cluster in building 425 (gridengine: qsub -l q=augn.q `script`): path: $WORK/corsika.trunk/ -------------------------- (1) run ./coconut to create an executable with parallelization by scripts by selecting `p - PARALLEL treatment of subshowers` and `1 - Special stack for shell script without MPI` i.e. name similar to `corsika74765Linux_QGSII_gheisha`. path: $WORK/corsika.trunk/run/ ------------------------------ (2) go to subdirectory run/ and rename the executable in a way to distinguish standard and thinning simulations, and to mark the parallelization by scripts by another additional appendix `_pll`, i.e. name `corsika74765_stnd_QGSII4_gheisha_pll`. (3) prepare a corsika steering file `parallel-002345` for the parallel simulation containing keyword PARALLEL with `T` as forth parameter by copying a formerly used parallel steering file or by `acreinphc3.f` (from the main corsika path $WORK/corsika.trunk/src/parallel/), example see appendix (A-3); (4) execute shell script `preprocess-ik3.sh` with two arguments, the first argument is the name of the parallel steering file, the second argument is the name of the corsika executable: ./preprocess-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll which displays following four lines: ... creating directory csk002345/ and copy utilities; ... copying all corsika model-dependent tabular files; ... cd csk002345/ ... then execute ./corsika-initial-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll ik3 switch to subdir: $WORK/corsika.trunk/run/csk002345/ ---------------------------------------------------- (5) starting the simulation with `./corsika-initial-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll ik3` gives the first message: Your job 7497970 ("corsika-initial.ik3.sh") has been submitted . . . . . . . . . . . . . . (many more submissions) . . . . . . . The simulation is finished when the terminal prompt is shown again. (6) still or again staying in the subdirectory csk002345/ let execute and run the shell script `postprocess-ik3.sh` which deletes corsika tabular files and creates the protocol files Job002345_ik3.[err,out] and the time statistic file `time.txt002345`. Remaining output files are now all particle data files of type `DAT002345-.........-......... and few files *.lst and the single file `DAT002345.899999999.long` renamed from file `fort.48`; ************************************************************************** (A-3) parallel steering file ============================ RUNNR 2345 PARALLEL 400. 400000. 1 T !!! this T is necessary. NSHOW 1 PRMPAR 14 EVTNR 1 SEED 7035 0 0 SEED 7036 0 0 SEED 7037 0 0 SEED 7038 0 0 SEED 7039 0 0 SEED 7040 0 0 ERANGE 7.4132E+07 7.4132E+07 THETAP 17.00 17.00 PHIP 26.43 26.43 OBSLEV 1452.e2 870.000 g/cm^2 MAGNET 19.71 -14.18 Auger MAXPRT 1 ECTMAP 1.E11 ECUTS 0.1000 0.1000 2.5e-4 2.5e-4 RADNKG 200.E2 HADFLG 0 0 0 0 0 2 ELMFLG T T QGSJET T 0 QGSSIG T MUADDI T MUMULT T STEPFC 1. LONGI T 5. T T HILOW 88.88 DIRECT ./ HOST iklx289 USER you EXIT ************************************************************************** (A-6a) preprocessing ==================== within main path: ./preprocess-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll ************************************************************************** (A-6b) initial submission ========================= within subdirectory: ./corsika-initial-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll ik3 ************************************************************************** (A-6c) all infos in Job002345_ik3.out ===================================== 1335689800 0:05.08 Real-time 0.21 TotalCPUseconds ./DAT002345-000000000-000000001.lst 1335690136 0:04.26 Real-time 0.23 TotalCPUseconds ./DAT002345-039830684-000000585.lst 585 Files 230 sec #!/bin/bash # # jobik3-002345 # ============= # CORSIKA using parallel scripts at ik3 computing cluster # ./corsika-initial-ik3.sh parallel-002345 corsika74765_stnd_QGSII4_gheisha_pll ik3 RUNNR 2345 PARALLEL 1000. 1000000. 1 T NSHOW 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . EXIT 48.053822 GBy (A-6d) optional histogram analysis script showanalyrun.sh002345 =============================================================== #!/bin/bash # # showanalyhc3.sh[002345] # ======================= # create histograms as ascii tables of all data files. # ls -1 DAT002345* | grep t -v | grep p -v | grep b -v > showanalyhc3.i002345 ./showanalyhc3 < showanalyhc3.i002345 > showanalyhc3.out002345 mv fort.9 showanalyhc3.fort002345 **************************************************************************