0

I am trying to run multiple qsub jobs using different parameters:

for i in {1..10}; do qsub -v Size=''$i'' test.sh; done

The contents of the test.sh script are :

#!/bin/bash 
#$ -cwd
#$ -S /bin/bash
#$ -N matrix_multiplication
#$ -o output
#$ -e output
TIMEFORMAT=%R
echo "Size of matrix ${Size}"

However, the output file only has two lines:

Size of matrix 9
Size of matrix 1
0

1-8 are not included in the output file.
I can't find where the problem may be.
Any help is greatly appeciated!

zx485
  • 2,249
  • 11
  • 24
  • 34
Car le
  • 3
  • 1

1 Answers1

0

Are these 10 jobs running simultaneously?

If so, I don't think you can do what you're trying to do. Because each of them are opening the output file at different times and writing to it. And they are "overwriting each other".

For example, suppose job 1 creates output at the same time job 2 creates output. At this point, both files are empty. If job 2 writes first, and then job 1, job 1 would be opening it at the time it was empty. And what job 2 has written would not appear.

If you have 10 jobs, you need to write to 10 different files. If you feel they have to be in one file, then you need to create a 11th job that waits until all 10 jobs have completed and then combine their output files. This post by someone else demonstrates how you could do this.

Hope this helps!

Ray
  • 1,987
  • 17
  • 27
  • Thanks Ray! I figured that maybe it's not a good idea to output multiple jobs into a single file. I used $JOB_ID and redirect command to combined the output files. – Car le Mar 19 '21 at 03:52
  • @Carle You're welcome! And yes, that's what you should be doing. Feel free to "accept" my answer if it's alright with you. Good luck with what you're doing! – Ray Mar 19 '21 at 03:57