Running Multiple Independent MATLAB Tasks

Described in the following is a procedure to run multiple independent MATLAB tasks. Typically, users who want to run MATLAB jobs in this mode do so for parametric studies of the same code under different conditions. These conditions are often controlled by an index or an input file. Since multiple copies of MATLAB are processed concurrently, the computed results from each must be saved in a unique file. While you can submit multiple single-processor MATLAB jobs one at a time, shown below is a batch submission script that submit N instances of single-processor MATLAB jobs. To distinguish one job from another, a loop index, essentially, is passed as an environment variable to your m-file. This index may be used as a parameter for your m-file to perform a specific task or it may be used as part of an input file for the specific task.

1. The batch submission script is called submitmjobs, its job is to submit N MATLAB jobs to the batch queue:

   
   #!/bin/csh
   #
   #Usage: submitmjobs ntasks batch_script "SGE options such as wallclock limit"
   #   ntasks -- the number of tasks
   #   batch_script -- batch script, see run_matlab_job for MATLAB app
   #   The OPTIONAL input (in red) must be valid SGE batch parameters.
   # Example: To submit 4 tasks with run_matlab_job and 9-hour wallclock limit
   # katana% submitmjobs 4 run_matlab_job "-l h_rt=09:00:00"
   #
   # The Sun Grid Engine batch preserves the current dir in batch.
   # A different dir may also be used
   # cd my_dir
   @ ntasks = $1
   set batch_script = $2
   set options = "$3"
   @ rank = 0
   while ($rank < $ntasks)
     setenv MATLAB_RANK $rank
     qsub $options -V $batch_script
     @ rank++
   end
   

   Shown below is the batch script for a single MATLAB application, run_matlab_job:

   
   #!/bin/csh
   # this script is a companion script for submitmjobs for MATLAB apps
   unsetenv DISPLAY
   @ rank = $MATLAB_RANK
   # IMPORTANT: DONOT indent any of the below statements
   matlab -nojvm  >! myoutput_$rank << MATLAB_ENV
   myrank = $rank           % works like a script m-file between MATLAB_ENVs
   outname = ['output.' num2str(myrank)] % define unique output file name
   myfunc(myrank, outname)  % how to handle a function m-file
   myscript                 % how to handle a script m-file
   exit                     % don't forget to exit
   MATLAB_ENV
   

   myfunc.m:

   function myfunc(myrank, outname)
   %function myfunc(myrank, outname)
   disp('This is a MATLAB function m-file')
   if myrank == 0
     disp('Perform a task')
   else
     disp('Perform a different task')
   end
   % can also construct outname here with myrank
   save(outname);    % save data
   

   myscript.m:

   disp('This is a MATLAB script m-file')
   % myrank is passed from submitmjob to run_matlab_job to myscript
   disp(['Do work based on the value of myrank = ' num2str(myrank)])
   

   In the above example, four instances of MATLAB are launched. The myoutput_* files trap any output sent to the screen by each MATLAB instance.

2. Change submitmjobs and run_matlab_job execute attribute

    katana:~ % chmod +x submitmjobs
    katana:~ % chmod +x run_matlab_job
   

3. Submit 4 single-processor jobs to the batch queue with a (default) two hours wallclock time each.

   katana:~ % submitmjobs 4 run_matlab_job

   
   Another example is to submit 16 single-processor jobs to the batch queue with a wallclock limit of 9 hours each.

   katana:~ % submitmjobs 16 run_matlab_job "-l h_rt=09:00:00"

Please call (617-353-8294) or email (kadin@bu.edu) Kadin Tseng if you'd like to explore other possibilities of multiprocessing with MATLAB.