Running Jobs

Single processor jobs that require approximately ten minutes or less of runtime may run on the login node cootie.bu.edu or in batch. Single processor jobs requiring over ten minutes of runtime and all multiprocessor jobs must be submitted to the batch queue for running on the compute nodes. There are four different types of batch jobs requiring somewhat different batch script preparations. They are explained in the Preparing a batch script section below.

Batch queue technical summary

• Batch job scheduler is PBS. Common commands are: qsub, qstat, and qdel.
• There is only one batch queue for the Cluster. It is named "dque."
• Nodes are assigned to the job at runtime and are not known a priori.
• Each node has 1 GB of physical memory which is shared by its two processors. By default, only one processor is active unless both are requested specifically.
• Virtual (swap) memory is available.
• The maximum number of nodes a user can request is 24 (for a total of up to 48 processors if ppn=2 is specified).
• See the Technical Summary section for additional information.

Preparing a batch script  

Depending on whether the job requires a single or multiple processors and whether it is to be run in the background or interactively, the batch scripting procedure may differ. Details on the different methods available are included below (as well as via the side bar on the right).

• Serial (single processor) batch jobs
• MATLAB batch jobs
• Multiprocessor MPI batch jobs
• Interactive batch jobs

Submitting a batch job

The batch scheduler for the Linux Cluster is the open source version of PBS. All batch jobs must be submitted to the batch queue along with a batch script furnished by the user. Shown below are various examples on how to submit, monitor, and cancel submitted jobs. All bracketed items are optional ([ ... ]).

• The simplest way to submit a batch job
  cootie% qsub MY_BATCH_SCRIPT
  
  where MY_BATCH_SCRIPT is a shell script that contains instructions for PBS, such as the number of processors, wallclock time limit, and executable name.
• To query the status of batch jobs
  cootie% qstat
• To delete a batch job in the batch queue
  cootie% qdel JOBID
• To request one or more nodes for interactive usage
  cootie% qsub -I [-l nodes=N]
• Overriding batch parameters of a batch script via qsub
  Often it is more convenient to make minor changes to a prepared batch script via the command line than to change the script.
  cootie% qsub [-l nodes=N[:ppn=P]] [-l walltime=TIME] [more flags] MY_BATCH_SCRIPT
• N is the number of nodes (up to a maximum of 24 nodes or 48 processors).
• P is the number of processors per node (P=1 or P=2); P=1 is default.
• TIME is the runtime of the job, specified as HH:MM:SS.
• additional (optional) FLAGS are documented in the qsub manpage.
• MY_BATCH_SCRIPT is a shell script that contains instructions for PBS. See above section for examples.
Example. Predefined with nodes=4:ppn=1 and a 2-hour wallclock in MY_BATCH_SCRIPT and overrided on the command line as follows:
cootie% qsub -l nodes=8:ppn=2 -l walltime=24:00:00 MY_BATCH_SCRIPT
• To charge a batch job to a project
  A batch job is normally charged to the user's default project. If the user works on a single project or if the charge should be levied against the default project, no user action is required. On the other hand, users working on multiple projects may, at times, need to charge a batch job to a non-default project. Note that the charging procedure varies among all SCV machines ( See FAQ, Project Accounting). Please consult the respective machine's runningjobs webpage for the correct procedure. Described below are the charging procedures for the Linux Cluster.
  1. Single processor jobs

     cootie:~ % qsub -W group_list=project-name my_batch_script

  2. Multiprocessor jobs
     Multiprocessor jobs are always charged to the default project. To charge the run to a non-default project, you must designate that project to be the default project.
• To find out the projects of which you are a member

  katana:~ % groups 
  my_default_project  my_second_project  my_third_project . . .
  

  The first on the list is always the default project which can be changed.