IBM pSeries 655

Table of contents

• Overview
• Help Information
• Allocations and Accounting
• File Systems
• Usage policies and batch
• Software
• Compiler notes

Overview 

Our IBM pSeries 655 is a 72-processor system composed of nine nodes, named twister.bu.edu, scrabble.bu.edu, marbles.bu.edu, crayon.bu.edu, litebrite.bu.edu, hotwheels.bu.edu, jacks.bu.edu, playdoh.bu.edu, and slinky.bu.edu. Twister is the interactive machine for the entire set of pSeries machines and is the only one of the machines which users can log in to. The other machines are all reserved for batch processing. The p655 nodes are dividied into two classes of machines. The first six listed above consist of 8 Power4 processors running at 1.1 GHz and sharing 16 GB of memory. The remaning three machines have eight 1.7 GHz processors sharing 8 GB of memory. On all of the p655 machines, there are three levels of cache. Each processor has a 32 KB L1 cache; each pair of processors share a 1.41 MB L2 cache; and each set of eight processors share a 128 MB L3 cache.

From September, 2002 through May, 2009 we also had a set of 16 and 32 processor IBM p690 series machines. However, these have now been phased out.

Users with accounts on these machines must use ssh to log in to twister.bu.edu. Passwords are shared over the Scientific Computing Facilities, so if you already have an account and password on our other systems, you will have the same login and password on this system.

Help Information 

This page provides only very basic information on the IBM pSeries systems. For additional information, please follow the sidebar links.

• Programming for the IBM pSeries systems (http://scv.bu.edu/computation/pseries/programming.html)
• "Introduction to Scientific Computing on Boston University's IBM p-series Machines" talk (PDF) (http://scv.bu.edu/documentation/presentations/pseries_intro/pseries_intro.pdf)

Other documents can be found in our documents section or on IBM's web site. You can also access system man pages.

If you are experiencing system problems, please send Email to help@twister.bu.edu.

For more information or help in using or porting applications to the pSeries systems, please contact Doug Sondak (sondak@bu.edu) or Kadin Tseng (kadin@bu.edu).

If you have questions regarding your computer account or resource allocations, please send Email to scfacct@bu.edu.

Allocations and Accounting 

Our allocations policy is outlined in our SCF Users Information document. On the 1.1 GHz IBM p655 machines, you will be charged 0.85 SUs for use of 1 processor hour, while on the 1.7 GHz machines, you will be charged 1.31 SUs per processor hour. Requests for additional time can be made via the Scientific Computing Facilities Resource Requests pages.

File Systems 

The IBM pSeries systems share a single user filesystem. Home directories are backed up nightly. If you accidentally remove a file, you can request that it be restored by sending email to help@twister.bu.edu. Make sure to specify what machine you were on when you deleted the file(s) and to provide us with the full file pathname(s).

Our file storage section has additional information on what resources are available for storing your files.

Usage policies and batch 

All but one of the nodes on the p655 systems are reserved for batch processing of jobs. The only interactive node is twister.bu.edu. General interactive login sessions are allowed on this machine only and interactive processes are limited to 10 minutes of CPU usage.

The batch system for the pSeries systems is the Load Sharing Facility (LSF). There are a number of different batch queues for various types of jobs. These include "short" and "long" queues for jobs of different running times and separate queues for 1, 4, and 8 processor jobs. In general, the short queues will run with a higher priority than the long queues. It is very important that you submit your job to the appropriate batch queue and you must always specify a queue name when submitting a job, e.g., bsub -q sp-short progname. Details on the available queues are included in the Scientific Computing Facilities Technical Summary.

All jobs that require more than 10 minutes of CPU time must be submitted through the batch system. A system process monitors the processor consumption of all running jobs and will automatically terminate any job which is not running under the batch system and uses more than 10 minutes of processor time.

There is a very nice X-Windows interface to the batch system using the command xlsbatch. You may also use traditional Unix-style commands. To submit a job, use the command bsub. To see the jobs that are queued, use bjobs. To see the queue parameters use the command bqueues. You may remove or kill a job with the command bkill. For more information on the batch system, see LSF Basics.

Software 

Please consult the packages page to see what software packages are available.

Compiler notes 

• For parallel processing, MPI and OpenMP are available.
• Prefixes, suffixes and other basics
  • Compiler names generally have "xl" prefixes, e.g., xlc, xlf, xlf90, xlf95, xlC
  • Highest level of optimization is -O5
  • To automatically link to the MPI library, prefix the compiler name with "mp," e.g. mpxlc
  • For SMP parallelization use "_r" suffix, e.g. xlf_r
  • OpenMP: use -qsmp=omp flag with "_r" compiler name
  • Automatic parallelization: use -qsmp flag with "_r" compiler name
• To run MPI code do NOT use mpirun, e.g., to run on 4 processors:

  myprog < my.in > my.out -procs 4

• For information on scientific libraries on the IBM systems, see http://scv.bu.edu/computation/pseries/programming.html