Addendum

Please refer the quick references guides for the BLACS, PBLAS, and ScaLAPACK for detailed listings of calling sequences.

CALL BLACS_PINFO( IAM, NPROCS )

This routine is used when some initial system information is required before the BLACS are set up. On all platforms except PVM, NPROCS is the actual number of processes available for use (i.e. NPROWS * NPCOLS <= NPROCS). In PVM, the virtual machine may not have been set up before this call, and therefore no parallel machine exists. In this case, NPROCS will be returned as less than one. If a process has been spawned via the keyboard, it will receive MYPNUM of 0, and all other processes will get MYPNUM of -1. This allows the user to distinguish between processes, so that only one reads in data, etc. Only after the virtual machine has been set up (via a call to BLACS_SETUP or SETPVMTIDS) will this routine return the correct values for MYPNUM and NPROCS.

CALL BLACS_SETUP( IAM, NPROCS )

This routine only accomplishes meaningful work in the PVM BLACS. On all other platforms, it is functionally equivalent to BLACS_PINFO. The BLACS assume a static system: you start with a given number of processes, and that is all you will ever have. PVM supplies a dynamic system, allowing processes to be added to the system on the fly. BLACS_SETUP is used to actually allocate the virtual machine and spawn off processes. It reads in a file called blacs_setup.dat, who's first line must be the name of your executable. The second line is optional, but if it exists, it should be a PVM spawn flag. Legal values at this time are 0 (PvmTaskDefault), 4 (PvmTaskDebug), 8 (PvmTaskTrace), and 12 (PvmTaskDebug + PvmTaskTrace). The primary reason for this line is to allow the user to easily turn on and off PVM debugging. Additional lines, if any, specify what machines should be added to the current configuration before spawning NPROCS-1 processes to the machines in a round robin fashion. NPROCS is input on the process which has no PVM parent (i.e. MYPNUM=0), and both parameters are output for all processes. Therefore, on PVM systems, the call to BLACS_PINFO informs you that the virtual machine has not been set up, and a call to BLACS_SETUP then sets up the machine and returns the real values for MYPNUM and NPROCS. Note that if the file blacs_setup.dat does not exist, the BLACS will prompt the user for the executable name, and processes will be spawned to the current PVM configuration.

CALL BLACS_GET( 0, 0, ICONTXT )

Gets the values the BLACS are using for internal defaults. Some values are tied to a BLACS context, and some are more general. The most common use is in retrieving a default system context for input into BLACS_GRIDINIT or BLACS_GRIDMAP. Some systems, such as MPI, supply their own version of context. For those users who mix system code with BLACS code, we therefore need to be able to form a BLACS context in reference to a system context. Thus, the grid creation routines take a system context as input. If you wish to have strictly portable code, you may use BLACS_GET to retrieve a default system context which will include all available processes. This value is not tied to a BLACS context, obviously, so the parameter ICONTXT is unused.

BLACS_GET returns information on three quantities which are tied to an individual BLACS context, which is passed in as ICONTXT. The information which may be retrieved is:

CALL BLACS_GRIDINIT( ICONTXT, 'Row-major', NPROW, NPCOL )

All BLACS codes must call this routine, or its sister routine BLACS_GRIDMAP. These routines take the available processes, and assign, or map, them into a BLACS process grid. In other words, they establish how the BLACS coordinate system will map into the native machine's process numbering system. Each BLACS grid is contained in a context (its own message passing universe), so that it does not interfere with distributed operations which occur within other grids/contexts. These grid creation routines may be called repeatedly in order to define additional contexts/grids.

The creation of a grid requires input from all processes which are defined to be in it. It is therefore a globally blocking operation. Processes belonging to more than one grid will have to agree on which grid formation will be serviced first, much like the globally blocking sum or broadcast.

These grid creation routines set up various internals for the BLACS, and so one of them must be called before any calls are made to the non-initialization BLACS.

Note that these routines map already-existing processes to a grid: the processes are not created dynamically. On most parallel machines, the processes will be actual processors (hardware), and they are "created" when the user runs his executable. When using the PVM BLACS, if the virtual machine has not been set up yet, the routine BLACS_SETUP should be used to create the virtual machine. If the PVM user wishes to use a virtual machine he has already set up using explicit PVM calls, the routine SETPVMTIDS may be used.

This routine creates a simple NPROW x NPCOL process grid. This process grid will use the first NPROW * NPCOL processes, and assign them to the grid in a row- or column-major natural ordering. If these process-to-grid mappings are unacceptable, BLACS_GRIDINIT's more complex sister routine BLACS_GRIDMAP must be called instead.

CALL BLACS_GRIDINFO( ICTXT, NPROW, NPCOL, MYPROW, MYPCOL )

Returns information on the current grid. If the context handle does not point at a valid context, all quantities are returned as -1.

ICALLER = BLACS_PNUM( ICONTXT, MYROW, MYCOL )

This function returns the system process number of the process at {PROW,PCOL} in the process grid.

CALL IGERV2D( ICONTXT, 1, 1, ICALLER, 1, RSRC, CSRC )

These calls receive a message from process {RSRC, CSRC} into the matrix A. This routine is globally-blocking, i.e. return from the routine indicates that the message has been received into A.

CALL SGEBS2D( ICTXT, 'All', ' ', 1, 1, B, 1 )

These routines start a broadcast along a scope. All other processes within the scope must call broadcast/receive for the broadcast to proceed. At the end of a broadcast, all processes within the scope will possess the data in the matrix A. Broadcasts may be globally-blocking. This means no process is guaranteed to return from a broadcast until all processes in the scope have called the appropriate routine (broadcast/send or broadcast/receive).

CALL IGEBR2D( ICTXT, 'All', ' ', 1, 1, NPOINTS, 1, 0, 0 )

These routines receive and participate in a broadcast along a scope. At the end of a broadcast, all processes within the scope will possess the data in the matrix A. Broadcasts may be globally-blocking. This means no process is guaranteed to return from a broadcast until all processes in the scope have called the appropriate routine (broadcast/send or broadcast/receive).

CALL SGSUM2D( ICTXT, 'All', ' ', 1, 1, PARTIALINT, 1, -1, -1 )

The sum is element-wise, i.e., each element of A is summed with the corresponding element of the other process's matrices. Combines may be globally-blocking, so they must be programmed as if no process returns until all have called the routine.

CALL BLACS_EXIT( 0 )

This routine should be called when a process has finished all use of the BLACS. It frees all BLACS contexts and releases all memory the BLACS have allocated. CONTINUE indicates whether the user will be using the underlying communication platform after the BLACS are finished. This information is most important for the PVM BLACS. If CONTINUE is set to 0, then pvm_exit will be called; otherwise, it will not. If the user sets CONTINUE not equal to 0, he is indicating that he will be calling explicit PVM send/recvs after the BLACS are done, so that the process cannot tell the virtual machine that it is done. It then becomes the user's responsibility to make sure his code calls pvm_exit. PVM users should either call BLACS_EXIT or explicitly call pvm_exit to avoid PVM problems.

CALL PSGESV( N, NRHS, A, IA, JA, DESCA, IPIV, B, IB, JB, DESCB, INFO )

Solves a general system of linear equations AX=B.

CALL SLTIMER( 1 )

SLtimer provides a "stopwatch" functionality cpu/wall timer (in seconds). Up to 64 separate timers can be functioning at once. The first call starts the timer, and the second stops it. This routine can be disenabled, so that calls to the timer are ignored. This feature can be used to make sure certain sections of code do not affect timings, even if they call routines which have SLtimer calls in them.

CALL SLCOMBINE( ICTXT, 'All', '>', 'C', 1, 1, CTIME )

SLCOMBINE takes the timing information stored on a scope of processes and combines them into the user's TIMES array.