petsc_test

petsc_test FORTRAN90 programs which demonstrates the use of the PETSc library.

PETSc is a library of high level mathematical routines that can be executed in a parallel environment, making it easier for a user to gain the advantages of MPI.

PETSc stands for the Portable Extensible Toolkit for Scientific Computations.

PETSc is particularly suitable for the numerical solution of systems of partial differential equations on high performance computers. PETSc includes parallel linear and nonlinear equation solvers and time integrators that may be used by application programs written in C, C++, FORTRAN77 or FORTRAN90.

PETSc consists of a number of libraries which work with a particular family of objects, such as vectors. Some of the PETSc modules deal with

• index sets, including permutations;
• vectors;
• matrices, especially sparse matrices;
• distributed arrays, as when the data for a single matrix is distributed over a set of processors;
• Krylov subspace methods for solving linear systems;
• preconditioners for linear algebra systems, including multigrid and sparse direct solvers;
• solvers for systems of nonlinear equations;
• partial differential equation solvers;
• timesteppers for solving time-dependent partial differential equations;

PETSc includes parts of the following software libraries:

• BLAS1 vector library,
• BLAS2 matrix-vector library,
• BLAS3 matrix-matrix library,
• LAPACK linear algebra library,
• LIBTFS, an efficient parallel direct solver of Henry Tufo and Paul Fischer,
• LINPACK linear algebra library,
• MINPACK minimization library,
• SPARSEKIT2, Yousef Saad's sparse matrix library,
• SPARSEPAK, matrix reordering routines.

PETSC interacts with, complements, or supports a variety of programs including:

• MATLAB; PETSC can dump data in a format that MATLAB reads, or a running PETSC process can interact with a running MATLAB process, passing data back and forth "interactively".
• METIS; the METIS mesh partitioning program is designed to partition a mesh before a parallel process begins; PETSC provides an interface to a parallel version of METIS, called PARMETIS, which can partition the mesh during the parallel computation.

PETSc runs in parallel by relying on MPI, the message-passing

When PETSc is used by a FORTRAN90 program, each FORTRAN90 routine must add one or more include statements, so that the constants and functions are properly declared. Depending on the routines being used, there may be several include statements required. However, if a FORTRAN90 routine uses any PETSc routine or constant, it is always necessary to have at least the following:

      #include "include/finclude/petsc.h"
      

Usage:

Before using PETSc, the user must set the environment variable PETSC_DIR, indicating the full path of the PETSc home directory. On Phoenix, for instance, this might be done with the command

setenv PETSC_DIR /usr/local/petsc

Before using PETSc, the user must set the environment variable PETSC_ARCH, indicating the architecture of the machine on which PETSc is to be run. On Phoenix, for instance, this might be done with the command

setenv PETSC_ARCH linux-gnu

The commands required to compile, link and load a program with PETSc are complicated. It is best to use a makefile for this purpose, in which case most of the complications can be hidden. A bare bones makefile for the example ex1.c, which is going to use PETSc's Krylov Subspace Package (KSP), would look like this:

include ${PETSC_DIR}/bmake/common/base

ex1: ex1.o
        -${FLINKER} -o ex1 ex1.o ${PETSC_FORTRAN_LIB} ${PETSC_LIB}
      

make ex1

The mpirun command may be used for small jobs that run for a short time on a few processors. Repeated use of the mpirun command for large long jobs is an abuse of the system and will not be allowed.

To run a program that uses PETSc, the user may invoke the mpirun command, specifying the number of processors.

mpirun -np 4 ex1

To run a program that uses PETSc, the CONDOR system is preferable. This is a batch system which tries to be fair and efficient in the allocation of computing resources. To use CONDOR, the user should create an executable program, then prepare a "CONDOR submit script" that describes how the executable is to be run, and on how many processors, and submit the script.

Languages:

petsc_test is available in a C++ version and a FORTRAN90 version.

Related Data and Programs:

BLACS is a FORTRAN90 example directory which illustrates how to use the Basic Linear Algebra Communication Subroutines for linear algebra computations on a distributed memory machine.

BLAS1 is a FORTRAN90 library containing vector-vector routines, and is included in PETSC.

BLAS2 is a FORTRAN90 library containing matrix-vector routines, and is included in PETSC.

BLAS3 is a FORTRAN90 library containing matrix-matrix routines, and is included in PETSC.

CONDOR, is a queueing system used locally to submit jobs to our parallel clusters.

LAPACK is a FORTRAN90 library which is an extensive linear algebra library, and is included in PETSC.

LINPACK is a FORTRAN90 library which is an extensive linear algebra library, and is included in PETSC.

MINPACK is a FORTRAN90 library which handles minimization and least squares, and is included in PETSC.

MPI is used as part of the PETSC library. You may be interested in seeing examples of that parallel programming system.

OPEN_MP is a directory of FORTRAN90 examples which illustrate the use of the OpenMP application program interface for carrying out parallel computations in a shared memory environment.

PESSL is a FORTRAN90 example directory which demonstrates the use of PESSL, the parallel mathematics library developed exclusively for use with IBM systems and compilers.

PETSc examples are also available in a C version, and a C++ version, and a FORTRAN77 version.

SPARSEKIT2 is a FORTRAN90 library of routines by Yousef Saad for sparse matrices, and is included in PETSC.