JTB_CODIST
Codistributed Array Example for Parallel Computing Toolbox

JTB_CODIST is a MATLAB program which demonstrates how the linear system associated with a finite element problem can be treated as a codistributed array whose entries are assigned to different MATLAB workers, so that the matrix is assembled in parallel.

The program produces an approximate (steady-state) solution to a heat equation on a rectangular domain (see 2D_heat_ex.pdf for details). The code uses the MATLAB Parallel Computing Toolbox's Single Program Multiple Data (spmd) mode. A matrix assembly step is carried out by the assemb_co.m code. It contains a line that invokes the spmd mode and this line must be visible (i.e. not a commented line).

Problem data are given in p_data.m In particular, the grid parameters are given by:

        param.nodesx =   51; % (=nx)
        param.nodesy =   51; % (=ny)
      

This code uses the ILU preconditioner and the GMRES solver to solve the linear system. These routines are currently not 'overloaded' for codistributed arrays, so we 'gather' the required arrays on lab #1. We expect that GMRES (at least) will be overloaded to handle codistributed arrays in upcoming releases of the software.

The code produces a collection of ASCII text files (outj, one for each lab). These files contain brief information about the columns of an FEM matrix assembled on that lab, and on what fraction of the elements were evaluated.

To run this code begin by opening a matlabpool (assuming one os not running), then issue the command run_spmd. The Command Window should look like this:

>> matlabpool open 'local' 2
Starting matlabpool using the parallel configuration 'local'.
Waiting for parallel job to start...
Connected to a matlabpool session with 2 labs.
>> run_spmd
Elapsed time is 1.570656 seconds.
gmres(200) converged at outer iteration 1 (inner iteration 70) to a solution with relative residual 7.4e-07
Elapsed time is 2.413968 seconds.
>> 
      

run_pmode.m performs the same overall task as run_spmd.m but uses the MATLAB Parallel Computing Toolbox interactive pmode interface. In this case the spmd command in assemb_co.m must be commented out. Problem data are specified as above.

This code uses the 'backslash' operator \ (mldivide or matrix-left-divide) on codistributed array(s). Unfortunately, while mldivide has been overloaded for codistributed arrays, it is not (yet ?) available for arrays with the 'sparse' attribute. Thus, the arrays are converted to 'full' before the solve step(s).

Here we output plots of the steady-state solution, and the state for transient step 1 and transient step 2.

For this run, first 'comment out' the spmd line (62) in assemb_co.m. Then initiate PMODE (pmode start 2). Check to be sure you're in the correct directory, else use 'cd' appropriately. In the PMODE window type run_pmode . This will produce some timing results in the PMODE Command Windows. along with the three .png files (plots).

Author:

Gene Cliff, Aerospace and Ocean Engineering / Interdisciplinary Center for Applied Mathematics, Virginia Tech.

Licensing:

The computer code and data files described and made available on this web page are distributed under the GNU LGPL license.

Related Data and Programs:

COLLATZ_PARALLEL is a MATLAB program which seeks the maximum Collatz sequence between 1 and N; it runs in parallel using MATLAB's "parfor" facility.

FDI_OPT, a MATLAB program which demonstrates the use of MATLAB's FMINCON constrained minimization function, taking advantage of MATLAB's Parallel Computing Toolbox for faster execution.

LINEAR_SOLVE_DISTRIBUTED is a MATLAB program which solves a linear system A*x=b using MATLAB's spmd facility, so that the matrix A is "distributed" across multiple MATLAB workers.

MATLAB_PARALLEL, examples which illustrate "local" parallel programming on a single computer with MATLAB's Parallel Computing Toolbox.

MD_PARALLEL is a MATLAB program which carries out a molecular dynamics simulation in parallel, using MATLAB's "PARFOR" feature.

PRIME_NUMBER_PARALLEL is a MATLAB program which counts the number of primes between 1 and N; it runs in parallel using MATLAB's "parfor" facility.

PRIME_NUMBER_SPMD is a MATLAB program which counts the number of primes between 1 and N; running in parallel using MATLAB's "SPMD" feature.

QUAD_SPMD is a MATLAB program which estimates an integral using quadrature; running in parallel using MATLAB's "SPMD" feature.

SATISFIABILITY_PARALLEL is a MATLAB program which demonstrates, for a particular circuit, an exhaustive search for solutions of the circuit satisfiability problem, running in parallel using MATLAB's "PARFOR" feature.

TIMING_PARALLEL is a directory of MATLAB programs which illustrates how to time a parallel MATLAB program.

Reference:

MathWorks documentation for the Parallel Computing Toolbox is available at http://www.mathworks.com/products/parallel-computing/.

1. Gene Cliff,
   Working Notes on a 2D Thermal Model with Parallel Sensitivity Calculations,
   2d_heat_ex.pdf.
2. The MathWorks,
   Parallel Computing Toolbox 4,
   User's Guide.
3. The Mathworks,
   Parallel Computing Toolbox Release Notes,
   The Mathworks, 2009.
4. Virginia Tech Advanced Research Computing,
   Notes on Enabling Remote Submission of MATLAB Jobs,
   matlab_remote_submission.pdf.

Source Code:

• run_spmd.m, the source code to run using SPMD.
• run_pmode.m the source code to run using PMODE.

Examples and Tests:

MATLAB codes:

• assemb_co.m assemble the finite element system matrix as a codistributed array.
• boundary2.m compute values of 2d quadratic basis functions at 1D Gauss points along the boundary.
• my_extract.m a routine for locating the nodes in node_local that correspond to columns on the lab.
• oned_f_int.m computes the integral of f(x) times a test function.
• oned_gauss.m Gauss quadrature rules for 1D intervals.
• p_data.m sets problem parameters.
• source.m evaluates source terms.
• steps.m evaluates a step function.
• twod_bilinear.m evaluates the 2D finite element integral.
• twod_f_int.m computes the integral of f(x,y) times a test function.
• twod_gauss.m Gauss quadrature rules for 2D regions.
• twod_shape.m evaluates test functions in 2D.

Other files:

• out1.txt, the report file from lab #1;
• out2.txt, the report file from lab #2;
• fig_ss.png, an image of the steady-state solution;
• fig_1.png, an image of transient step 1;
• fig_2.png, an image of transient step 2;

You can go up one level to the MATLAB source codes.