REGRESSION
Linear Regression

REGRESSION is a FORTRAN90 library which handles problems in linear regression.

These routines were collected and described by Helmut Spaeth. The most common task for such routines is to solve an overdetermined system of linear equations.

In particular, many routines will produce a least-squares solution. That is, given an M by N matrix A, and an M vector B, the routines will seek an N vector X so which minimizes the L2 norm (square root of the sum of the squares of the components) of the residual

        R = A * X - B
      

The code is in a very provisional state. Many routines have not been carefully proofread or debugged or tested yet.

Licensing:

The computer code and data files described and made available on this web page are distributed under the GNU LGPL license.

Languages:

REGRESSION is available in a FORTRAN90 version.

Related Data and Programs:

REGRESSION, a dataset directory which contains datasets for testing linear regression;

References:

1. I Barrodale, C Phillips,
   Algorithm 495: Solution of an Overdetermined System of Linear Equations in the Chebyshev Norm,
   ACM Transactions on Mathematical Software,
   Volume 1, pages 264-270, 1975.
2. I Barrodale, F Roberts,
   Algorithm 552: Solution of the Constrained L1 Linear Approximation Problem,
   ACM Transactions on Mathematical Software,
   Volume 6, pages 231-235, 1980.
3. Golub, Van Loan,
   Matrix Computations,
   The Johns Hopkins University Press, 1983.
4. Richard Gunst, Robert Mason,
   Regression Analysis and Its Applications: a data-oriented approach,
   Dekker, 1980,
   ISBN: 0824769937,
   LC: QA278.2.G85.
5. Charles Lawson and Richard Hanson,
   Solving Least Squares Problems,
   Prentice-Hall, 1974,
   Revised edition, SIAM, 1995.
6. Reinsch, Golub,
   Singular Value Decomposition and Least Squares Solutions,
   Numerische Mathematik,
   Volume 14, 1970.
7. Helmuth Spaeth,
   Mathematical Algorithms for Linear Regression,
   Academic Press, 1991,
   ISBN 0-12-656460-4.

Source Code:

• regression.f90, the source code;
• regression.sh, commands to compile the source code;

Examples and Tests:

• regression_test.f90, sample calling program;
• regression_test.sh, commands to compile, link and run the sample calling program;
• regression_test.txt, the results file.

List of Routines:

• A328_LI minimizes the L-infinity norm of A*x-b.
• A478_L1 minimizes the L1 norm of A*x-b using the modified simplex method.
• A478_L1_COL is used by A478_L1.
• A495_LI minimizes the L-Infinity norm of A*x-b using a simplex method.
• ABD_LI minimizes the L-infinity norm of A*x-b using a simplex method.
• AFK_L1 minimizes the L1 norm of A*x-b.
• AFK_L1_CALBET is used by AFK_L1 to compute the solution.
• AFK_L1_UPDATE updates the LU decomposition of a matrix.
• AVLLSQ carries out average linear regression.
• BLOD_L1 minimizes the L1 norm of A*x-b.
• BLOD_L1_CRIT returns three sums of entries of U.
• BLOD_L1_GET1 is used by BLOD_L1.
• BLOD_L1_GET2 is used by BLOD_L1.
• BLOD_L1_MED3 is used by BLOD_L1.
• CH_CAP capitalizes a single character.
• CH_EQI is a case insensitive comparison of two characters for equality.
• CH_IS_DIGIT returns .TRUE. if a character is a decimal digit.
• CH_TO_DIGIT returns the integer value of a base 10 digit.
• C01M generates a new combination from an old one efficiently.
• CON_L1 minimizes the L1 norm of A * X - B subject to linear constraints.
• CON_L2 minimizes the L2 norm of A * X - B subject to linear constraints.
• CON_LI minimizes the L-infinity norm of A * X - B subject to linear constraints.
• CWLR_L1 minimizes the L1 norm of A*x-b using clustering techniques.
• CWLR_L2 minimizes the L2 norm of A*x-b using clustering techniques.
• CWLR_LI minimizes the L-infinity norm of A*X-B.
• DIGIT_INC increments a decimal digit.
• DIGIT_TO_CH returns the character representation of a decimal digit.
• EXAMPLE_MULTI_SIZE returns values for a multiple system file.
• EXAMPLE_PRINT prints data from an example file.
• EXAMPLE_READ reads data from an example file.
• EXAMPLE_SIZE returns the values of M and N in an example file.
• FILE_NAME_INC generates the next filename in a series.
• G1 computes an orthogonal rotation matrix.
• GEN generates a random matrix A and right hand side B.
• GET_UNIT returns a free FORTRAN unit number.
• GIVR_L2 minimizes the L2 norm of A*x-b using fast Givens rotations.
• HFTI minimizes the L2 norm of A*x-b using Householder transformations.
• HFTI_L2 minimizes the L2 norm of A*x-b using Householder transformations.
• H12 constructs or applies a Householder transformation.
• I_LOG_10 returns the integer part of the logarithm base 10 of ABS(X).
• I_RANDOM returns a random integer in a given range.
• I_SWAP switches two integer values.
• I_TO_S_ZERO converts an integer to a string, with zero padding.
• ICMGS_L2 uses modifed Gram Schmidt on a problem with nonzero intercept.
• INEXCL computes auxilliary arrays F, T and R used to control exchanges.
• IVEC_IDENTITY sets an integer vector to the identity vector A(I)=I.
• IVEC_PRINT prints an integer vector.
• LDP_L2 implements least distance programming algorithm.
• MGS_L2 minimizes the L2 norm of A*x-b using the modified Gram-Schmidt method.
• NN_L1 minimizes the L1 norm of A * X - B with linear constraints and X >=0.
• NN_L2 minimizes the L2 norm of A * X - B with X >=0.
• NN_LI minimizes the L-infinity norm of A * X - B with linear constraints and X >=0.
• NORMAL_L2 minimizes the L2 norm of A*x-b using the normal equations.
• NPART_ENUM enumerates the number of partitions of N with NPART parts.
• NPART_RSF_LEX_RANDOM returns a random RSF NPART partition.
• NPART_RSF_LEX_UNRANK unranks an RSF NPART partition in the lex ordering.
• NPART_TABLE tabulates the number of partitions of N having NPART parts.
• ORTH_L1 carries out orthogonal regression in the L1 norm.
• ORTH_L2 carries out orthogonal regression in the L2 norm.
• ORTH_LI carries out orthogonal regression in the L-infinity norm.
• ORTH_LM is a least squares solver for linear manifolds.
• ORTH_LP carries out orthogonal regression in the LP norm.
• PERM_RANDOM2 selects a random permutation of N objects.
• PYTHAG computes SQRT ( A**2 + B**2 ) carefully.
• QRBD uses the QR algorithm for the singular values of a bidiagonal matrix.
• R_DIFF computes the difference ( X - Y ) of two real numbers.
• R_NEXT "reads" real numbers from a string, one at a time.
• R_RANDOM returns a random real in a given range.
• R_SWAP switches two real values.
• RANDOM_PARTITION generates a random partition.
• RANDOM_PARTITION2 generates a random partition with occupancy constraints.
• REGR_LP minimizes the LP norm of A*x-b for P > 1.
• RESIDUAL calculates the residual vector A*X-B and related information.
• RMAT_CHOLESKY_FACTOR computes the Cholesky factor of a symmetric matrix.
• RMAT_CHOLESKY_SOLVE solves a Cholesky factored linear system A * x = b.
• RMAT_DIAG_ADD_SCALAR adds a scalar to the diagonal of a matrix.
• RMAT_INDICATOR sets the indicator matrix.
• RMAT_L_SOLVE solves a lower triangular linear system.
• RMAT_LT_SOLVE solves a transposed lower triangular linear system.
• RMAT_PRINT prints a real matrix.
• ROBUST carries out robust regression, with eight choices for the method.
• RR_L1 carries out ridge regression in the L1 norm.
• RR_L2 carries out ridge regression in the L2 norm.
• RR_LI carries out ridge regression in the L-infinity norm.
• RVEC_NORM_LI returns the L-infinity norm of a vector.
• RVEC_NORM_L1 returns the L1 norm of a vector.
• RVEC_NORM_LP returns the LP norm of a vector.
• RVEC_NORM_L2 returns the L2 norm of a vector.
• RVEC_BIN bins a real vector, returning the population of each bin.
• RVEC_INDICATOR sets a real vector to the indicator vector.
• RVEC_PRINT prints a real vector.
• S_TO_I reads an integer value from a string.
• S_TO_R reads a real number from a string.
• SCR selects M by NA submatrices from an M by N matrix.
• SCRF_L1 minimizes the L1 norm of A*X-B using NA variables out of N.
• SVD computes the singular value decomposition for a real matrix.
• SVDR_L2 minimizes the L2 norm of A*x-b using the SVD.
• SVDRS: singular value decomposition with a right side vector.
• TIMESTAMP prints the current YMDHMS date as a time stamp.
• UNIFORM_01_SAMPLE is a portable random number generator.
• URAND returns a uniformly distributed pseudo random number.

You can go up one level to the FORTRAN90 source codes.