D.7.3. FMS Matrix Factoring Parameter NBITSZ

Data Type

Integer

Default Value

Description

This parameter controls the tolerance on zero when computing diagonal pivots during the factoring process.

During matrix factoring the diagonal terms (pivots) must be inverted. If their value is zero, their inverse cannot be computed. Because machines have finite precision, numbers "close" to zero should be treated as zero for this condition. This parameter defines what FMS uses for "close".

In order to establish a meaningful definition of "close to zero", it is necessary to examine how the diagonal pivots are computed. The actual calculation performed by FMS to factor the diagonals is given by

   D = A - S
   P = 1/D

where

A is the original matrix diagonal,
S is the dot product of the ith row of [AL] and the ith column of [AU] and
P is the pivot reciprocal for the diagonal that is being computed.

If the value of S is close to A, the calculation of P will be in error. To see how this error occurs it is useful to examine this calculation from a machine's point of view. The values of D and S are stored as floating point numbers. For most machines today, that means 64 bits in IEEE format. Within these 64 bits, 11 are used to represent the exponent and 53 for the mantissa. In general, this provides about 15 decimal digits of accuracy.

When two numbers are "close" in value, their exponents and leading mantissa bits will be identical. If these numbers are subtracted, the result will have the same exponent but the leading mantissa bits which were identical will be set to zero. The machine then normalize the result by shifting the remaining nonzero mantissa bits to the left, inserting zero's in their place and appropriately adjusting the exponent. As you can expect, the number of zero's inserted into the mantissa during this process provides a measure of how close the numbers are in value and how much precision is lost in computing their difference.

FMS monitors this process and provides a way to treat the result of the subtraction as zero if a large number of zero's were inserted. The default value of NBITSZ=39 will treat the difference of two numbers as zero if their exponent and leading 39 bits of mantissa are identical. If you want to loosen this tolerance and increase the range of numbers that will be treated as zero, set the value of NBITSZ to a smaller number. If you want to tighten the tolerance, increase the value of NBITSZ.

NOTE: If the absolute value of (A - S) is less than the FMS Parameter PIVMIN, a zero pivot condition also results. (For complex data the value of (A - S) squared is compared to PIVMIN).