> '` bjbj rKdHHHHCCCDEXudF8H(`H`H`H;I.iI}Iwqyqyqyqyqyqyq$vh$yqQHU;I;IUUqHH`H`Ht.jjjUH`HH`HwqjUwqjjHHj`HxFУWCbjKn,u<XujyiyjjyHjIcMj#P4WRIIIqqYj^IIIXuUUUU;CCHHHHHHQualitative Matrices and Information
by
Andrew J. Buck
George M. Lady
ABSTRACT
This paper proposes a method for assessing the information content and validity of a mathematical structural model for which only the directions of influence among its endogenous and exogenous variables are known, as expressed by the sign patterns of associated arrays. The traditional literature on this issue presents extremely restrictive conditions under which such a qualitative analysis can be conducted. As a result, there have been very few successful applications of the traditional method. We propose a means of vastly expanding the scope of such an analysis to virtually any applied model. Our method works with the restrictions found for the sign patterns of complete rows and columns, or even the entire sign pattern, of the reduced form, rather than only individual entries. The information provided by the model is measured by the Shannon entropy of the possible sign patterns of the reduced form and the frequency of occurrence of each possibility. An example of the method is provided for Kleins Model I. Although this model has been used for over fifty years for a variety of purposes, we found that the sign pattern of the estimated, unrestricted reduced form from the original data set was not consistent with the proposed, structural directions of influence among the models variables.
Key Words: Qualitative Analysis, Entropy, Falsification, Comparative Statics
Department of Economics
College of Liberal Arts
Temple University
Philadelphia, PA 19122
Qualitative Matrices and Information
I. Introduction. This paper considers the enterprise of reaching conclusions about the sign pattern of a matrix inverse based upon a knowledge of the sign pattern of the matrix being inverted. Or more generally, as spelled out in the next section, reaching conclusions about the sign pattern of the matrical, reduced form of a model based upon the sign pattern(s) of the array(s) specified by a structural hypothesis. Traditionally, such a qualitative analysis focused upon the sign pattern of the reduced form on an entry by entry basis. The conditions under which the sign of any entry of the reduced form can be determined based upon a qualitative analysis are extremely restrictive and are rarely found in applied models. We expand the scope of a qualitative analysis to include an inspection of the sign patterns of entire rows and columns of the reduced form, and even the sign pattern of the reduced form itself. In doing this we find that even though no specific entry of the reduced form may be determined by a qualitative analysis, it can nevertheless be shown that the sign patterns of rows, columns, or even the entire reduced form, may be limited, often extremely limited, to a relatively small number of allowable possibilities. We propose further that the information content of the sign patterns of the structural arrays can be expressed by the Shannon (1948) entropy computed from the frequency distribution of the corresponding, allowable forms of the sign patterns for the reduced form. Expanded in this way, a qualitative analysis can be applied to a very much larger class of applied model, perhaps to any applied model with a fully expressed, qualitative structural form. As a result, structural models expressed qualitatively may now be directly compared to one another on the basis of their information content. The can also be brought to the data to determine if the sign pattern of the estimated reduced form falls outside the limits implied for it by the sign patterns of the structural array(s).
In the next section a brief summary is provided of the basis for an interest in qualitative analyses in economics, the source of our interest in the subject matter. In section III an expanded qualitative analysis is outlined. A Monte Carlo procedure for conducting an expanded qualitative analysis is briefly specified in section IV. In section V an example is provided for Kleins Model I, a model that has received repeated attention in the literature on qualitative analyses and which is detailed in many undergraduate and graduate econometrics textbooks. Conclusions and some outstanding issues are summarized in section VI.
II. Background. For economists, working with a matrix sign pattern to see if something could be determined about the sign pattern of the matrix inverse, or more generally the reduced form as outlined below, was first brought up by Samuelson (1947). The issue arose as follows: Samuelson proposed that economic theory should be understood to organize aspects of how the economy works by mathematical models expressed by systems of equations, such as:
f i (Y, Z) = 0, i = 1, 2, , n, (1)
where Y is an nvector of endogenous variables and Z is an mvector of exogenous variables. The system is studied by the method of comparative statics. This technique assesses the effects of disturbances in the entries of Z on the entries of Y with respect to a referent solution as specified by a linear system of differentials:
EMBED Equation.DSMT4
In econometrics, the relationships in (1) are often assumed to be (at least approximately) linear; and, given this, (1) and (2) can be represented by the linear system (absent an error term),
Y = Z, (3)
where and are appropriately dimensioned matrices. (3) is usually called the structural form of the model. The hypothesis represented by (3) can be brought to the data by estimating the entries of in what is usually called the reduced form:
Y = Z, for = 1. (4)
The scientific content of the theory represented by (1) and (2), i.e., the degree to which it embodies refutable hypotheses; or, following Popper (1934), the degree to which it can be falsified, relates to how the hypothesis (3) limits the outcome of the estimate of (4); and, whether these limits are observed when is estimated from data.
Samuelson (op. cit.) pointed out that in economics the arrays that express the hypothesis (3) might not have all, or even any, entries expressed quantitatively; but instead, the information about each entry might only be its sign. And, we are quick to add, not even necessarily that, but only whether or not the value involved is nonzero. As a result (say), if only the sign patterns of {, } were known, it becomes problematic as to what can necessarily be deduced about the nature of . An immediate hope would be that, given the signs of the entries of {, } and working through the algebra of the relationship = 1, something could be determined about at least some of the signs in . The process of doing this is termed a qualitative analysis. If the sign of an entry of the reduced form found by a qualitative analysis is not also found when the reduced form is estimated, then the hypothesis presented by the signs of {, } is falsified.
The main burden of a qualitative analysis is finding signs in 1 based only upon the sign pattern of . Samuelson felt that the chances for a successful outcome of such a qualitative analysis were not promising. He argued, in effect, that it was most unlikely that knowledge of the sign pattern of would enable anything to be said about the signs of any of the entries of 1. His conclusion was based upon the simple observation that all of the terms of the expansions of a matrix determinant and (at least some) cofactors would be very unlikely to all have the same sign in any one of those expansions.
III. An Expanded Qualitative Analysis. Not withstanding Samuelsons misgivings, a literature on the conditions under which a qualitative analysis could be successfully conducted evolved. That literature, typically, considered a special case of (3) and (4) in which n = m and = I. Lancaster (1962) provided sufficient conditions for the form of s sign pattern that allowed at least some of the signs in 1 to be determined. Basset, Maybee, and Quirk (1968) provided necessary and sufficient conditions for a successful qualitative analysis for the sign pattern of put into a standard form. Lady (1983) provided similar necessary and sufficient conditions for a successful qualitative analysis for s sign pattern put into a slightly weaker standard form, plus algorithmic principles for constructing such systems. Starting with Lancaster (op.cit), there was attention in the literature to the problems of conducting a successful qualitative analysis. A good deal of this is cited in Hale, et al (1999). The conditions on the sign patterns of {, } that allow at least some of the signs of the entries of to be necessarily determined are well inhand.
None of this literature dispelled Samuelson s original observation that a successful qualitative analysis was unlikely. Further, the literature on attempts to conduct a qualitative analysis is sparse. Of the attempts that were made that we know of (e.g., Ritschard (1983), Maybee and Weiner (1988), Lady (2000), and Buck and Lady (2005)) the conditions for a successful qualitative analysis were not satisfied (Hale and Lady (1995) is a notable exception). These attempts added quantitative information to help figure out the signs of the sums of terms in the expansions of the determinant and (at least some) cofactors when terms of the opposite sign were present (as Samuelson predicted would generally be the case). Usually, as practiced, the qualitative analysis of an actual model provided a useful inspection of the inference structure of the model; however, it was in general an open ended process of considering special cases utilizing other information in addition to the sign patterns of {, }.
The major point of this paper is that this entire literature, starting with Samuelson to the current day, is unnecessarily restrictive and fails to take into account that a hypothesis provided by the sign patterns of {, }, or even in part by simply knowing that some entries are nonzero and some of them not, can provide considerable information about (i.e., impose limitations upon) the signs that may be taken on by the entries of . Indeed, based upon our past experience with qualitiative inverses and our experience in developing the examples presented here, we conjecture (but do not try to prove) that any fully specified sign patterns for {, } place some kind of limit on the sign patterns that can be taken on by . And, given this, any such hypothesis regarding {, } can be potentially falsified. Further, as described below, the hypothesis can also have its information content measured.
In the development of concepts and examples below we limit our scope to computationally nonsingular instances of . This is not overly restrictive from the perspective of either the mathematical content of the approach or its practical significance. To now, a successful qualitative analysis usually required that be sign nonsingular (qualitatively invertible), the conditions for which are very much more stringent than the computational restrictions on invertibility. Accordingly, the significant departure of our approach, compared to that of the literature cited above, is that we will consider matrices which, based upon their sign patterns and/or other conditions on their nature, don't, necessarily, meet the criteria for sign nonsingularity and could conceivably be singular. In general, almost all actual applied models could be singular, but nevertheless they virtually never are singular..
In addition, to facilitate the analysis we will assume that is irreducible, i.e., that no entries of 1 must be zero and additionally that no entries of 1 are otherwise computationally equal to zero. We will note the implications of relaxing these assumptions in the next section. Finally, the arrays (, ) are specified as follows:
(a) which entries are zero and which not;
(b) the signs of (at least some of) the nonzero entries; and,
(c) distributional rules to which the values of the nonzero entries must conform.
Given this, let CQ(, ) be the set of all quantitative realizations of {, } consistent with nonsingular and the assumptions (a), (b), (c) above; and, let RF(sgn ) be the set of sign patterns for the corresponding reduced forms, where = 1. The issues to resolve are: Given CQ(, ), what are the members of RF(sgn ); and, what are the frequencies of their occurrence?
As an example, let n = m = 2, = I, and the hypothesis is,
EMBED Equation.DSMT4
Further, let the absolute values of each entry of be randomly chosen from the uniform distribution,
0 < abs(ij) < 10.
For this simple example it is easy to see that when is nonsingular, as it almost inevitably is, its determinant will be positive or negative, each half of the time. Accordingly, each of the entries of 1 will be all positive or all negative, each half of the time. For a traditional qualitative analysis, that generally would be the end of the story. The given sgn is not qualitatively invertible and none of the entries in 1 can be conclusively signed. Our point is that there is, nevertheless, quite a bit of information provided by sgn about the characteristics of RF(sgn ). Specifically, s adjoint is entirely signed (although our ideas do not depend on this) and has all negative entries. As a result, sgn 1 can only be all positive, or all negative, each half the time. A 2 x 2 matrix, barring zeros, can have any of sixteen sign patterns. For our example, the hypothesis sgn and = I, limits the members of RF(sgn ) to just two of the sixteen possibilities, each appearing half the time. That is, the hypothesis sgn precludes any outcomes for 1 other than the ones in which the elements of 1 are either all negative or all positive.
We propose that the information provided by the hypothesis sgn and = I be measured by the Shannon entropy of the frequency distribution found for the members of RF(sgn ). Let Fi be the frequency of the ith sign pattern that appears in RF(sgn ) and Q be the corresponding set of all such indices (we will show how i can be assigned to a sign pattern in the next section). Then,
EMBED Equation.DSMT4
where log(Fi) is to the base 2. For our example, with only two possible sign patterns, each with a 50% chance of occurring, the corresponding measure of entropy is 1. This measure is to be understood as follows: In general, an n x m pattern of signs, barring zeros, has n x m bits of information. A bit for each entry, with (say) a value of 0 for a negative entry and a value of 1 for a positive entry. The message eventually received is the outcome of estimating and revealing its sign pattern. The entropy of the frequency distribution can be used to measure the information content of the message, i.e., the amount of information that the frequency distribution does not provide that will be learned from the estimation of . For example, if all possible n x m sign patterns were members of RF(sgn ); and each was equally likely, then the entropy of the estimated sgn would be n x m (the maximum entropy). That is, a priori, the frequency distribution told us nothing about what to expect from the estimated sgn . Alternatively, if only one sign pattern had been possible (with frequency = 1), then the entropy of the estimated sgn is zero, i.e., we already know the answer before receiving the message. For our example, the information provided by estimating contains only one bit (as determined using (5) above), i.e., the estimation shows whether it is the all negative or all positive case. The remaining information, i.e., that all entries of have the same sign, is already provided by the constraints on the members of RF(sgn ) imposed by the hypothesis expressed by the sign patterns of {, }.
We would like to manipulate the measure of information to reflect the information content of the hypothesis represented by the model, rather than what is left to be determined by the estimation of . Accordingly, we propose that the information provided by the model be given by:
EMBED Equation.DSMT4
In our current example the hypothesis provides 75% of the four bits of information required to express the 2 x 2 sign pattern of = 1. For this example, the hypothesis is falsified if any of the other fourteen conceivable sign patterns for = 1 is exhibited by the data when is estimated.
The method is not limited only to expressions of the sign patterns of {, }. Any additional information about these entries provided by the model embedded in (3), or conjectured by a practitioner, can be used to determine what limits are placed on the members of RF(sgn ) and the corresponding frequencies of their occurrence. Further, less rather than more information may be processed. As before let n = 2, = I, and
0 < abs(ij) < 10.
But now, let,
EMBED Equation.DSMT4
In this case, the signs of are as before with the exception that, besides being nonzero, the sign of 12 is otherwise unknown. Assume for the example that the sign of 12 can be positive or negative, each 50% of the time. When 12 is positive, then the possibilities for the sign pattern of 1 are as before, all positive or all negative, each half of the time. When 12 is negative, then is qualitatively invertible and only one sign pattern for 1is possible. That is,
EMBED Equation.DSMT4
Now, RF(sgn ) contains three sign patterns: all positive and all negative, each 25% of the time; and, the above sign pattern 50% of the time. Now, the hypothesis forbids thirteen of the sixteen possible sign patterns for . If any of these are estimated, then the hypothesis is falsified. The entropy of this frequency distribution using (5) is 1.5 and the corresponding information content of the posited CQ(, ) is
INFO%(CQ(, )) = 62.5.
The entropy measure contains a fractional bit in the sense of an average, e.g., the average attendance last month to Tuesdays 11am lecture was 30.5 students.
It is easy to construct additional examples for larger systems, assuming for computational reasons, that they are kept sufficiently sparse in terms of the number of nonzero entries. For example take the case of an irreducible matrix, (with = I) with a negative maindiagonal and a single inference cycle involving all of the endogenous variables, e.g., with the only nonzero offdiagonal entries being those of the first lower subdiagonal and 1n. This matrical form will have only two terms in the expansion of its determinant, the product of the maindiagonal entries and the product of the offdiagonal entries. As before, assume that the absolute values of the nonzero terms are randomly chosen from the uniform distribution, 0 < abs(ij) < 10. For any value of n the adjoint of this matrical form is fully signed. Further, if the sign of the product of the offdiagonal entries is negative, then the sign of the determinant is always (1)n. If the product of these entries is positive, then, when is nonsingular, the determinant is positive or negative, each half the time. For any value of n, the corresponding RF(sgn ) has only two members, each appearing half the time for the value of the offdiagonal cycle positive. When the offdiagonal cycle is negative, there is just one member of RF(sgn ).
Nevertheless, the algorithmic principles that enable any specification of {, } to be worked through to a specification of the members of RF(sgn ) and their frequencies of occurrence can be problematical. For example, suppose n = m = 3, = I, 0 < abs(ij) < 10, and,
EMBED Equation.DSMT4
Of the 512 conceivable sign patterns for 1, barring zeros, only nine of them are possible for the inverse of (when nonsingular) with the above sign pattern. Looked at individually, each main diagonal cofactor, when nonzero, should be positive or negative half of the time. But, taken together, the signs of the maindiagonal cofactors are intercorrelated, since they share some entries of in common. Deriving the frequency distribution of the members of RF(sgn ) for this example or, for that matter, the members and frequencies of RF(sgn ) for any system is case specific and for sufficiently large systems problematic. We did not attempt to formulate a method to solve this problem. Instead, we developed the Monte Carlo approach described in the next section.
IV. A Monte Carlo Approach for Investigating the Characteristics of RF(sgn ). The Monte Carlo approach is to sample many times the possible outcomes for sgn for the sign patterns proposed by CQ(, ) consistent with (4). For the purpose of facilitating the development of the algorithm, we invoked assumptions that are not required for the analytic point of view we are proposing. Specifically, we assumed that was irreducible and that 1 and did not otherwise have zero entries. The reason for this was a simple practicality. We wanted to base the index system for sign patterns, as described below, on binary numbers, i.e., 0 for  and 1 for +. Were zeros allowed, the index system would have to be based on base three numbers. There is nothing wrong with this; however, the binary numbers are easier to work with and there are many applied systems that conform to these additional assumptions, including Kleins model, discussed in the next section. In any case, we do not mean in any way to limit the analysis in this way in its general application.
Our method is as follows:
(i) The sign patterns of {, } are specified, including nonzeros with uncertain signs.
(ii) For a single trial, CQ(, ) is sampled as the values of the nonzero entries chosen in the range 0 < ij, ij < 10. The sign pattern of the nonzero entries is then applied. Nonzeros with unknown signs are set positive or negative each half of the time.
(iii) If there is additional information about the entries in the two arrays, this is now imposed, e.g., the entries in accounting equations are often 1 or 1. As discussed below, for purposes of falsification, there are advantages to skipping this step.
(iv) Given the quantitative realizations of {, }, = 1 is computed.
(v) Given this result, the resulting sign patterns of s individual entries, rows, columns, and of itself are recorded. For each row or column of a particular , the sign pattern found is expressed as a binary number with 0 for negative and 1 for positive signs. The base 10 number corresponding to this binary number is computed and used as the index for the sign pattern found. For in its entirety, the binary number used is formed by writing out the sign pattern of all of its rows written endtoend as row vector. The base 10 number corresponding to this binary number is the index of the sign pattern found for all of .
(vi) For each sign pattern index observed, increment the corresponding frequency.
(vii) Stop if the preset number of samples has been reached, or return to (i).
To summarize the algorithm, for a single simulation, the number of samples (N) is usually set in the hundreds of thousands. As the simulation is under way, the sign patterns for each row and column of that appeared and their frequency of occurrence are tabulated. For sufficiently small systems (say, n x m < 26), the sign patterns of itself and their frequency of occurrence are also tabulated and the information and entropy measurements outlined above are computed.
For purposes of falsification, the sign pattern of a particular databased estimate of , denoted EMBED Equation.3 , is specified. If the signs of any of EMBED Equation.3 's rows or columns, or of EMBED Equation.3 itself, do not appear across sometimes millions of samples of CQ(, ), then the hypothesis being tested is, or at least appears to be, falsified. This, even though no individual sign of the reduced form failed to appear.
From the standpoint of falsification, we are reluctant to impose specific, quantitative values on any of the entries as identified in step (iii) above, even when these are known. When only processing the sign patterns of the arrays, then the simulation can draw any member of CQ(, ) subject to scaling. For example, take any whatsoever. Let MAX be the largest absolute value of any of its entries. Given this, form * by multiplying each entry of by, say, (1/MAX). The sign pattern of the inverse of * is the same as that of . And, even if cannot be sampled due to the distributional rules specified for the values of the entries of , * can be. As a result, the member of RF(sgn ) corresponding to will not (necessarily) be missed by our sampling procedure. Specifically, the members of RF(sgn ) with restrictions added in (iii) above will also be members of RF(sgn ) without the restrictions added. Accordingly, if a row, column, or entire sign pattern of does not appear without the restrictions in (iii) added, it would not appear with the restrictions added. The same conclusion holds for the posited sign pattern of , its rescaling, and the possible outcomes for the sign pattern of .
The risk of the method is to fail to find a member of RF(sgn ) that actually exists, but only with an extremely small frequency. Failing to find a member of RF(sgn ) could result in incorrectly estimating the entropy of the system and misjudging the information content of the proposed model. Also, failing to find a member of RF(sgn ) could result in incorrectly falsifying a model, which would be tantamount to a Type I error in classical statistics. There are three parts to the response to this issue. First, since the Monte Carlo method used is essentially a process by which the empirical probability distribution of sign patterns is built up, one must explore the statistical properties of this distribution estimator. Second, the estimator for the entropy of the model should have desirable statistical properties as well. Third, even if the empirical density has desirable statistical properties, what is known about the probability of the Monte Carlo method missing a possible outcome of RF(sgn )?
The possible sign patterns for a given row or column of , or even the entire matrix, is a multinomial distribution with unknown proportions. The Monte Carlo method used to generate the data on the proportions of sign patterns of is a maximum likelihood estimator. As a class, maximum likelihood estimators are known to be unbiased estimators for the first moment, the case here. They are also known to be efficient and consistent.
Regarding the second point, from equation (5) EMBED Equation.3 is an estimate of the entropy of a system calculated from a sample of size n, an event set of q outcomes of determined by the number of indices in Q, and Fi is the observed relative frequency of a member of the event set. The mean and variance of the entropy estimator (Basharin(1959)) are
EMBED Equation.3
EMBED Equation.3
The entropy estimator underestimates the actual entropy, but the bias always can be made smaller by choosing the sample size to be larger, and it vanishes in the limit. Also, since the sample size is in the denominator of the variance, the entropy estimator is statistically consistent.
The final issue is the probability of missing the ith sign pattern in the Monte Carlo experiment. Chebyshev's inequality, EMBED Equation.3 , can be used to put an upper bound on that probability. t is an arbitrarily chosen small number and EMBED Equation.3 is the variance of the ith parameter in the multinomial process that generates the data. The variance for the ith term is given by EMBED Equation.3 . Since the number of samples is in the denominator, the probability of missing a sign pattern can be made smaller by making n larger (see note 10)..
V. Falsifying Kleins Model I. Kleins model (Klein (1950)) is an overidentified, econometric model of the U.S. economy. It has been considered in the literature for a variety of methodological and pedagogical purposes. Maybee and Weiner (1988), and later Lady (2000), analyzed the qualitative properties of (for the model expressed in the form of (3) above). Generally, both of these efforts were intended to demonstrate how to cope with the fact that the sign pattern for as proposed for the model did not submit to a successful qualitative analysis. Instead, it was shown how to use additional, quantitative information in signing some of the entries of 1. None of this was focused on testing if the models specification survived an estimation of the reduced form as was done in Buck and Lady (2005).
Kleins model is given by,
EMBED Equation.DSMT4, where
EMBED Equation.DSMT4
In Kleins model the endogenous variables are private consumption (C), investment (I), the private wage bill (W1), income (Y), profits or nonwage income (P), the sum of private and government wages (W), and private product (E); and the exogenous variables are: the government wage bill (W2), lagged profits (P1), end of last period capital stock (K1), lagged private product (E1), years since 1931 (Year), taxes (TX), and government consumption (G).
The sign patterns of the arrays proposed by Klein are as follows,
EMBED Equation.DSMT4 and EMBED Equation.DSMT4 (7)
The array is not qualitatively invertible and no entry in 1 can be signed. Nevertheless, based on a variety of assumptions in Buck and Lady (2005), a small number of signs in the reduced form could be signed, a priori. Further, these signs appeared in the estimated reduced form (given below) based on Kleins original data set for the years 1921 1941. Accordingly, the model was not falsified by these findings.
EMBED Equation.DSMT4
When data from more recent years were combined with the original Klein data to provide a sample for the period 1921 2000 some of the estimated entries of changed sign, falsifying the structural hypothesis.(Buck and Lady (2005)). This was generally attributed to the transitory nature of economic relationships and, given the austerity and highly aggregated nature of Klein s model, it was proposed that there was a reasonable likelihood that some of the directions of influence between exogenous and endogenous variables would change over many decades of economic activity.
The issue of falsification of Kleins original specification using the above reduced form can be revisited in terms of the limitations on the members of RF(sgn ) imposed by the structural hypothesis presented by the sign patterns of {, } in (7). The sign pattern of the entire reduced form itself is expressed by 49 bits. This made the range of base 10 indices for the sign patterns that could be taken on by the entire reduced form matrix to be from 0 to (249 1). The upper values in this range were too large to represent as integers within the processing platform we were using for the Monte Carlo simulation. As a result, we did not tabulate the occurrence of entire sign patterns of the reduced form for each sample drawn. Instead, we tabulated the sign patterns for each row and each column of . Each of these is expressed by seven bits; so that, there are 128 possible sign patterns for each row and each column of .
Inspection of the specification of the model above shows that there are only 27 nonzeros in the arrays {, }. Of these, ten are estimated. The remaining are either 1 or 1 appropriate to accounting relationships among the model's endogenous and exogenous variables. We wanted to take this information into account; however, we were concerned about the scaling issue noted above, since we had no basis for setting the bounds on the other, estimated entries, given that some entries were pegged to be 1 or 1. To avoid this issue, the absolute value of 11 (which by definition is equal to 1) was chosen in the open interval 0 < 11 < 10 as were all the other non zero entries; and then, all of the other entries equal to 1 or 1 per the above specification, were set equal to 11 with the appropriate signs applied. Each time the simulation run was for 1,000,000 draws from CQ(, ) subject to the above rules (actually, many millions of trials were run while developing the simulator, all with the same results, as given below). Below are the results for one simulation for a sample of 1,000,000 draws for {, ). The results are presented first for the row sign patterns found for the reduced form (Table 1), then for the column sign patterns found (Table 2); and finally, a comparison of these findings to the nonrestricted reduced form estimation results, given in (8) above, is displayed in (Table 3).
Table 1. Reduced Form Sign Pattern Frequency Distributions By Row for Sign Patterns That Appeared At Least Once
RowNum Row# 1 Row# 2 Row# 3 Row# 4 Row# 5 Row# 6 Row# 7 Row Sign Patterns
0 0 .025181 0 0 0 0 0       
2 0 .040643 0 0 0 0 0      + 
14 0 .062238 0 0 0 0 0    + + + 
16 0 0 0 .002902 .100672 0 0   +    
18 .009805 .021949 .016365 .01371 .093798 .006635 .082676   +   + 
30 .084651 .123596 .252873 .063636 .218843 .082433 .186562   + + + + 
32 0 .075491 0 0 0 0 0  +     
33 .107076 .176939 .139029 .114448 .176939 .07908 .197563  +     +
34 0 .031206 0 0 0 0 0  +    + 
45 .034089 .009235 .095306 .02853 .009235 .033534 .036772  +  + +  +
46 0 .033009 0 0 0 0 0  +  + + + 
64 0 .012053 0 0 0 0 0 +      
66 0 .021837 0 0 0 0 0 +     + 
78 0 .014535 0 0 0 0 0 +   + + + 
80 .036831 0 0 .047804 .066372 0 0 +  +    
82 .040228 .021458 .012959 .04802 .057005 .022689 .02976 +  +   + 
94 .287898 .071775 .153387 .259512 .09113 .323827 .136586 +  + + + + 
96 0 .054319 0 0 0 0 0 + +     
97 .257444 .179439 .288041 .395943 .179439 .34799 .312828 + +     +
98 0 .01371 0 0 0 0 0 + +    + 
$%FGOP ) 2
>
D
E
ef

9
n
o
?@AQmȻȬȧȟyqyh1hv/f5h1h+z5h1h+zh1h/w45h1hgy5h1h/w4h$Sh2j5 h*G5 hbU5h,L5B*phh\ h,L5 h5 h2j5 hXb5 h,L5 h$S5h,Lh$S>* h,L>*h,Lh$Sh1h$S5h1h$S,%(7FGPo
p
q
r
@A01
$dha$gd/w4
$dha$gd2j
$dha$gd$S
$dha$gd/w4
$dha$gd$S^Bmuv6724)*UXefg'()9HI[rǿǿǿǿǿǿǿǿǨ h,L5jh1hG*0J5Uh1h}5h1hv/f5h1hG*5h1hf5h1haM5h1h#5h1h+z5h1h r5h1hs5@./0gufq^@x>?TUbkpr~wjkFE
h1h/w45UVjh1h/w45Uh1h/w456]h1h/w45H*h1h?25h1hF5h1h=5h1h/w45h1h/w4h1h+zh1h r5h1h}5h1h+z5h1hv/f5h1haM5.mn~""**f.g.66
$dha$gdb
$dha$gd/w4
$dha$gd/w4Vt.02R 4 """"R%^%b%%&&d&p't'P(("),)****ڼڪڢښzpzfzzzph1h#gJ56h1h#gJ5H*h1h#gJ5h1h}5h1h+z5h1h=5h1h5h1hpO5h1h/w45H*h1hwD^5h1haM5 h25h1h/w456h1h/w4h1h/w45jh1h/w45Ujh1h/w45EHH*U(*&+:+,,.!.2.6.D.c.e.f.g......#/2/4/>/?/h/p/00
11"4#44444xi_h1hb56jh1h#B0J5Uh1hb5H*h1h#B5hKu5B*phhnghng5 hng5h1hpO5h1hr5h1hb5h1h5h1hh1hKu5 h5h1hf5h1hbb5h1h#gJ5H*h1hF5h1h#gJ5%466C9t9y9999<:?:G:J:T:d:e::::: ;<<<>>????(@@@AAB$BjBBBB&C(C*C,C.CXC詸}h1hW5 hW5h1hKu5h1hp5>*hKuhKu5 hKu5h1h#Bh1hY5h1hp5h1h[5h1hF5h1hf5h1hw5h1hr5h1h#B5h1h/w4h1hb5.6<<(C*CBHDHJJDKKhLjLhOjOOOP PPP
$dha$gdw
$dha$gdw
$dha$gd
$dha$gdW
$dha$gdKu
$dha$gd/w4
$dha$gd#BXCpCrCCCCCDDD"EEEFFF!F*F+FpFF}F~FIGNGPG\G]GGGG@HDH^HHHIIII>IXIIIIIJJJJJFKHKKKdLh1hw5 h!5h1h5H*h1h5hR+5B*phhnghR+5 hR+5h1hp5h1hW56 hW5h1hW5hW5B*phhnghW57dLfL(M*M,MJMLMlMMMMMMMMMMMZN^NlNNNNNNNNOOOO6O]@]x]PiRikkkknnfqhqqqq
$dha$gdw\P
$dha$gdw\P
$dha$gdeY
$dha$gdeY
$dha$gdTx
$dha$gdw
$dha$gdwTTTVVZWfWWW"X*X6X8XXXY~YYYY8ZZZ[[@]B]p]r]t]v]]]!^"^^^^^c_`~`qٯjh1h05EHUjZKM
h1h05UVjh1heY5Uh1heY5H*h1h]5h1heY5h1hTx5H*h1hTx5h1hlN5h1h25 hSq5h1hw5H*jh1heY0J5Uh1hw5)~````NaPaRaaaRbTbZbbbccccccccfgg>g@gNgghhBhHhXhhLiPiRikkkkkkkkkklײַj<h1h}5EHUjTM
h1h}5UVjh1heY5Uh1hw\P5hKheYh1hlN5h1heY56h1h=5h1h]5h1h05>*h1hTx5h1heY5h1h050lllllXmhmjmmmmmnn0oZoooopppzq~qqqqqqqrrrs>sBsss6t:ttttuxuξξΪΝ~ΪyΪΪoΪoh1hw\P5H* h.5jh1hw\P5EHUjtAM
h1hw\P5UVjh1hw\P5Uh1hw\P5H*h(8*5B*phhngh(8*5h1hTx5h1hw\P5h1heY5H* h(8*5h1heY5h1hu^5H*h1hu^5+qqqquuuu*y,y.y^y`yPzQzjlƆȆڌdhgd(8*
$dha$gdLR
$dha$gdLR
$dha$gdTx
$dha$gdw\P
$dha$gdw\Pxuuuuuuuuzxxx*yzzzN}Z}}}}ˀ̀băRThjl(6\nﶮxxphh`h1hTk5h1h85h1h5h1hX5h1hs5h1hs5H*h1hs5H*h1hs5h1hw\P5H*h1h(8*5 h(8*5h1hTx5j'h1h(5EHUjUNM
h1h(5UVjh1hw\P5Uh1hw\P5h1h(5%nrĆƆȆԆކ $2fl҇
tz&ɺٵwogobZbZbZh1h(8*5 h(8*5hnghR+5h1hLR5jh1hTk0J5Uh1hD5 h'yJ5h1h,A5h1h85H*h1h85h1h.5 hTk5jch1h'yJ5EHUj6^M
h1h'yJ5UVjh1hTk5Uh1hs5h1hTk5h1hTk5H*#&,:^:<>ڌ܌zTX:V~[\~~th1h)0J5h1h:5H* hd5 hO1\5h1h:5h^ ;h:5B*phh1h)5H*hngh^ ;5h1h)5h1h$Z5h1ho5h1hohO1\5B*phhnghO1\5hngh(8*5h1h(8*5 h(8*5'ڌ܌^_bchi
$dha$gdv
$dha$gdO1\
$dha$gd:
$dha$gdodhgd+q\]`cdlpZ\ڙj_ghiƠ *@ġʡnz(2F鳩靕{{{ hv5h1hv5h1h
5 h.5h1hlu5jhO1\0J5Uh^ ;5B*phhngh^ ;5h1h35hnghO1\5h1hO1\h1hO1\5h1h:5H* hO1\5h1h:5jh1h)0J5U.FHnprtv£ģ$&LNPRȤФx/?hjxЫ
ǎ~~~vvv~h1hz}5h1h35hnghv5h1h:5j8 h#hv5EHUj'h#hv5EHU$jnM
hv5CJPJUVaJh1hv5jh#hv5EHU$jnM
hv5CJPJUVaJ hv5jhv5U. ,.ʸ˸rtvػ
QRS
$dha$gd=
$dha$gd
$dha$gdo
$dha$gd_J
$dha$gdz}
$dha$gd3*4&ʸ˸
LMS]bdFfnpv굦꜐}sejh!0J5PJUh}h35PJh1h35H*PJ
h!5PJhv5B*PJphh1h35PJjI"h.h85EHU*jpM
h1h85CJPJUVaJjh1h35U hv5 h^ ;5h1h_J5h1h35h1h:h1h3!vxлһԻֻ29ELtuvwAC窙珈~tfZh1h35H*PJjh{0J5PJUh1hv5PJhvhv5PJ
hv5PJh1h_J5PJ j)h1h35EHPJU*jiZM
h1h35CJPJUVaJ j&h1h35EHPJU*jiZM
h1h35CJPJUVaJh1h35PJjh1h35PJUؽٽڽ۽689BϾ־)6p_UKh1h_J5PJhngh{5PJ ji2h1h35EHPJU*jZM
h1h35CJPJUVaJh1h35H*PJ jJ0h1h35EHPJU*j+ZM
h1h35CJPJUVaJ
h{5PJh1h35PJjh1h35PJU j}h1h35EHPJU*jZM
h1h35CJPJUVaJ67>?IKMNOPQRSr.6`b[^¸rh]Oj[FE
h1h=UVjh1h=Uh1h=5H*jh1h=0J5U h=5jh=0J5Uh1h=5h1h= h5h1ho5h1h35PJhngh35PJ jhngh{0J5PJUhngh5PJhngh{5PJ
h{5PJh1h_J5PJ
h5PJPQ>@\^ 'z~ʽ߳߳߳߳߳إ؊}soeh1h=5H*h=jh=0JUjj@h1h=EHUjpM
h1h=UVjK=h1h=EHUjYFE
h1h=UVh1h=5H*j6h1h=EHUjZFE
h1h=UVh1h=h1h=5jh1h=Uj4h1h=EHU&()}gd=$
dha$gd=
$dha$gd=
$dha$gd=`d
HZ^`
P$(r,h=5H*h1h=5H* h5jh=0J5Uh1h=5H* h=5jh=0JUjDh1h=EHUj~)rM
h1h=UVh1h=jh1h=Uh1h=5h1h=564YZm~07777>TBTDTVThTBUFUUUUUUUpWWZ]]bbbbeeeff2g4g6g>g0, therefore it must be the case that [1]23 and [1]53 have the same sign since one is just a multiple of the other, and hence 24 and 54 also have the same sign. Indeed, (9a) shows many instances when certain sets of entries in 1 must have the same sign. Moreover, apart from the sign of the determinant (which is contingent on the magnitudes of the unknown a's, b's and c's) there are many instances of same sign restrictions in (equation 10) for which the Monte Carlo provided ample evidence and which restrictions are not imposed at the time of estimating the reduced form in standard practice. In more complex systems this sort of post hoc analytic examination would be extremely difficult, if not impossible, but for which the Monte Carlo method outlined here provides evidence.
EMBED Equation.3 (9a)
EMBED Equation.3 (9b)
EMBED Equation.3
(10)
An important question is that of how the proposed falsification procedure based on Monte Carlo simulation of model sign patterns brings more to the analytic table than the classical econometric approach. Consider the pedagogy found in Goldberger(1964) and which is propagated and expanded upon in Berndt(1990) and Greene(2008).
In the empirical results reported by Goldberger(1964, pps. 325 and 368) the difference between the unconstrained reduced form estimates and the reduced form derived from the constrained ML estimates of the structural model differ in sign in eight out of forty nine instances, all involving only three of the seven exogenous variables. There are differences in magnitude (i.e. one coefficient estimate is two or more times as large as the other) for seventeen out of forty one cases where the unrestricted and constrained coefficients have the same sign, all involving only four of the seven exogenous variables. Noting "there are substantial differences in parameter estimates based on the unrestricted and restricted reduced form estimation," Berndt(1991) takes these differences as warranting a statistical test of the zero restrictions in the matrix of the structural model. In a joint test of the restrictions Berndt rejects the hypothesis, but does not argue that the model has been falsified.
Reasoning from the multiplicity of estimates (contingent on the estimator used) of the structural parameters, Greene (2008) comes to the same conclusion that a test of the zero restrictions is necessary. He finds that the zero restrictions are rejected only for the third or wage equation of the model and subsequently argues that that equation may be misspecified. Like Berndt, he does not argue that the model has been falsified.
Having found a problem with the zero restrictions, neither author notes that there is only one restrictedunrestricted reduced form pair that are statistically different from one another. This begs the question of whether the conduct of the test of hypothesis regarding the zero restrictions was motivated by their pedagogical interest or was a prescriptive test based on the observed empirical results. In any case, their approach leaves open the question of whether the model has been falsified. Comparatively, the results we present here are decisive: The sign pattern of the estimated reduced form is impossible, given the hypothesized sign patterns for {, }.VI. Conclusions and Pending Issues. The qualitative analysis that we have proposed here enables virtually any structural hypothesis expressed at the level of sign patterns, or even in some cases only identifying zero and nonzero entries of the structural arrays, to be assessed for its information content; and, ultimately, whether or not it is consistent with the data. Compared to a traditional qualitative analysis, the expansion of applicability is vast. We believe that such an expanded qualitative analysis has the potential to substantially change the assessments made of actual models; and, hopefully, result in an increased quality, or at least a better understanding of the quality, of applied systems.
Notwithstanding the promise of our proposed methods, there are substantial theoretical and practical issues that remain to be confronted and resolved. In terms of the theory, the central issue to resolve is whether or not any fully specified sign patterns for {, } will result in some kind of limitation on the members of RF(sgn ). Our experience in developing examples and working with the Monte Carlo simulation suggests that there will always be such limits. If so, this would mean from the standpoint of falsification and information content, the expanded qualitative analysis that we have proposed will have universal applicability.
There are also substantial practical problems that we have not approached. Investigating the relationship between the signs of {, } and the corresponding nature of RF(sgn ) could be expressed exactly by deriving the distributional rules for the values of = 1 from the distributions for the entries of {, }. Further, the distributional rules for (, ) may be expressed in a substantial variety, compared to our use of uniform distributions; and further, might themselves have interesting and important bases for their nature, based upon analysis of the appropriate data. Finally, there may be more efficient, or faster, or otherwise better computational routines to use along the lines of the Monte Carlo approach utilized here. And, in connection with this, there is a potential severity of difficulty in applying these ideas to very large systems that suggests a need for computational innovation.
It is our hope that a review of the ideas for an expanded qualitative analysis that we have provided here will excite an interest in approaching and resolving these, and other related issues, in the general enterprise of assuring the quality of applied systems.
References
Basharin, G.P. (1959), On a statistical Estimate for the entropy of a sequence of independent random variables, Theory of Probability and its Applications, Vol 3, No 4, Pps. 333336.
Bassett, Lowell, John Maybee, and James Quirk, 1968, Qualitative Economics and the Scope of the Correspondence Principal, Econometrica, 36, pp. 544563.
Berndt, Ernest, 1990, The Practice of Econometrics, Classic and Contemporary, New York, NY: Addison Wesley.
Buck, Andrew and George Lady, 2005, "Falsifying Economics Models," Economic Modelling, 22, September 2005, pp. 777810.
Cover, Thomas and Joy Thomas, 1991, Elements of Information Theory, Wiley Series in Telecommunications, New York.
Goldberger, Arthur S., 1964, Econometric Theory, New York: John Wiley and Sons.
Greene, William, 2008, Econometric Analysis, Upper Saddle River, New Jersey: Prentice Hall.
Hale, Douglas and George Lady, 1995, "Qualitative Comparative Statics and Audits of Model Performance" , Linear Algebra and Its Applications, 217, 1995, pp. 141154.
Hale, Douglas, George Lady, John Maybee, and James Quirk, 1999, Nonparametric Comparative Statics and Stability, Princeton, New Jersey: Princeton University Press.
Klein, Lawrence, 1950, Economic Fluctuations in the U.S., 19211941, Cowles Commission for Research in Economics Monograph No. 11, New York, NY, John Wiley and Sons.
Lady, George, 1983, The Structure of Qualitatively Determinate Relationships, Econometrica, 51, pp. 197218.
______________, 1993, SGNSOLVE.EXE Analysis Package, prepared as a job of work for the Energy Information Administration, U.S. Department of Energy, Washington, D.C.
Lady, George and John Maybee, 1983, "Qualitatively Invertible Matrices," Mathematical Social Sciences, Vol. 6, 1983, pp. 397  407.
Lady, George and Marc Sobel, 2006, Qualitative Inverses and Simulation, Memeo, Temple University.
Lancaster, Kelvin, 1962, The Scope of Qualitative Economics, Review of Economic Studies, 29, 99132.
________________, 1966, The Solution of Qualitative Comparative Statics Problems, Quarterly Journal of Economics, 53, pp. 278295.
Maybee, John and Gerry Weiner, 1988, From Qualitative Matrices to Quantitative Restrictions, Linear and Multilinear Algebra, 22, 229248.
Maybee, Stuart, 1986, A Method for Identifying Sign Solvable Systems, M.S. theses, University of Colorado.
Pierce, John, 1980, An Introduction to Information Theory: Symbols, Signals, and Noise, 2nd Edition, Dover Publications, Inc., New York.
Popper, Karl , [1934] 1959, The Logic of Scientific Discovery, reprint, New York: Harper and Row.
Ritschard, Gilbert, 1983, Computable Qualitative Comparative Statics Techniques, Econometrica, 51, 11451168.
Samuelson, Paul, 1947, Foundations of Economic Analysis, Cambridge: Harvard University Press.
Shannon, C.E., 1948, A Mathematical Theory of Communication, Bell System s Technical Journal, 27, 379423, 623656.
Shannon (1948) shows how to use entropy as a measure of information. For an introduction see Pierce (1980) and Cover and Thomas (1991).
Actually, Lancaster (1962) and Lady (1983) were studying a slightly different qualitative problem: the conditions under which the sign pattern of Y could be determined based upon the sign pattern of 1 and Z, i.e., the conditions under which at least one entire column of 1 could be determined based upon sgn . Lady and Maybee (1983) showed that sometimes, although some entries of 1 could be signed, nevertheless no entire column could be signed.
The distributional rule is in the nature of a Bayesian prior. The only stipulation is that the prior not admit values for the matrix entries that violate the proposed sign pattern. To facilitate the examples presented here a uniform distribution is assumed. It is in no way intended to limit the analysis to the assumption of uniform distributions.
Strictly, sgn a = 1, 1, or 0 as a > 0, a < 0, or a = 0. We will use the symbols +, , 0 instead to facilitate the presentation.
Of course for this simple 2 x 2 case the fact that s adjoint is known would presumably be taken into account; and, similarly for larger systems, e.g., Buck and Lady (2005). Still, in general, if no entry of can be signed, a qualitative analysis has failed and is abandoned unless other, quantitative information is added.
For this sign pattern all of the offdiagonal cofactors are positive. When nonzero, all of the ondiagonal cofactors can be positive or negative, each half of the time. If any ondiagonal cofactor is negative, then the determinant is positive. If all ondiagonal cofactors are positive, then, when nonsingular, the determinant can be positive or negative; but, most of the time positive, since five of the six terms in the expansion of the determinant are positive.
A similar method was reported on in Lady and Sobel (2006). In that application only a tabulation of signs of the entries of 1 was recorded.
In principle the system can be of any size. We were limited by the size and speed of our computer.
The final term in each of Basharin's expansions is interpreted as 'big O." That is, the sum of the remaining terms in the expansion is no greater than the inverse of the square of the sample size. In this application the sample size is the number of iterations of the Monte Carlo.
Or at least it gets sufficiently small in principle. For larger systems, the required sample size may be large compared to the available computing capability.
For large systems, notwithstanding these observations, the magnitudes of the number of possible sign patterns for (even barring zeros), the possible degree to which some sign patterns might be unlikely, and a correspondingly sensible size for the number of samples taken are all issues that would benefit from future innovation.
A model is overidentified if the estimable reduced form provides too many linearly independent equations in the unknowns of the structural model.
Klein's model was the basis for much of the macroeconomic policy modeling spawned by the Cowles Foundation. Goldberger (1964), Berndt (1991) and Greene (2000) all used it pedagogically to demonstrate alternative econometric approaches for dealing with the identification problem at the time of estimation.
All of the unknown entries were hypothesized to be positive except b3, which was hypothesized to be negative. As a result, the hypothesized 23 > 0.
This is the unrestricted reduced form estimated by Goldberger (1964).
We did conduct a simple search by sampling {, }, computing = 1, and then checking to see if the sign pattern found conformed to that given above, all without keeping track as to what sign patterns were otherwise found. The sign pattern given in (8) was never found. As given below, we checked the algebra for = 1 and found out why.
PAGE \* MERGEFORMAT 25
PAGE \* MERGEFORMAT 29
789:;<=>?ABC&D2E>FJGVHbInJzKLMNOPQzR\S>Tgd=>T@TBTDTZ[eeeeeeeee6g8g:ggggggggh$$Ifa$gd$a$gd=
$dha$gd=hhhhhh$h(h,h0h4h8hiBiFiJiViXibiniziiiFf`Ff]$$Ifa$gdFfcZiiiiiiiiiJnLnFpHpppqq~ssss~vvdhgd;m
$dha$gd=dhgd=
$dha$gd=Ffc$$Ifa$gdppppprrrrrr6s8sssssssssssssssssssssttt:tuuuuuuzvyyyyyyyyyzzzzzzzzzhh=5H*hh=5hngh/5hngh;m5 h;m5 h/5h=5H*hLh=5h
h=5 h=5h=5H*Fzzh{l{
2468RTz~H˼}gT}OJ h;m5 h5$jph=h5CJEHUaJ*j0M
h=h=5CJPJUVaJh=h=5CJaJ jh=h=5CJUaJjmh+h=5EHU$j
yM
h=5CJPJUVaJjehHVh=5EHU$jxM
h=5CJPJUVaJjh=5UhD6h=5H* h=5hh=5
HJLNPR"$`b,.
$dha$gdH,dhgd;m$dha$gd=o_Ԇ
$dha$gd=
$dha$gd=HJĒL.*,6FJ\^hlf,Tº²²vnf[fSh1hkjn5h1h56]h1h5h1h85hVh856PJh1h85PJh1h/w4h1h5h1hm"5 h5h1h15H*h1h15h1hw\P5h1hA5h1hQum5h1hQumh;mh=5B*phhngh/5hngh;m5h;m5B*ph.68\^jl,.ܬެ~$a$gd+gdm"$a$gdkjn$a$gd$a$gdG*gd8
$dha$gd
$dha$gddhgdA
$dha$gd1
$dha$gdm"TVZȪ@ڬܬ<֭8
Pd̰Xز(ЀvЀqih5h55 h55h1hkjn56h1h56]h1h6]h1hh1h?v56]h1h?v5h1h+5hgh+56 h5h1h56h1h5h1hm"6h1hm"h1hkjn56]h1hkjn5 h+5%8̰ΰHJLTV "$a$gdG*$a$gd5$a$gdgd$a$gd?v(fJTضڷH@λt"jln.þөþ~өvlvldh1hm"5h1h/w456h1h/w45h1hG*56]h1hG*5H*h1hG*56h1hG*5h5h1h56h5h556] h55 h5h1h56]h1h5h1h56]h1h5h5h55h5h556'̻λlnZ\^rtr"Hdgd)$a$gdLRgd
$da$gd#BgdG*
$dha$gd/w4$a$gd$a$gd/w4$a$gdG*.Z\^`bprtvx:<
&.Z\
HJprƽƯƯƽƯƧߟڟڟڟڏڇڇheYhp5h
hp5hvhp5hnghp5hhp5h#Bhp5h#Bhp5CJH*aJhp5CJaJh#Bhp5CJaJhR@hp5 hp5hpjhp0JU hff5 hG*5h1hG*51rtv "$& "$*FHJLNPRTV,024FZ\^`ʶ֦֞֞օyqlq hng5h=hp5h{hp5CJaJh{hp5CJPJaJh{hpB*phhnghp5hO1\hp5hxhp5hp5CJH*aJhp5CJaJhxhp5CJaJhpjhp0JU hp5hTkhp5jhTkhp0J5U(HJR.0Z\@BFHLNRTXZgdG*$a$gd=gd=$a$gd{gdO1\gd)`RT
>@BDHJNPTVZ\öhngmHnHujhpUjhcdUhcdhphp5h
hp5H*h
hp5hphp5B*phhp5B*phhnghp5h
hp5hp5H*jhp0JUhp hp5h~hp52Z
$dha$gd/w4$a$ hff521h:p/ =!"#$%51h0:p= /!"#$%51h0:p= /!"#$%21h:p/ =!"#$%? 01h:p/ =!"#$%C{f?1h0:p= /!"#$%C{f51h0:p= /!"#$%21h:p,A/ =!"#$%21h:p,A/ =!"#$%rDd
\
c$A??#"`2xTuiDJ`!xTuin F)(+dxڵV_HSQιwV۴45H3DO
ZHNz͚s8kΠD1"boaJ"dRz2bR91mw;w=6xC"!BQ<@QP`&d60(xOb0bD~P=eQ'/>ɶ::iFZbU"F/
%,7BIx4l2͒{Npd5[D/b?ǋN;4+,u6nζMx@_*l
~%ː35WTS̄
MdNR{%]SX8N#/HxůqTayKh}mk{,MXZL
[0
2p@#E%3rk+52_Ap*z;>4,juʖQWahu{iY_/]ju\7WyZ>XzUE?jᬮfg??2n^
1Do wC 0IU}Ss,ԃRg=7/ >ktCޓ_#2Xɷ+ǃ~'n`!TʁVǐiX/R:1`9l0^Y1t%1?TVl T;!WhsC\)49i1s"RcԍyHHRaCѼ&
.#GGKVI~
ȇφ 4uKwu8E%X°xzx>s Dd
b
c$A??3"`?2dbC
gwt^J`!dbC
gwt^`
yxuSkQvI&i=kAGA[Ml+P%M.$Ĝ,ArOE҃GѫHO_Ş"{&Q7_yofL5АpqF%zA#ngvqJ3p#X# dخi w8;BBy[ʆVnkEnD`B{+ޞk#Q$8VL>\0xɽh#שUh_Ď`gFXf3ϒ&0^?1LSG5>FyƖIXǃdñlNZ~7_^GdKCdjaeiReTL%W6tǔur`7
SVWQwl[I%~iMj!K~jI:!yf;)Y'\p+BFG?,֜˹5T!T ü1+aBci1k![_ KB%L"*AEC"
QA"MnM >ncAQτ{{v(:VKM*3Z(RN,O=@OGd%xJ'oh=qut
3w6=qWɫWt\m 3o(l膚v'3Ѽdn)hBɃE,XޓSYa^!+NIwa6FgP?arC8רDP3<㑹FU\9\rG_na&Fxme4,4m[y)pE[y%%Puu{8
mjw UsPjُNcTL ^_+I2Aؕ ǭ1oTvžn!0V͉S47jцN{/()XJlMi矬pBcs:OLDd
Tlb
c$A
??3"`?2Z]g &'rJ`!jZ]g &' "048xUkSAݗ6KjJi4m=/5/BgH^BT"$CTă_QBT"9qf_6
%fgg7$sF"Iu!9ن lF^ý]0:+(^(n*#P1DG>qLR/JΓE\U~?_8`{(vP"`NEUc'B%(YxKlGla^4'cVqdFĄmMgX`v*CTxTfGl}kp;j>˰Ix,b8krGϹ5qB>@i![t/a?d }cdsֲOe*Grj=}1`1NBa߰Z:W+JHf:5;`<[c8iceHtܕ>Q(D\c E)sAxk ՕLFs;}CYƕ/YjˬwoeJn!u{;:ƽ%ϸk&®oP3歍Jad6
wq55LF9bK&ṋ/Fl n㹴3˺\Bn
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{}~
/! #"%$&'()*+,0.1T23457698;:<>=?@BACROEFGHIJKLMNPQUS[lVWXYZ\b]^_`adckefghijmsnopqrtuwvxyz{}~Root Entry~ FW@Data
{WordDocument}ObjectPool2WW_1166427755F2W2WOle
CompObjiObjInfo
!"#$%&'(*+,./012369:;<=@CDEFGHILOPQRSTUVY\]^_`cfghijklorstuvy~
FMathType 5.0 EquationMathType EFEquation.DSMT49qA 5.@Times New Roman@Times New Roman OlePres000 * Equation Native )_1296042214PF2W2WOle
44,.
@Times New Roman. VTimes New Roman
2
V11tTimes New Roman2
0,1,2,...,.(2)1(&0UE$2PTimes New Roman
2
PiitV5Times New Roman
2
V5nmsTimes New Roman
2
jks$Times New Roman
2
$jks$Times New Roman
2
$jkt}Times New Roman
2
}ff(Times New Roman2
(dydzin22
Times New Roman
2
yz:Symbol
2
:==t}Symbol
2
}2
+==
2
jSymbol
2
,"System'"SystemHI0`G0Z9'''
2
4 "System`G0o{DSMT5WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
p"fi
"yj
dyj
++j==1n
"
p"fi
"zk
dzk
==0,i==1,2,...,n.k==1m
"
(2)
FMathType 6.0 EquationMathType EFEquation.DSMT49qCompObj
5iObjInfo7Equation Native 8P_1296805978
#F2W2W4tPRTDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
sgnb=="++++"[].
FMathType 6.0 EquationMathTyOle
>CompObj?iObjInfoAEquation Native Bpe EFEquation.DSMT49qPRDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
Entropy(CQ(b,g))=="PFi
log(Fi
),i"Q
"
(5)
FMathType 6.0 EquationMathType EFEquation.DSMT49qPRDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/AP_1297413399<F2W2WOle
JCompObjKiObjInfoMEquation Native N_1296110196F2W2WOle
WCompObjXiG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
INFO%(CQ(b,g))==100(1"Entropy(CQ(b,g))nm).(6)
FMathType 6.0 EquationMathType EFEquation.DSMT49q3PRDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
sgnb=="?++"ObjInfoZEquation Native [O_1296979419 F2W2WOle
a[].
FMathType 6.0 EquationMathType EFEquation.DSMT49qºPRDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_ECompObj!biObjInfo"dEquation Native e_1298020055n%F2W2W_A
ifb12
<<0,thensgnb"1
=="++""[].
FMathType 6.0 EquationMathType EFEquation.DSMT49qOle
mCompObj$&niObjInfo'pEquation Native qx\PRDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
sgnb=="++++++"++++++"[]._1305112236s*F2W2WOle
wCompObj)+xfObjInfo,z
FMicrosoft Equation 3.0DS EquationEquation.39qP,
"
FMicrosoft Equation 3.0DS EquationEquation.39qEquation Native {_1305112266/F2W2WOle
CompObj.0}fObjInfo1Equation Native _12992348224F2W2WOle
P,
"
FMathType 6.0 EquationMathType EFEquation.DSMT49qTPRDDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_ECompObj35iObjInfo6Equation Native _12977709479F2W2W_A
Entropy(CQ(b,g))=="PFi
logFi
()i"Q
"
FMicrosoft Equation 3.0DS EquationEquation.39qOle
CompObj8:fObjInfo;Equation Native (s
E(2E ntropy(" ))=Entropy"q"1nlog(e)+)1n2
()
FMicrosoft Equation 3.0DS EquationEquation.39q_12977709737F>F2W2WOle
CompObj=?fObjInfo@Equation Native p_1297784566CF2W2WOle
CompObjBDfTp4
Var(2E ntropy(" ))=1nFii"Q
"
logFi
()()2
"Entropy(" )2
()+1n2
()
FMicrosoft Equation 3.0DS EquationEquation.39q e
PrFi
"EFi
>t()d"i2
t2
FMicrosoft Equation 3.0DS EquationEquation.39qObjInfoEEquation Native _1297784619AKHF2W2WOle
CompObjGIfObjInfoJEquation Native ;_1297784730MF2W2W`oy
i2
FMicrosoft Equation 3.0DS EquationEquation.39qJe
Fi
(1"Fi
)nOle
CompObjLNfObjInfoOEquation Native f_1166427739VRF2W2WOle
CompObjQTiObjInfo
FMathType 5.0 EquationDS EquationEquation.DSMT49q4 k."SystemknY9Xy @Times New RomankOlePres000SUEquation Native 9_1166427738\XF2WУWOle
,.
@Times New Roman .Times New Roman 2
[!Z7 2
[IY9Symbol 2
[g) 2
[b9Symbol 2
[=7"System'"System@{HI0XyZ9'''
2
[ok $
Y=Z
FMathType 5.0 EquationMathTyCompObjWZiObjInfoOlePres000Y[D4Equation Native zpe EFEquation.DSMT49q^DSMT5WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
"10a1
0a2
000"100b9]
."System
nY9 @Times New Roman
,.
@Times New Roman. Times New Roman
2
1213sgTimes New Roman2
g123sTimes New Roman
2
1123tfrTimes New Roman2
fr
1000000000}~RTimes New Roman2
R01000000000~RTimes New Roman2
R00100000000(QTimes New Roman2
(Q11010000000011RTimes New Roman2
R00011100000000TRTimes New Roman2
TR00100101000000RTimes New Roman2
R00010011000010fTimes New Roman2
faaCaaxMP@Times New Roman
2
@bIbbX%Times New Roman
2
cWcc(Times New Roman 2
(YTimes New Roman 2
PTwTimes New Roman 2
TwWTimes New Roman 2
Ef;Symbol2
f;bm2
m82
82
2
82
82
2
8&# 2
&#=(w
2
(wo M2
M82
86
2
62
8T
2
Tl2
l82
8
2
t2
8
Times New Roman 2
2t!
Times New Roman 2
!
1t.
Times New Roman 2
.
1tA)
Times New Roman 2
A)
1sfTimes New Roman 2
fWTimes New Roman 2
PTimes New Roman 2
K(Times New Roman 2
(ETimes New Roman
2
Year5+2TTimes New Roman
2
TTX9Times New Roman 2
G
Symbol 2
t
2
sA
2
A
tmSymbol
2
m
2

2

2
2
M
2
M
2
2
l
2
l
2
2
"System'"Systemxh;*Z9'''
2
&
1
0000"1000c1
110"10000001"1"1000100"10000100"1[]CIW1
YPWE[]=="a1
"a3
000000"b2
"b3
0000000"c2
"c3
00000001"10000000"100000010000"10[]W2
P"1
K"1
E"1
YearTXG[]
FMathType 5.0 EquationDS EquationEquation.DSMT49q_1166427737^FУWУWOle
CompObj]`iObjInfoOlePres000_aEquation Native }_12992349982hdFУWУWOle
!"#$%&'()*+,./012345679:;<=>?@ABEHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnpqrstuvwxyz{(]~ ?w."SystemwnY9`G0 @Times New Romanx,.
@Times New Roman .Symbol 2
E& 2
wE& 2
E& 2
E& 2
TE& 2
E& 2
E& 2
1E& 2
E& 2
E& 2
nE& 2
P& 2
wP& 2
P& 2
P& 2
TP& 2
P& 2
P& 2
1P& 2
P& 2
P& 2
nP& 2
7 2
C+7 2
Ss7 2
S+7 2
s7 2
6 2
C+7 2
'C7 2
'+7 2
'z+7 2
+7 2
7 2
+7 2
7 2
e+7 2
e+7 2
ez7 2
&=7 2
&b9Times New Roman 2
v02 2
02 2
02 2
02 2
}02 2
S
02 2
S02 2
SF02 2
S02 2
S}02 2
02 2
02 2
02 2
}02 2
'
02 2
'v02 2
'02 2
'02 2
v02 2
02 2
F02 2
02 2
}02 2
02 2
v02 2
F02 2
02 2
}02 2
e
02 2
ev02 2
eF02 2
e022
&sgn'12"System'"SystemHI0`G0Z9'''
2
&x a}{
sgn="0+0+000"00+0000"000+++0"000000+""000+00"0000+00"[]
FMathType 6.0 EquationMathType EFEquation.DSMT49q]~ ?q."SystemqnY99* CompObjcfiObjInfoOlePres000egEquation Native 8@Times New Romanr,.
@Times New Roman .Symbol 2
?& 2
w?& 2
?& 2
?& 2
T?& 2
?& 2
?& 2
1?& 2
?& 2
?& 2
n?& 2
N& 2
wN& 2
N& 2
N& 2
TN& 2
N& 2
N& 2
1N& 2
N& 2
N& 2
nN& 2
n7 2
w+7 2
Sx7 2
'7 2
'm+7 2
7 2
?7 2
+7 2
7 2
e7 2
ex7 2
&=7 2
&g)Times New Roman 2
02 2
02 2
B02 2
02 2
02 2
S02 2
Sp02 2
S02 2
SB02 2
S02 2
S02 2
02 2
p02 2
02 2
B02 2
02 2
02 2
z02 2
'02 2
'B02 2
'02 2
'02 2
'z02 2
02 2
p02 2
02 2
02 2
z02 2
02 2
p02 2
02 2
B02 2
z02 2
e02 2
ep02 2
e02 2
eB02 2
e022
&sgn'12"System'"Systemh;*@{9*Z9'''
2
&xr LPRDDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A
sgng==""000000"++0000000""0000000++"0000000"000000++0000"0[].
FMathType 6.0 EquationMathType EFEquation.DSMT49q% _1299327358jFУWУWOle
CCompObjilDiObjInfoFOlePres000kmG Equation Native oX_1301379314b(pFУWУWOle
}*."SystemnY9 @Times New Roman+,.
@Times New Roman. H6Times New Roman
2
H6**utTimes New Roman2
t*****+Tt
Times New Roman2
*****+Tt7Times New Roman2
7*****+TtTimes New Roman2
*****+Tt*4Times New Roman 2
*4
/yTimes New Roman
2
/ysgn.'2/Times New Roman2
/OLSestimatedH8P,'H2,/Symbol 2
/p
oSymbol
2
ou2
u+++++
2
2
+++/
2
/
2
2
++++
2
/t 2
/t=
72
7++++O
2
O
2
2
++++
2
c2
c+++++n
2
n
2
2
+++++
2
2
"System'"Systemx_Z9'''
2
/+
<PRDDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/'_!!/G_APAPAE%B_AC_A
%!AHA_D_E_E_A
Estimatedsgnp =="++"++++"++"++""++"++"++"++++"++"++"++++"++"++"++++"++"++"++++"++"++"++++"++[].(8)
FMicrosoft Equation 3.0DS EquationEquation.39qCompObjoq~fObjInforEquation Native O
_1301379338xuFУWУWO3
(\
1"b1
(1"c1
)a1
c1
+a2
(1"c1
)a1
(1"b1
)"a2
a1
c1
+a2
(1"c1
)c1
a1
b1
+a2
"(c1
a1
b1
+a2
)c1
(a1
(1"b1
)"a2
)b1
(1"c1
)1"a1
c1
"a2
(1"c1
)b1
(a1
"1)b1
(1"c1
)"b1
(a1
c1
"1)b1
(a1
c1
"1)b1
c1
(a1
"1)c1
c1
1"(a2
+b1
)c1
c1
(a2
+b1
)"c1
(a2
+b1
)c1
(1"a2
"b1
)11a1
"(a2
+b1
)1a2
+b1
"(a2
+b1
)c1
(a1
"b1
"a2
)1"c1
1"c1
a1
"11"c1
"(c1
a1
"1)c1
a1
"1c1
(a1
"1)c1
c1
1"(a2
+b1
)c1
c1
(a2
+b1
)"(c1
a1
"1)"(a2
+b1
)c1
(1"a2
"b1
)11a1
"(a2
+b1
)1a2
+b1
"(a2
+b1
)1"(a2
+b1
)[]
FMicrosoft Equation 3.0DS EquationEquation.39qOð$
=Ole
CompObjtvfObjInfowEquation Native a2
+b1
()1"c1
()+c1
a1
"1
FMicrosoft Equation 3.0DS EquationEquation.39qt+l
(1"_1301491929zFУWУWOle
CompObjy{fObjInfoEquation Native G1TableySummaryInformation(DocumentSummaryInformation8c1
)(a2
"a1
(1"b1
))(b1
(1"c1
)"1)a3
"(c1
a1
+a2
(1"c1
))b2
"(c1
a1
+a2
(1"c1
))b3
(a2
"a1
(1"b1
))c2
(a2
"a1
(1"b1
))c3
a2
+a1
b1
c1
"c1
a1
"a2
(1"c1
)b1
(1"c1
)(1"a1
)"b1
(1"c1
)a3
+(c1
a1
+a2
(1"c1
)"1)b2
(c1
a1
+a2
(1"c1
)"1)b3
b1
(1"a1
)c2
b1
(1"a1
)c3
b1
(1"a1
c1
)"b1
(1"c1
)c1
(1"a1
)"c1
(a3
+b2
)"c1
b3
(a2
+b1
"1)c2
(a2
+b1
"1)c3
c1
(a2
+b1
)"c1
(1"c1
)(a2
+b1
"a1
)"(a3
+b2
)"b3
(a2
+b1
"a1
)c2
(a2
+b1
"a1
)c3
1"(a1
"a2
"b1
)"1(1"c1
)(1"a1
)"(1"c1
)(a3
+b2
)"(1"c1
)b3
(1"a1
)c2
(1"a1
)c3
1"a1
c1
"(1"c1
)(1"c1
)(a2
+b1
"1)"c1
(a3
+b2
)"c1
b3
(a2
+b1
"1)c2
(a2
+b1
"1)c3
c1
(a2
+b1
)"c1
1"a1
"(a3
+b2
)"b3
(a2
+b1
"a1
)c2
(a2
+b1
"a1
)c3
a2
+b1
"1[]Oh+'0
@LX
dpx(Qualitative Matrices and InformationgmladyNormalAndrew Buck9Microsoft Office Word@a@v@zLW8%
(ɋ_[C\wN߹ L̋ܲ`^p!^!k˴j*iyq
Fjo3rfJUWզZip8쭇{aD Յ<>mfmaG/jMA1zX(tyK8,,f'O}Zxs>x%J+'a[esoQj/,u.u(
jzr)cpJ;ꥢ&%U+!V`dˋ9TU﨩\Q''
bZȉz83甔,gԟӧ>QHgb)'FW7afWɹ)x6aЧx8.'?߉m"\%d'
QDτHBcyD#ќ43XEG̿L7
lOzoK9^L?tV<Dd
\
b
c$A??3"`?2ДǻȖ4㨥bkJ`!ZДǻȖ4㨥F`"!(xUkQvӏMĢ`顕Il=&q.$Ā`I/T/=xc)x?@
^`n!y3Ǜ0
'g1^
YjCWO/.v8potopz=WDhWDťxyY5t=N}\gS
]O+9Y%.On8R+<~܍63mc.%&]~t,&m&T8&
3htQ['l@
w`{Y?Ǭ闄vbqIV0ޑEėX);Qb\*Ҿ;2KLV(!H_5OYZc =:')ny}pGnU;1
}Bog zz,tzYpyY^
la˚Q"i罦`cX4~XME
fIMIwdFek.Q
.[(zqo~7tTy]r$?ϞA/r3DjDLUC2߇
&NaǁȰ BS>W"H?赐46&R37vM+YqDD&6!Wd
44]k&a`1AtoyP1Mf$h
a;VHM7.ugs(FXmS
ZQ8Sr3h%D^}qϏX}8p&Dd
`b
c$A
??3"`?2vߵ1B
J`!vߵ1B
_X
xTMhA~346?CKkAnARv9`KLɦd+1'KDHA(zɃx lZIq73
'gEZGQW6m;
.z0h"]gY`edDfnfl3bx~ o6BRaKFU^.eLX痪[S]"h}^r"7h@B @7FZ)[.B7ڇ9脷rQZ{+rAG鵾x.((a>U~5hUW@"tQ>8'E=Rw
Ɵ[?Z2mmJ:bQ*M9+fΘfND*S+2V!o˙aAP>wP*BA1c+ӱztZ[\Yv̾`҈~z̞L/uG2ETih:fO8>{,S/VߴP \Jrx&8P<$ED%n&"m
uMT!ǽK` i_6YF v'd903CNDԝMRU(ɦDd
b
c$A??3"`?2\ޙa^vD6:#RoD8Yw`!0ޙa^vD6:#RoD@Hxcdd``> @c112BYL%bpu1,IDd
b
c$A??3"`?2[fpBOsbW7kw`!/fpBOsbW@Hxcdd``> @c112BYL%bpu @c112BYL%bpu;al`C(l)BP˂\K Xz&k
r6n%2E][*7M\%Ke(z~rPA](oRf>%N_l:ʻ}>ы*\b,%M3U~YgD*6D܅x7J'gk~SrHI7ӗ{2txhyRq(aݞ\8'5gAUCUtsaT*3s\)B\ô'Gs _w5WKJpsKAJ'?k27uWUEq/([Qd?b*,MhA#gZgiΪB)ͻؤDfT,^['EUϣ}n@hosK^)i}#n͞d]N[ƢyDd
$Hb
c$A??3"`? 2
umKIѢ
)J`!umKIѢ
&@ ;"xڍMhAl?I4Mj?RcCVln*TJK"$B$[b.^OţP"jU`&"kqFa5@ӔAQNQ1T*h*uE}J~OX(jd熜aȮ$Eԋ>9 ;
Pt[wpjvyg/U)t
5juݿ^~U_ETow&63+e7Jw7JN8*A^:_'@t!&GY>4yS3c^~I%ϛ^ۑjlC=S]n
d9{1*;בKfCɩ>akF<܇9*o,f=s*b@^2y2!\#\Lr%N&8/AȮAB~8 2.0an9W<Dd
b
c$A??3"`?
2۷5QJ`!۷5Q P(+xcdd``>$d@9`,&FF(`Ti A?d,@=P5<%!8 :@Abj&K)ff7ƿ'I9o._ sy"+8&b#~#{Lը$dU.1܇ #4+!no`
q{^]1 b`abPp,LZrqbԮx@FUBOP.LL(߁Y?a ]aɢjXNT~+(9@npH>*)7Ȉ"=@p*oE\ДNaMLLJ%?A`Pdk)Y=̘Dd
Tb
c$A??3"`?2ibO&K(B
P%^E0J`!=bO&K(B
P%^ XJhxcdd`` @c112BYL%bpufU9>GGAxCJ(äV^AX$`79nDd
p@J
CA?"
2r=+xn~1N5J`!F=+xn~1
xcdd``fed``baV d,FYzP1n:&V! KA?H1:
@eGqC0&dT20 `[YB2sSRsW)bF: pi#LT%/6)L +ss\ǰ? ٳ"n+a{xYVF=}L \P,D.Pp\021)Wx ],`YF('f8QnDd
\
c$A??#"`2KU!NtH$*D!7J`!KU!NtH$*D`41Qj`xڵYMLG{mֆ)Qi1uI6
\F +82?6VK!Z(*4T4TQQCoPNDm*=$HU^8\z!顪 Hugv;bkvg<7a
Ax
0 BQ6Uz/ژ(%'N K %#)ҿON>]DS+U
fEmY_AA6<935Do FD]MZ?Oo#_ڰoV7qVS]T]n+X];nHCOިH:tT+իc:H!{{4ZY$N#UDtiNsA((5"odqeĀQirH%~}O P7@{'P6_HmoNǧշ̭Oض=_7݊k*ODd
3
J
CA?"2PՃ[jI
0:s]=J`!UPՃ[jI
0:s @ j#xmkA쏹6z'1p>$x)$pc!ErHX_c6ϙݽeCfg>5D
>q,߭_"cv:ӂ[42gJMO7k~/@oO;ia~~52~oπ4#;7)iyO!_J9'@ލ
.'վ^iyO!_J9'@ލ
Y{~mWm=zvݞ5U
Bx믪h9jiB&ѯs/2RRzRScc%ꋥO'ɏp#0~&ԓ '?x)x)ku#Ӣi`[MQmwer?)EODd
\
b
c$A??3"`?2#s}e7{X/@J`!#s}e7{X/ `"!jtxVMLSAR
cB@}x0Hlix4Tx&PHA.ji8Bh0z4xh̓ğDCA֙>^ظٝofgfw4?EHX9#(NX#GBsMe
υuXm7FYN].j8G ǉjYvю;,+V+2am{af:=мp~sѼd*F08頵$
x:b+ Op*xe+,%5~řDʿ
m1m#xYGxmm6Xѣb5=*e?e^Dνsl̊ƾkBoF70}uf
݊JW59T)obȲqVqyRôRt?CVi/6ZgޒXkm^ŕ崳BjJZ{ڻmorYrגv>o97$NjYMVkŕ
kǔʵw{t{gg3CuaLI5KAb Cxt(z
b2g㡙 m$2*PJNjiEs8x~+
]@,8umJr_)n9
쌅cm(_gJiQv̉Ǖl:럭0yrea~k7FԵ"((CWNow4kg99M&T`e0̖=FWȴ{ԃG
TT1 tjj1̂ǳeNTT;^[GL,P'nVaò@[^L$OD"&? y
.%d2
a]nє`Jay LK}THӔW7Je'=
>
N6AlC{Rk4w2*̑yAI@rw4Rb?SfryŪ0n/*i($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdSI$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdL$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdO$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdR$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdV$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkd9Y$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdg\$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkd_$$Ifl 5Y2]~ $
t06$$$$44
layt($$If!v h5t55$5'5{575+5!5 ~#vt#v#v$#v'#v{#v7#v+#v!#v ~:Vl
t0655555555 ytkdb$$Ifl 5Y2]~ $
t06$$$$44
laytDd
3
b
c$A??3"`?27b?租̝HBA5fJ`!b?租̝HBAR(H8x}ylUl__m^R豭VtKDŢ&؈J4#W4+&F⑪
$G&ݺ·m·}oN!=BDHٯG.M>\SSSFl
s<.߳>2OTzۼ>JگM+H110'\Vzkӟ)M]Ⱥ5\$D4`q?m_~ Xhe"]H^ByB7=<_'oeyrwkY}Ds%$i'yc>#uf_7O?ryA7D)}Dg}ʳ.J@n~{I(Z#$Ϫ t'[do)[oJ)l5<ߒkJ))yo.1Wg ^D5[8݃ȲtGRAꢖJ+^k?e_V]/Voj"H.w~(mѐ'2qMdp3\S{ܒcws.
o~_[Mf*.gA{s}_vrӲ/I:hVΧ~O6>H[MJ?uueqNNO
:%g}hK;ߓFW~GX_aNm_X騣YK ~G8.n,5>TAZ1N}{X\bXyb)扥'bXyb)扥'bXyb)扥'bXybJ)'dybJ)'<[Uo췥~3*d}~9w=^7qr~K迧~^?GgGX:j.^mye#ї~U[`o)
={m\F]z ڭo{+ǿL5o<ί}.\u0qͦ>ZuUrOuZ):fp>_~a?3)Srnkp.WDv9NBm9%{PcϊP?KUPCk{7R?<N/vmכϪ#UEA?7pe#!?&ȯ6zr,GbKyNSgL}O=z?cV?cgLq)IHIȪnr062KBr$$KBr$$~"ol!_N=yrd)?^D}$߅(gOj'.%?Gԗt$[xl$#:s!o
gRr^/7NfLad> ܶD^ڝDsjVb:q(}vlqs Js3NDd
b
c$A??3"`?2::XqTha*"nJ`!::XqTha*zPthxڕKA<Ci%
!A$I tlѱC%P©>AN+ǥ\'Њ)BO
GV
Dd
G
Lb
c$A??3"`?2 !9]hw/k6 pJ`!t !9]hw/k6"r8B x} pTݼ $!HIXC,ZZ'3MvXaTmj"t'S~;?Jط)MITB"_Q{l*^wf9˦Ğ<*j$(wCJlz%fEy%&#\Ju{7V3P۶1 %6vwlܠ(UCnli~z#8#gFH(0&rA5lh~yۂRe)!%rA5lh~y`OϐpEYFKŴ̡H2{"pܳp}RO{>iߞB{rO(?f}]dzr}+SG~"\E6Ҭ0&N}oȩV^NJ^דY?^Y?^Y?^o9ɟjCi/h/h/h/h/6OG}ٗo #_A#/*%6\Gb6~r*c41&?ЊWiu"giS'؏~Tώ!w٢ǾU6%xU;}ߪJ5w=1feɹj qBJrə7Ui[W+
irFBDE$R_q~\otY'apY\VkzMc}Wk+5ϡa[)~ko0߶tHmO'R"r~~$s93aK~,.32FseJ>?`i0w2J{]rS7b=?SkSoa9RoW1w͜}Ի˦^K6,dO$G4>,:WbH^[I}O$}"Yɷ
/dOsObHDD}"A(ϏѨ0Ѫ?M{k}!`_<;F{B+_kf6R拫Z?SK/
VP%ԿPLEԏ, ϣ^\{y\_eY}372y<1ԟ3ә/",%z7&2 >8o8PwMN~tH6pP_FsnsuI7.ב[NJA5\K#5Ca,3$0]EUf~oU2ReJ0߯gc<<ߕGrywh&Oe<3xLyPz:1M$My"Xe4G3ߧq~o_EhuӿI?܈ 9pWy*}%ܿa0U㌝aT9/6k_cΖkmF.&;j(%)yQ*M}+n{vr
^{\כeSiKɥ韚H.'yVn6Fk~zVX9htO>u?LR?pqe6_[wzwzwzwzwzwzcDH_=ç?ç?ç?ç?ç?ç?ç?ç?ZA_+:s~8v1TOrO%PNboC{.̢}LiϟBm#ǉd{yi_6Cߑ?w6݀a[kɓ;G8l9lmԑ+A
_,!M.wd;4OqgTS;&QbD7Ik+*&3Dz{]ݭC"%/l՜.+,D՜.+,Php
~k'%Qualitative Matrices and InformationTitleH 6>
MTWinEqns
FMicrosoft Office Word Document
666666666vvvvvvvvv666666>6666666666666666666666666666666666666666666666666hH666666666666666666666666666666666666666666666666666666666666666666D@DI_Normal5CJ_HaJmH sH tH DA@D
Default Paragraph FontRi@R0Table Normal4
l4a(k(
0No List>@>
G*0
Footnote TextCJaJ66G*0 Char Char2CJaJ@&@@
G*0Footnote ReferenceH*J^@"J 0Normal (Web)dd[$\$5PJ424
=0Header
H$:A:=0 Char Char15CJaJ4 @R4=0Footer
H$8Oa8=0
Char Char5CJaJB'qB!Comment ReferenceCJaJ<<!Comment TextCJaJ>j>!Comment Subject\HH!Balloon TextCJOJQJ^JaJ( /083MUO[ijYZ[\]§ħʧ˧ͧЧҧԧ֧اڧܧ
$%')+/135;<>@BDFHJLRSUWY[]_acijou{
}~³óOP:;TUlmst89A,}~$%;<hi78KLABtuwxDE,wOIJgUV!^_abdeghjk00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@00%0s@000@000@000@000@0@00
0\@0@000\004 ++++++++++++++++++++++++++++++++++++++++++JJJJJJJJJMm*4XCdLNOT~`lxun&\Fv6hpzHT(.r`vyz}~6Pqڌ 7>Thzhhi.HZw{x000777?0?2?BEBGB)DADCDLLLe\y\{\\\\\\\khhhiiiiiiwkkkk lllllpppppprrrrs
stttó׳ٳ::::::::::::::::::::::::$'+CFM!!8@0(
B
S ? OLE_LINK1 OLE_LINK2FL'qLnNoTnp'q[pqrn'q\p^pnrnmt>o47oBonlnnnnDmmmdmmmSomtmm4m?A@BCDE a6OfPdWf&gRk]m`mmmzyooүze+//**ff??Xallee
!"#$%&'()+*,.0/123456789:;<=>?A@BCDE9F*urn:schemasmicrosoftcom:office:smarttagsplace=C*urn:schemasmicrosoftcom:office:smarttags PlaceName=D*urn:schemasmicrosoftcom:office:smarttags PlaceType9=*urn:schemasmicrosoftcom:office:smarttagsState8>*urn:schemasmicrosoftcom:office:smarttagsCityB.*urn:schemasmicrosoftcom:office:smarttagscountryregion><*urn:schemasmicrosoftcom:office:smarttags
PostalCode 0urn:schemas:contactsSnZFFDCFDDF>=<FFF>FFFFFFF00.FF=FFFFFFF>F=F==FF>=>=FCD>F=DFDDFC=F=F>FCDFF>mupq~ &+ . "# ##''@+M+//W0Z0^1a12233333333=4@444555576:666666666<7=777::;;;;<<==mApABBNDQDuH}HIIIIIIKKKK`LcLLLNN"O%OOPRPPPWTXTTTUU U#UdUlUaAabb1c4cccffi!iii/k2k_kjk$l'l{l~l!n'nwwĀʀUV!"/0;<̈́΄˅̅مڅɆʆ׆؆ABOP]^klǇȇՇևMN[\ijwxňƈefstqr}~ .{+,9:prFGTUbcpq~DERSʒ˒ْؒlmz{jkxy
()
&'45$%23@AƗǗLMÙЙљޙߙ,:;HIVWdeΚϚܚݚFGTUbcpq67DERS`ano}ќBHϝНqt *OU()4LRgnfp__aabbdeghjk%'2>JUno:!B!$$,,[.`.....2050Z1^11133446677::::`=g=)@@BBBBKDNDEEEEGGII^J`JKKKKILPLLLNNLPOPPP(QQQQRRTTVVYY]\e\\\^^__w`}```bb.c1ccc}ffiiukwkkkllrruuT\=AÂǂIMσӃUYۄ߄aemqy}#'/3;?ō؏ڏ
"*.6:BFȖ̖NRԗؗZ^fjrv~]eFN!.9@}7?Bmv,1JP bf__aabbdeghjk3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333w77D)Dpptt__aabbdeghjk__aabbdeghjk\$]Y1X!}U;m5frU
2w=gwq )9H,"/sL72gy^'.[+\ !!m"''GT'(8*R+?2d23 %3Si3/w468^ ;u< !=tj>.y>,AF?Bm6CDDB:Dp^DF*G#gJ'yJKMAJMhNpOw\PLR$S_XeY$Z<[O1\qV\wD^u^I_^`bb%#bXb<]bdcdwd\ev/fff`HgQggh2jTk[l'mQum 0nLSnkjnq+qSq r.rlu?v#xx+zW{z}}}aeHbU1v=1bj+
xvs=7`ra#}WYxg89>pp>p1v;)mKuhyYO
V
7)x/yE6]=Q{,LF_J!\A(06c(Cen
`9:XpoSng#=Tx{X,UG*lN#BaM\]§ħʧ˧ͧЧҧԧ֧اڧܧ
$%')+/135;<>@BDFHJLRSUWY[]_acijou{
}³%e0@P@PBP@PHPLPPPZP@P`PbPhPlPnPxP@PP@PPPPPPP$@PP,@PPP<@PPPPL@PPPPPh@PPp@PPx@PP@PP@PP@PP@PPP@P0UnknownAndrew BuckGz Times New Roman5Symbol3&z Arial?5 z Courier New7&{ @Calibri5&zaTahoma"1h&fӦ l{rk~!s!4d2QHX $P/w42$Qualitative Matrices and InformationgmladyAndrew BuckCompObjqMSWordDocWord.Document.89q