Economics Letters 5 (1980) 0 North-Holland Publishing
53-57 Company
A SIMPLE FORM OF THE MAXIMUM ENTROPY MOMENT MATRIX AND ITS INVERSE *
Henri THEIL University of Chicago, Chicago, IL 60637, USA Received
16 May 1980
The maximum entropy (ME) moment matrix is always non-singular due to a ridge above the diagonal. This paper gives an explicit expression for this ridge and for the inverse of the ME moment matrix.
1. Introduction
Theil and Laitinen (1980) proposed a maximum entropy (ME) fit to a sample (XI, .‘.>x,) from a continuous distribution, the result of which may be summarized as follows. We arrange the sample values in ascending order, x1 < x2 < *..
=
;(xc + xifl)
)
i = 1, . .. . n - 1
(1)
for the II - 1 midpoints between the sample values. These midpoints define two open-ended intervals, II = (--, tlI), and Z, = ({n-1, -), and n - 2 bounded interVdS, Zi = ((i-1, .\$i) for i = 2, .. . . II - 1. The ME fit selects the density function with maximum entropy subject to mass- and mean-preserving constraints. ’ This density function is exponential in the open-ended intervals and uniform in each bounded interval. Therefore, the cumulated distribution function of the ME fit is continuous everywhere and piecewise linear except for exponential tails. The expectation 5? of
* Research supported in past by NSF Grant SOC7 6-8 27 18. Helpful discussions with James F. Meisner are gratefully acknowledged. 1 That is, subject to constraints on moments of the order zero and one. Since the main objective is the second-order moments, a natural procedure is to organize the information supplied by the sample in terms of lower-order moments. Also, the ti’s do not have to be defined as midpoints. We may replace (1) by
53
H. Theilj A simple form of the ME moment matrix and its inverse
54
the ME distribution xi = \$(ti_r
+ ti) )
of interval li equals i= 1. . . . . ~1,
(2)
where to = x1 and t, = x”. Thus, when designating the g’s as primary midpoints (including to and t, which are ‘honorary’ primary midpoints), we can view the interval means Fr, ... . x’” as midpoints between primary midpoints and, therefore, as secondary
midpoints.
Next consider a sample (x1 ,yr), . . . . (xn,yn) from a non-degenerate continuous bivariate distribution. Again, we indicate by superscripts arrangement in ascending order; thus, fork = 1, .... n, (xk,yk) = (x’,yj) for some (i,j), which means that the kth observation is the ith in ascending order of the first variable and the ith in ascending order of the second. The n f 1 primary midpoints of the latter variable are qO =y’, qj = \$Cy’ +yi+l) forj = 1, . . . . n - 1, and qn = y”, while the n secondary midpoints are 7 = i(77j-r + ?j). Since (xk, yk) = (xi, y’) holds each k and some (it i), this defines (&, yk) = (2, v’/) for k = 1, .., n. Then
(3) is the second-order cross-moment of the bivariate ME distribution which is obtained by maximizing the bivariate entropy subject to mass- and mean-preserving constraints. The ME covariance is found by subtracting the product of the sample means from (3). These means are also means of the ME distribution because of the meanpreserving constraint.
2. The ME moment matrix and its inverse Eq. (3) does not apply to X = Y because IZ(X*) exceeds (l/n) in section 3 that this excess equals
which is thus equal to a fraction 1 /12 of the mean between the primary midpoints plus two end term of the ME moment &(X2) over (l/n) c& implies ME moment matrix has a ridge above the diagonal non-singular.
c&-.
It is shown
square successive difference corrections. * This positive excess in conjunction with (3) that the which causes the matrix to be
* Theil and Laitinen (1980) proved that G(X*) exceeds (l/n) zk?k, but they did not give an explicit expression for the excess: The formulation in terms of primary and secondary midpoints is also new.
55
H. Theil /A simple form of the ME moment matrix and its inverse
Let there be p variables and n observations on each. The ME moment matrix is then of order p X p and equals C + D, where C is the matrix of mean squares and products of the secondary midpoints and D is the diagonal matrix with the righthand side of (4) as typical diagonal element. Then
Z'DZ=I
Z’CZ=A,
(5)
is a simultaneous diagonalization of C and D,where Z = [zr .. . zP] is a matrix of characteristic vectors and A is a diagonal matrix with the latent roots hr Z Xz > ... Z h, > 0 on the diagonal. It is shown in section 3 that
(c+ D)-’ =D-l -
\$ kziz;
,
1
where k = min(p, n). It is also shown in section 3 how this result can be used to obtain the inverse of the ME moment matrix for more than n variables from the inverse of the ME moment matrix of a subset of n variables.
3. Derivations By combining
&~~=Yjl
eqs. (A.lO) to (A.12) of Theil and Laitinen (1980) 3 we obtain
~(X21X~Z~)-~~‘(4j-ii-~)’ + [z(xlx~zr)]”
+ [S(XlXW,)]’
.
Since IS,“,r&(X2jXEli)=n&(X2)and we can write this as [~(XIX~Ij)12,
~(X2JXEli=‘var(XIXEIi)+
n&(X*) - kt
tvar(XIXEIr)tvar(XIXEZ,).
%izAng
(gi-li_r)*
(7)
For the last variance, (AS) and (A.6) are applicable with (II= &_r and fl= i(tn-r + .!&).Therefore, the variance implied by (A.6) equals (a: - 0)’ = d(.\$, - ,\$+r)‘. Similarly, var(XIXEIr) = \$(gr - to)’ so that the right-hand (7) equals
side of
which completes the proof of (4). 3 The equations cited in this paragraph
are from Appendix
A of Theil and Laitinen
(1980).
56
H. Theil/ A simple form of the ME moment matrix and its inverse
From(5),Z’(C+D)Z=I+A,orCtD=(Z’)-’(ItA)Z-’ Z(I + A)-’ Z’, or
sothat(CtD)-‘=
From Z’DZ = I, D = (Z’)-’ Z-’ and hence D-’ = ZZ’ = I\$, zizi. On combining this with (8) we obtain (6), recognizing that the number of non-zero hi’s does not exceed n. Let n be fixed and let the number of variables increase from p 2 n to p t q. This changes C into [“WC F&/l
= [ ;:;;!;Y;-r
\$;Z,;Y;Y
J
(9)
for some p X q matrix W, and it changes the ME moment matrix C f D into
(Z’)_’ (Z t A) z-’
(z’)-’ AZ-’ w
w’(z’)-’ AZ-’
D, t W'(Z')-'
1
AZ-’ W ’
(10)
where D, is the q X q diagonal matrix whose diagonal elements are of the form of (4) for the q added variables. The inverse of (10) is Z(Z t A)?Z i _ V-r B’
t BV-‘B’
-BV-’ v-l
12
(11)
where B = Z(Z t A)-’ AZ-’ Wand I/= D, t (Z-’ W)‘(Z + A)-’ AZ-’ W. Thus, the inverse of the ME moment matrix of order (p + q) X (p t q) can be written as Al + A2, where A1 is the inverse Z(Z t A)-‘Z’ of the p X p ME moment matrix bordered by zeros, and
which is a positive semidefinite matrix of rank q. We can view B and V as the coefficient matrix and the residual moment matrix, respectively, of a set of ridge regressions. 4 It would be worthwhile to pursue this interpretation further.
4 See Hoer1 and Kennard (1970) and Smith and Campbell (1980) as well as the comments following the latter article. Note, however, that the ridge of the ME moment matrix results from an explicit criterion, viz., that of maximum entropy subject to mass- and mean-preserving constraints. Also, it may be possible to relate the result (11) to Meisner’s (1980) empirical finding that the mean squared errors of the elements of the inverted ME moment matrix for fixed n increase with p until about p = n and then decrease.
H. Theil /A simple form of the ME moment matrix and its inverse
References Hoerl, A.E. and R.W. Kennard, 1970, Ridge regression: Biased estimation for nonorthogonal problems, Technometrics 12, 55-67. Meisner, J.F., 1980, A simulation study of the maximum entropy moment matrix, Economics Letters 4, no. 4, 333-336. Smith, G. and F. Campbell, 1980, A critique of some ridge regression methods, Journal of the American Statistical Association 75, 74-81. Theil, H. and K. Laitinen, 1980, Singular moment matrices in applied econometrics, in: P.R. Krishnaiah, ed., Multivariate analysis-V (North-Holland, Amsterdam) 629-649.
57 In stock

Free download or read online From This Moment pdf (ePUB) (After We Fall Series) book. The first edition of the novel was published in October 5th 2017, and was written by Melanie Harlow. The book was published in multiple languages including English, consists of 352 pages and is available in Paperback format. The main characters of this romance, romance story are , . The book has been awarded with , and many others.

Suggested PDF: Free Fall: A Prelude to Hidden Order by Brad Thor pdf ## From This Moment PDF Details

 Author: Melanie Harlow Original Title: From This Moment Book Format: Paperback Number Of Pages: 352 pages First Published in: October 5th 2017 Latest Edition: October 5th 2017 Series: After We Fall #4 Language: English category: romance, romance, contemporary romance, contemporary, adult, new adult, childrens, adult fiction, erotica, doctors, death, fiction Formats: ePUB(Android), audible mp3, audiobook and kindle.

The translated version of this book is available in Spanish, English, Chinese, Russian, Hindi, Bengali, Arabic, Portuguese, Indonesian / Malaysian, French, Japanese, German and many others for free download.

Please note that the tricks or techniques listed in this pdf are either fictional or claimed to work by its creator. We do not guarantee that these techniques will work for you.

Some of the techniques listed in From This Moment may require a sound knowledge of Hypnosis, users are advised to either leave those sections or must have a basic understanding of the subject before practicing them. DMCA and Copyright: The book is not hosted on our servers, to remove the file please contact the source url. If you see a Google Drive link instead of source url, means that the file witch you will get after approval is just a summary of original book or the file has been already removed. ## PDF's Related to From This Moment

 Free Fall: A Prelude to Hidden Order by Brad Thor Free Fall by Fern Michaels Free Fall in Crimson by John D. MacDonald Free to Fall by Lauren Miller Free Fall by Kyle Mills Free Fall by Robert Crais In Free Fall by Juli Zeh Free Fall in Crimson: A Travis McGee Novel by John D. MacDonald