Discussiones Mathematicae Probability and Statistics 20(2) (2000) 167-176
K.B. Athreya School of Operations Research |
J.D.H. Smith Department of Mathematics, Iowa State University |
Entropy maximization subject to known expected values is extended to the case where the random variables involved may take on positive infinite values. As a result, an arbitrary probability distribution on a finite set may be realized as a canonical distribution. The Rényi entropy of the distribution arises as a natural by-product of this realization. Starting with the uniform distributionon a proper subset of a set, the canonical distribution of equilibriumstatistical mechanics may be used to exhibit an elementary phase transition, characterized by discontinuity of the partition function.
Keywords: canonical distribution, canonical ensemble, Gibbs state, phase transition, entropy maximization, Rényi entropy.
1991 Mathematics Subject Classification: 82B26, 94A17.
[1] | J.W. Gibbs, Elementary Principles in Statistical Mechanics Developed with Especial Reference to the Rational Foundation of Thermodynamics, Yale University Press New Haven, CT 1902. |
[2] | E.T. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106 (1957), 620. |
[3] | M. Tribus, Rational Descriptions, Decisions and Designs Pergamon, Elmsford, NY 1969. |
[4] | R.D. Levine and M. Tribus (eds.), The Maximum Entropy Formalism, MIT Press, Cambridge, MA 1979. |
[5] | J.H. Justice (ed.), Maximum Entropy and Bayesian Methods in Applied Statistics, Cambridge University Press, Cambridge 1986. |
[6] | W.T. Grandy, Jr., Foundations of Statistical Mechanics, Volume I, Reidel, Dordrecht 1987. |
[7] | A. Rényi, On measures of entropy and information, in Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability 1960, Volume 1, University of California Press, Berkeley, CA 1961. |
[8] | J. Aczél, Measuring information beyond information theory - why some generalized information measures may be useful, others not, Aequationes Math. 27 (1984), 1. |
[9] | W. Pauli, Die allgemeinen Prinzipien der Wellenmechanik, in ``Handbuch der Physik, Bd. 24/1 Quantentheorie'', Springer, Berlin 1933. |
[10] | C. Shannon, A mathematical theory of communication, Bell Systems Technical Journal 23 (1948), 349. |
[11] | K.B. Athreya, Entropy maximization, IMA Preprint Series # 1231, Institute for Mathematics and its Applications, University of Minnesota, Minneapolis, MN 1994. |
Received 3 February 1999
Revised 15 March 2000