Discussiones Mathematicae Probability and Statistics 23(1) (2003) 5-44

STOCHASTIC DYNAMIC PROGRAMMING WITH RANDOM DISTURBANCES

Regina Hildenbrandt

Technical University Ilmenau, Institute of Mathematics
PSF 10 05 65, D-98684 Ilmenau, Germany

Abstract

Several peculiarities of stochastic dynamic programming problems where random vectors are observed before the decision ismade at each stage are discussed in the first part of this paper. Surrogate problems are given for such problems with distance properties (for instance, transportation problems) in the second part.

Keywords: Stochastic dynamic programming, Markov decisions process, dominant policy, distance properties, (partial) certaintyequivalence principle.

2000 Mathematics Subject Classification: 90C39, 90C40.

References

[1]R.K. Ahuja, T.L. Magnanti and J.B. Orlin, Networks Flows. In: Handbooks in Operations Research and Management Science, Vol. I. Ed. by Neuhauser, G.L.; Rinnooy Kan, A.H.G.; Todd, M.J. Elsevier Science Pub. B.V.; Amsterdam 1989. 
[2]D.P. Bertsekas, Dynamic Programming and Optimal Control, I and II, Athena Scientific, Belmont, Massachusetts. New York-San Francisco-London 1995.
[3]D.J. Daley, Stochastic monotone Markov processes, Z. Wahrsch. Verw. Gebiete, 10 (1968), 305-317.
[4]W. Dinkelbach, Entscheidungsmodelle, Springer-Verlag, Berlin-New York 1982.
[5]S.E. Dreyfus and A.M. Law, The Art and Theory of Dynamic Programming, Academic Press New York-San Francisco-London 1977.
[6]H.J. Girlich, P.M. Köchel and H.U. Küenle, Steuerung dynamischer Systeme, Birkhäuser, Basel 1990.
[7]O. Hernández-Lerma, Adaptive Markov control Processe, Springer-Verlag, New York-Berlin 1989
[8]R. Hildenbrandt, A special stochastic decision problem, Optimization 28 (1993), 95-110.
[9]R. Hildenbrandt, Methoden aus ganzzahliger Optimierung und Verbandstheorie zur Behandlung eines stochastischen dynamischen Transportproblems, Habilitationsschrift, TU Ilmenau 1995, (Libri BoD, 2000).
[10]R. Hildenbrandt, Zu stochastischen dynamischen Optimierungsproblems, Preprint No. M17/96, TU Ilmenau 1996.
[11]R. Hildenbrandt, Notes of stochastic dynamic programming, where the random disturbances are observed before the decision is made at each stage and corresponding problems with distance properties, Preprint No. M08/00, TU Ilmenau 2000.
[12]P. Kumar and R.P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, New Jersey 1986.
[13]A. Müller, Integralinduzierte Ordnungen und Metriken auf Mengen von Wahrscheinlichkeitsma\ßen mit Anwendungen bei Markovschen Entscheidungsprozessen, Dissertation, Universität Karlsruhe 1995.
[14]K. Neumann, Operations-Research-Verfahren, Band II Carl Hanser, München 1977.
[15]K. Neumann and M. Morlock, Operations Research, Carl Hanser Verlag München, Wien 1993.
[16]C. Schneeweiss, Dynamisches Programmieren, Physica-Verlag Würzburg-Wien 1974.
[17]H.J. Sebastian and N. Sieber, Diskrete dynamische Optimierung, Geest & K.G. Porting; Leipzig 1981.

Received 15 January 2002
Revised 6 May 2003