Theory of Probability

Boris V. Gnedenko

• Publisher: Routledge
• Year: 2018
• ISBN:
  9781351408585
• Edition: 6
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

This book is the sixth edition of a classic text that was first published in 1950 in the former Soviet Union. The clear presentation of the subject and extensive applications supported with real data helped establish the book as a standard for the field. To date, it has been published into more that ten languages and has gone through five editions.
The sixth edition is a major revision over the fifth. It contains new material and results on the Local Limit Theorem, the Integral Law of Large Numbers, and Characteristic Functions. The new edition retains the feature of developing the subject from intuitive concepts and demonstrating techniques and theory through large numbers of examples. The author has, for the first time, included a brief history of probability and its development. Exercise problems and examples have been revised and new ones added.

Chapter 1

INTRODUCTION

The goal of this book is to present the basis of probability theory – a mathematical discipline studying the principle of random events.

The beginning of probability theory can be traced to the middle of the seventeenth century and connected with the names of Hyugens (1629–1695), Pascal (1623–1662), Fermat (1601–1665), and Jacob Bernoulli (1654–1705). In correspondence between Pascal and Fermat, one can find the first notions which step-by-step crystallized into a new branch of mathematics. The problems of interest were stimulated by tasks connected with hazard games; these problems lay beyond mathematics of that time. We should understand that the outstanding scientists analyzing hazard games foresaw a fundamental philosophical role of the science of studying random events. They were convinced that on the basis of mass random events there could be built strict mathematical principles. Only a current state of natural sciences led to the situation that hazard games were a main experimental basis for probability theory. Of course, it put its seal on the formal mathematical methods restricted by simple arithmetical calculations and combinatorial technique. Development of probability theory under the influence of natural sciences, in first order, physics, showed that the classical methods are still interesting even now.

Serious demands from the practice of natural and social sciences (first of all, theory of errors, gunfire tasks, and demography) led to the further development of probability theory and involved advanced analytical tools. An extraordinary role in developing probabilistic methods belongs to DeMoivre (1667–1754), Laplace (1749–1827), Gauss (1777–1855), and Poisson (1781–1840). The work by Lobachevsky (1792–1856), the creator of Non-Euclidean geometry, dedicated to the theory of errors of measurements on a sphere, also has a close relation to this direction.

From the middle of the nineteenth century up to the twenties of the current century, the development of probability theory is connected mainly with names of Russian scientists – Chebyshev (1821–1894), Markov (1856–1922), and Lyapounov (1857–1918). The success of the Russian science was prepared by the activity of Bounyakovsky (1804–1889) who widely propagated research in applied probability, especially in insurance and demography. He wrote the first textbook on probability in Russian which influenced the development of works in applied and theoretical probability. The main influence on the development of probability theory was due to works by Chebyshev, Markov, and Lyapounov who introduced and widely used the concept of random variable. Chebyshev’s results connected with the Law of Large Numbers, Markov chains, and Lyapounov’s central limit theorem will be examined later in this book.

The modern state of probability theory is characterized by an increasing interest in it and its wide penetration into practical applications. The Russian probabilistic school continues to dominate in the area. Among the first generation of Soviet scientists, the names of Bernshtein (1880–1968), Kolmogorov (1903–1987), and Khinchine (1894–1959) are prominent. During the presentation of the material in this book, we will inform the reader about main results and their influence on the development of probability theory. Thus, even in Chapter 1 we will discuss the fundamental works by Bernshtein, Mises (1883–1953), and Kolmogorov which are the basis of modern probability theory. In the 1920’s, Khinchine, Kolmogorov, Slutsky (1880–1948), and Levy (1886–1971) established a connection between probability theory and the metric theory of functions. This allowed the discovery of a final solution of the problems which were formulated by Chebyshev and widened the content of probability theory. In the ‘30’s Kolmogorov and Khinchine created the theory of stochastic processes which contains now the main direction of analysis in probability theory. This theory can serve as an excellent example of organic synthesis of mathematics and physics where a mathematician with an understanding of a physical problem is able to create for its solution an adequate mathematical tool. This part of probability theory will be considered in Chapter 10.

After molecular theory of substance became widely accepted, the use of probability theory was inevitable in chemistry and molecular physics. Notice that from the molecular physics viewpoint, any substance consists of a huge number of particles which are in continuous movement and interacting. The nature of these particles, the principles of their interaction, the character of their movement, etc., are not well known to us. The only reliable information is that there are many such particles and, in a homogeneous substance, their characteristics are close. Naturally, in such conditions traditional mathematical methods became powerless. For instance, differential equations methods cannot lead to success. Indeed, we usually know very little about these particles. Even if we knew almost everything about them, it would be impossible to make a solution with the help of standard methods of mechanics because of the enormous number of equations describing the physical object.

At the same time such an approach would be unacceptable from a methodological viewpoint. Really, we are interested in behavior of the mass of moving particles rather than in a collection of individual behaviors. The mass behavior cannot be obtained by a simple summation of behavior of individual particles. Moreover, in some limits the mass behavior becomes independent of the behavior of individual particles. Doubtless, an investigation of the new phenomena needs a new mathematical technique. What requirements should these methods satisfy? First of all, the huge number of particles must make the analysis of the phenomenon easier. Further, the lack of detailed knowledge about individual behavior of a particle must not be a principle obstacle for the analysis. These requirements are best of all satisfied by probability theory.

To avoid the erroneous impression that probability theory is used because of lack of knowledge, we emphasize the following. The philosophical basis for the use of probability theory lies in the fact that “mass” phenomena generate new principles. When a analyzing a complex natural phenomenon one needs to take into account only crucial features and avoid incorporating superfluous ones.

The measure of the fruitfulness of mathematical formalization is concordance between theory and practice. The development of the natural sciences, especially physics, confirm the extreme usefulness of probability theory for the purposes of mathematical modelling.

This connection of probability theory with practical needs explains why this branch of mathematics has flourished in recent decades. New results allow the solutions to new practical problems. New problems force the development of the theory. Of course, everything said before does not mean that probability theory is just an applied branch of mathematics. On the contrary, the experience of the last decades demonstrates that it became an elegant mathematical discipline with its own problems and methods of proving.

It was said in the beginning that probability theory deals with “random events.” The strict definition will be given in Chapter 1. Here we restrict ourselves to several remarks. In everyday life we refer to random events as something rare and irregular, coming against usual perception. We will reject such an understanding. Random events in probability theory possess a set of properties, in particular, they occur as a mass phenomenon. Such phenomena are described by an enormously large number of equal or almost equal objects and do not significantly depend on the nature of individual objects. Such mass phenomena are usual in physics, econometrics, telecommunications, and military applications. Statistical quality control of mass production is exclusively based on probabilistic concepts. One of the most serious problems of contemporary engineering is a reliability problem which uses as its dominant tool varied probabilistic methods. It is time to mention that in its turn the needs generated by reliability theory have an influence on probability theory.

At this point it is appropriate to remember words said by the Founder of the Russian school of probability theory, Chebyshev: “Closeness of theory and practice gives the most fruitful results. Not only practice gains from this. Science is developing under the influence of practice: new objects for investigation appear, new sides of known objects become open…. Science profits from new applications of old methods, but even more so it gains from new methods by choosing a reliable guide among practical applications.”

Chapter 2

RANDOM EVENTS AND THEIR PROBABILITIES

1 INTUITIVE UNDERSTANDING OF RANDOM EVENTS

For a very long time people have investigated and used for practical purposes only so-called deterministic laws. Most of the knowledge obtained in school courses on physics, chemistry, and mathematics relates to this field. Consider some examples.

If a pyramid has as its base a square with a side equal to a and its height equals h, then the volume of the pyramid equals (1/3) a²h.

If a body falls to the Earth’s surface, then the path coming in t seconds after the beginning of the fall equals S = (gt²)/2.

If chemically pure water is heated up to 100°C under the atmospheric pressure 760 mm of mercury, it begins to vaporize.

The number of such example’s can be increased with no limit. But not all situations which we meet in practical and scientific activity can be described by such forms of laws. For example, consider several questions. What is the number of tomorrow’s traffic incidents in the New York area? What is the maximal level of water that will be observed at the Mississippi River in the St. Louis area at the next flood? How long will it take to repair the fiber trunk of telecommunication network cut by diggi...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Translator’s Preface
• Author’s Preface to the English Translation of the Russian Sixth Edition
• Author’s Preface to the Russian Sixth Edition
• From the Author’s Preface to the Russian First Edition
• Foreword
• Biography
• 1 INTRODUCTION
• 2 RANDOM EVENTS AND THEIR PROBABILITIES
• 3 SEQUENCES OF INDEPENDENT TRIALS
• 4 MARKOV CHAINS
• 5 RANDOM VARIABLES AND DISTRIBUTION FUNCTIONS
• 6 NUMERICAL CHARACTERISTICS OF RANDOM VARIABLES
• 7 THE LAW OF LARGE NUMBERS
• 8 CHARACTERISTIC FUNCTIONS
• 9 THE CLASSICAL LIMIT THEOREM
• 10 THE THEORY OF INFINITELY DIVISIBLE DISTRIBUTIONS
• 11 THE THEORY OF STOCHASTIC PROCESSES
• 12 ELEMENTS OF STATISTICS
• APPENDIX: THE HISTORY OF PROBABILITY THEORY
• Tables of Function Values
• References
• Index