An Introduction to Statistical Signal Processing
The origins of this book lie in our earlier book Random Processes: A Mathematical
Approach for Engineers, Prentice Hall, 1986. This book began as
a second edition to the earlier book and the basic goal remains unchanged
— to introduce the fundamental ideas and mechanics of random processes
to engineers in a way that accurately reflects the underlying mathematics,
but does not require an extensive mathematical background and does not
belabor detailed general proofs when simple cases suffice to get the basic
ideas across.......
An Introduction to
Statistical Signal Processing
f −1 (F )
f
F
Pr(f ∈ F ) = P ({ω : ω ∈ F }) = P (f −1 (F ))
✲
May 5, 2000
ii
An Introduction to
Statistical Signal Processing
Robert M. Gray
and
Lee D. Davisson
Information Systems Laboratory
Department of Electrical Engineering
Stanford University
and
Department of Electrical Engineering and Computer Science
University of Maryland
iv
c 1999 by the authors.
v
to our Families
vi
Contents
Preface xi
Glossary xv
1 Introduction 1
2 Probability 11
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Spinning Pointers and Flipping Coins . . . . . . . . . . . . 15
2.3 Probability Spaces . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Sample Spaces . . . . . . . . . . . . . . . . . . . . . 28
2.3.2 Event Spaces . . . . . . . . . . . . . . . . . . . . . . 31
2.3.3 Probability Measures . . . . . . . . . . . . . . . . . . 42
2.4 Discrete Probability Spaces . . . . . . . . . . . . . . . . . . 45
2.5 Continuous Probability Spaces . . . . . . . . . . . . . . . . 56
2.6 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.7 Elementary Conditional Probability . . . . . . . . . . . . . 71
2.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3 Random Objects 85
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.1.1 Random Variables . . . . . . . . . . . . . . . . . . . 85
3.1.2 Random Vectors . . . . . . . . . . . . . . . . . . . . 89
3.1.3 Random Processes . . . . . . . . . . . . . . . . . . . 93
3.2 Random Variables . . . . . . . . . . . . . . . . . . . . . . . 95
3.3 Distributions of Random Variables . . . . . . . . . . . . . . 104
3.3.1 Distributions . . . . . . . . . . . . . . . . . . . . . . 104
3.3.2 Mixture Distributions . . . . . . . . . . . . . . . . . 108
3.3.3 Derived Distributions . . . . . . . . . . . . . . . . . 111
3.4 Random Vectors and Random Processes . . . . . . . . . . . 115
3.5 Distributions of Random Vectors . . . . . . . . . . . . . . . 117
vii
viii CONTENTS
3.5.1 Multidimensional Events . . . . . . . . . . . . . . . 118
3.5.2 Multidimensional Probability Functions . . . . . . . 119
3.5.3 Consistency of Joint and Marginal Distributions . . 120
3.6 Independent Random Variables . . . . . . . . . . . . . . . . 127
3.6.1 IID Random Vectors . . . . . . . . . . . . . . . . . . 128
3.7 Conditional Distributions . . . . . . . . . . . . . . . . . . . 129
3.7.1 Discrete Conditional Distributions . . . . . . . . . . 130
3.7.2 Continuous Conditional Distributions . . . . . . . . 131
3.8 Statistical Detection and Classification . . . . . . . . . . . . 134
3.9 Additive Noise . . . . . . . . . . . . . . . . . . . . . . . . . 137
3.10 Binary Detection in Gaussian Noise . . . . . . . . . . . . . 144
3.11 Statistical Estimation . . . . . . . . . . . . . . . . . . . . . 146
3.12 Characteristic Functions . . . . . . . . . . . . . . . . . . . . 147
3.13 Gaussian Random Vectors . . . . . . . . . . . . . . . . . . . 152
3.14 Examples: Simple Random Processes . . . . . . . . . . . . . 154
3.15 Directly Given Random Processes . . . . . . . . . . . . . . 157
3.15.1 The Kolmogorov Extension Theorem . . . . . . . . . 157
3.15.2 IID Random Processes . . . . . . . . . . . . . . . . . 158
3.15.3 Gaussian Random Processes . . . . . . . . . . . . . . 158
3.16 Discrete Time Markov Processes . . . . . . . . . . . . . . . 159
3.16.1 A Binary Markov Process . . . . . . . . . . . . . . . 159
3.16.2 The Binomial Counting Process . . . . . . . . . . . . 162
3.16.3 Discrete Random Walk . . . . . . . . . . . . . . . . 165
3.16.4 The Discrete Time Wiener Process . . . . . . . . . . 166
3.16.5 Hidden Markov Models . . . . . . . . . . . . . . . . 167
3.17 Nonelementary Conditional Probability . . . . . . . . . . . 168
3.18 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4 Expectation and Averages 187
4.1 Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
4.2 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
4.2.1 Examples: Expectation . . . . . . . . . . . . . . . . 192
4.3 Functions of Several Random Variables . . . . . . . . . . . . 200
4.4 Properties of Expectation . . . . . . . . . . . . . . . . . . . 200
4.5 Examples: Functions of Several Random Variables . . . . . 203
4.5.1 Correlation . . . . . . . . . . . . . . . . . . . . . . . 203
4.5.2 Covariance . . . . . . . . . . . . . . . . . . . . . . . 205
4.5.3 Covariance Matrices . . . . . . . . . . . . . . . . . . 206
4.5.4 Multivariable Characteristic Functions . . . . . . . . 207
4.5.5 Example: Differential Entropy of a Gaussian Vector 209
4.6 Conditional Expectation . . . . . . . . . . . . . . . . . . . . 210
4.7 Jointly Gaussian Vectors . . . . . . . . . . . . . . . . . . . 213
CONTENTS ix
4.8 Expectation as Estimation . . . . . . . . . . . . . . . . . . . 216
4.9 Implications for Linear Estimation . . . . . . . . . . . . . 222
4.10 Correlation and Linear Estimation . . . . . . . . . . . . . . 224
4.11 Correlation and Covariance Functions . . . . . . . . . . . . 231
4.12 The Central Limit Theorem . . . . . . . . . . . . . . . . . 235
4.13 Sample Averages . . . . . . . . . . . . . . . . . . . . . . . . 237
4.14 Convergence of Random Variables . . . . . . . . . . . . . . 239
4.15 Weak Law of Large Numbers . . . . . . . . . . . . . . . . . 244
4.16 Strong Law of Large Numbers . . . . . . . . . . . . . . . . 246
4.17 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
4.18 Asymptotically Uncorrelated Processes . . . . . . . . . . . . 256
4.19 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
5 Second-Order Moments 281
5.1 Linear Filtering of Random Processes . . . . . . . . . . . . 282
5.2 Second-Order Linear Systems I/O Relations . . . . . . . . . 284
5.3 Power Spectral Densities . . . . . . . . . . . . . . . . . . . . 289
5.4 Linearly Filtered Uncorrelated Processes . . . . . . . . . . . 292
5.5 Linear Modulation . . . . . . . . . . . . . . . . . . . . . . . 298
5.6 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
5.7 Time-Averages . . . . . . . . . . . . . . . . . . . . . . . . . 305
5.8 Differentiating Random Processes . . . . . . . . . . . . . . 309
5.9 Linear Estimation and Filtering . . . . . . . . . . . . . . . 312
5.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
6 A Menagerie of Processes 343
6.1 Discrete Time Linear Models . . . . . . . . . . . . . . . . . 344
6.2 Sums of IID Random Variables . . . . . . . . . . . . . . . . 348
6.3 Independent Stationary Increments . . . . . . . . . . . . . . 350
6.4 Second-Order Moments of ISI Processes . . . . . . . . . . 353
6.5 Specification of Continuous Time ISI Processes . . . . . . . 355
6.6 Moving-Average and Autoregressive Processes . . . . . . . . 358
6.7 The Discrete Time Gauss-Markov Process . . . . . . . . . . 360
6.8 Gaussian Random Processes . . . . . . . . . . . . . . . . . . 361
6.9 The Poisson Counting Process . . . . . . . . . . . . . . . . 361
6.10 Compound Processes . . . . . . . . . . . . . . . . . . . . . . 364
6.11 Exponential Modulation . . . . . . . . . . . . . . . . . . . 366
6.12 Thermal Noise . . . . . . . . . . . . . . . . . . . . . . . . . 371
6.13 Ergodicity and Strong Laws of Large Numbers . . . . . . . 373
6.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
x CONTENTS
A Preliminaries 389
A.1 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
A.2 Examples of Proofs . . . . . . . . . . . . . . . . . . . . . . . 397
A.3 Mappings and Functions . . . . . . . . . . . . . . . . . . . . 401
A.4 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . 402
A.5 Linear System Fundamentals . . . . . . . . . . . . . . . . . 405
A.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
B Sums and Integrals 417
B.1 Summation . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
B.2 Double Sums . . . . . . . . . . . . . . . . . . . . . . . . . . 420
B.3 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
B.4 The Lebesgue Integral . . . . . . . . . . . . . . . . . . . . 423
C Common Univariate Distributions 427
D Supplementary Reading 429
Bibliography 434
Index 438
Preface
The origins of this book lie in our earlier book Random Processes: A Math-
ematical Approach for Engineers, Prentice Hall, 1986. This book began as
a second edition to the earlier book and the basic goal remains unchanged
— to introduce the fundamental ideas and mechanics of random processes
to engineers in a way that accurately reflects the underlying mathematics,
but does not require an extensive mathematical background and does not
belabor detailed general proofs when simple cases suffice to get the basic
ideas across. In the thirteen years since the original book was published,
however, numerous improvements in the presentation of the material have
been suggested by colleagues, students, teaching assistants, and by our own
teaching experience. The emphasis of the class shifted increasingly towards
examples and a viewpoint that better reflected the course title: An Intro-
duction to Statistical Signal Processing. Much of the basic content of this
course and of the fundamentals of random processes can be viewed as the
analysis of statistical signal processing systems: typically one is given a
probabilistic description for one random object, which can be considered
as an input signal. An operation or mapping or filtering is applied to the
input signal (signal processing) to produce a new random object, the out-
put signal. Fundamental issues include the nature of the basic probabilistic
description and the derivation of the probabilistic description of the output
signal given that of the input signal and a description of the particular oper-
ation performed. A perusal of the literature in statistical signal processing,
communications, control, image and video processing, speech and audio
processing, medical signal processing, geophysical signal processing, and
classical statistical areas of time series analysis, classification and regres-
sion, and pattern recognition show a wide variety of probabilistic models for
input processes and for operations on those processes, where the operations
might be deterministic or random, natural or artificial, linear or nonlinear,
digital or analog, or beneficial or harmful. An introductory course focuses
on the fundamentals underlying the analysis of such systems: the theories
of probability, random processes, systems, and signal processing.
xi
xii PREFACE
When the original book went out of print, the time seemed ripe to
convert the manuscript from the prehistoric troff to L TEX and to undertake
A
a serious revision of the book in the process. As the revision became more
extensive, the title changed to match the course name and content. We
reprint the original preface to provide some of the original motivation for
the book, and then close this preface with a description of the goals sought
during the revisions.
Preface to Random Processes: An Introduction for
Engineers
Nothing in nature is random . . . A thing appears random
only through the incompleteness of our knowledge. — Spinoza,
Ethics I
I do not believe that God rolls dice. — attributed to Einstein
Laplace argued to the effect that given complete knowledge of the physics
of an experiment, the outcome must always be predictable. This metaphys-
ical argument must be tempered with several facts. The relevant param-
eters may not be measurable with sufficient precision due to mechanical
or theoretical limits. For example, the uncertainty principle prevents the
simultaneous accurate knowledge of both position and momentum. The
deterministic functions may be too complex to compute in finite time. The
computer itself may make errors due to power failures, lightning, or the
general perfidy of inanimate objects. The experiment could take place in
a remote location with the parameters unknown to the observer; for ex-
ample, in a communication link, the transmitted message is unknown a
priori, for if it were not, there would be no need for communication. The
results of the experiment could be reported by an unreliable witness —
either incompetent or dishonest. For these and other reasons, it is useful
to have a theory for the analysis and synthesis of processes that behave in
a random or unpredictable manner. The goal is to construct mathematical
models that lead to reasonably accurate prediction of the long-term average
behavior of random processes. The theory should produce good estimates
of the average behavior of real processes and thereby correct theoretical
derivations with measurable results.
In this book we attempt a development of the basic theory and ap-
plications of random processes that uses the language and viewpoint of
rigorous mathematical treatments of the subject but which requires only a
typical bachelor’s degree level of electrical engineering education including
PREFACE xiii
elementary discrete and continuous time linear systems theory, elementary
probability, and transform theory and applications. Detailed proofs are
presented only when within the scope of this background. These simple
proofs, however, often provide the groundwork for “handwaving” justifi-
cations of more general and complicated results that are semi-rigorous in
that they can be made rigorous by the appropriate delta-epsilontics of real
analysis or measure theory. A primary goal of this approach is thus to use
intuitive arguments that accurately reflect the underlying mathematics and
which will hold up under scrutiny if the student continues to more advanced
courses. Another goal is to enable the student who might not continue to
more advanced courses to be able to read and generally follow the modern
literature on applications of random processes to information and commu-
nication theory, estimation and detection, control, signal processing, and
stochastic systems theory.
Revision
The most recent (summer 1999) revision fixed numerous typos reported
during the previous year and added quite a bit of material on jointly Gaus-
sian vectors in Chapters 3 and 4 and on minimum mean squared error
estimation of vectors in Chapter 4.
This revision is a work in progress. Revised versions will be made avail-
able through the World Wide Web page
http://www-isl.stanford.edu/~gray/sp.html .
The material is copyrighted by the authors, but is freely available to any
who wish to use it provided only that the contents of the entire text remain
intact and together. A copyright release form is available for printing the
book at the Web page. Comments, corrections, and suggestions should be
sent to [email protected]. Every effort will be made to fix typos and
take suggestions into an account on at least an annual basis.
I hope to put together a revised solutions manual when time permits,
but time has not permitted during the past year.
xiv PREFACE
Acknowledgements
We repeat our acknowledgements of the original book: to Stanford Univer-
sity and the University of Maryland for the environments in which the book
was written, to the John Simon Guggenheim Memorial Foundation for its
support of the first author, to the Stanford University Information Systems
Laboratory Industrial Affiliates Program which supported the computer
facilities used to compose this book, and to the generations of students
who suffered through the ever changing versions and provided a stream of
comments and corrections. Thanks are also due to Richard Blahut and
anonymous referees for their careful reading and commenting on the orig-
inal book, and to the many who have provided corrections and helpful
suggestions through the Internet since the revisions began being posted.
Particular thanks are due to Yariv Ephraim for his continuing thorough
and helpful editorial commentary.
Robert M. Gray
La Honda, California, summer 1999
Lee D. Davisson
Bonair, Lesser Antilles summer 1999
Glossary
{ } a collection of points satisfying some property, e.g., {r : r ≤ a} is the
collection of all real numbers less than or equal to a value a
[ ] an interval of real points including the end points, e.g., for a ≤ b
[a, b] = {r : a ≤ r ≤ b}. Called a closed interval.
( ) an interval of real points excluding the end points, e.g., for a ≤ b
(a, b) = {r : a < r < b}.Called an open interval. . Note this is empty if
a = b.
( ], [ ) denote intervals of real points including one endpoint and exclud-
ing the other, e.g., for a ≤ b (a, b] = {r : a < r ≤ b}, [a, b) = {r : a ≤ r < b}.
∅ The empty set, the set that contains no points.
Ω The sample space or universal set, the set that contains all of the
points.
F Sigma-field or event space
P probability measure
PX distribution of a random variable or vector X
pX probability mass function (pmf) of a random variable X
fX probability density function (pdf) of a random variable X
FX cumulative distribution function (cdf) of a random variable X
xv
xvi Glossary
E(X) expectation of a random variable X
MX (ju) characteristic function of a random variable X
1F (x) indicator function of a set F
Φ Phi function (Eq. (2.78))
Q Complementary Phi function (Eq. (2.79))
Chapter 1
Introduction
A random or stochastic process is a mathematical model for a phenomenon
that evolves in time in an unpredictable manner from the viewpoint of the
observer. The phenomenon may be a sequence of real-valued measurements
of voltage or temperature, a binary data stream from a computer, a mod-
ulated binary data stream from a modem, a sequence of coin tosses, the
daily Dow-Jones average, radiometer data or photographs from deep space
probes, a sequence of images from a cable television, or any of an infinite
number of possible sequences, waveforms, or signals of any imaginable type.
It may be unpredictable due to such effects as interference or noise in a com-
munication link or storage medium, or it may be an information-bearing
signal-deterministic from the viewpoint of an observer at the transmitter
but random to an observer at the receiver.
The theory of random processes quantifies the above notions so that
one can construct mathematical models of real phenomena that are both
tractable and meaningful in the sense of yielding useful predictions of fu-
ture behavior. Tractability is required in order for the engineer (or anyone
else) to be able to perform analyses and syntheses of random processes,
perhaps with the aid of computers. The “meaningful” requirement is that
the models provide a reasonably good approximation of the actual phe-
nomena. An oversimplified model may provide results and conclusions that
do not apply to the real phenomenon being modeled. An overcomplicated
one may constrain potential applications, render theory too difficult to be
useful, and strain available computational resources. Perhaps the most dis-
tinguishing characteristic between an average engineer and an outstanding
engineer is the ability to derive effective models providing a good balance
between complexity and accuracy.
Random processes usually occur in applications in the context of envi-
1
2 CHAPTER 1. INTRODUCTION
ronments or systems which change the processes to produce other processes.
The intentional operation on a signal produced by one process, an “input
signal,” to produce a new signal, an “output signal,” is generally referred
to as signal processing, a topic easily illustrated by examples.
• A time varying voltage waveform is produced by a human speaking
into a microphone or telephone. This signal can be modeled by a
random process. This signal might be modulated for transmission,
it might be digitized and coded for transmission on a digital link,
noise in the digital link can cause errors in reconstructed bits, the
bits can then be used to reconstruct the original signal within some
fidelity. All of these operations on signals can be considered as signal
processing, although the name is most commonly used for the man-
made operations such as modulation, digitization, and coding, rather
than the natural possibly unavoidable changes such as the addition
of thermal noise or other changes out of our control.
• For very low bit rate digital speech communication applications, the
speech is sometimes converted into a model consisting of a simple
linear filter (called an autoregressive filter) and an input process. The
idea is that the parameters describing the model can be communicated
with fewer bits than can the original signal, but the receiver can
synthesize the human voice at the other end using the model so that
it sounds very much like the original signal.
• Signals including image data transmitted from remote spacecraft are
virtually buried in noise added to them on route and in the front
end amplifiers of the powerful receivers used to retrieve the signals.
By suitably preparing the signals prior to transmission, by suitable
filtering of the received signal plus noise, and by suitable decision or
estimation rules, high quality images have been transmitted through
this very poor channel.
• Signals produced by biomedical measuring devices can display spe-
cific behavior when a patient suddenly changes for the worse. Signal
processing systems can look for these changes and warn medical per-
sonnel when suspicious behavior occurs.
How are these signals characterized? If the signals are random, how
does one find stable behavior or structure to describe the processes? How
do operations on these signals change them? How can one use observations
based on random signals to make intelligent decisions regarding future be-
havior? All of these questions lead to aspects of the theory and application
of random processes.
3
Courses and texts on random processes usually fall into either of two
general and distinct categories. One category is the common engineering
approach, which involves fairly elementary probability theory, standard un-
dergraduate Riemann calculus, and a large dose of “cookbook” formulas —
often with insufficient attention paid to conditions under which the formu-
las are valid. The results are often justified by nonrigorous and occasionally
mathematically inaccurate handwaving or intuitive plausibility arguments
that may not reflect the actual underlying mathematical structure and may
not be supportable by a precise proof. While intuitive arguments can be
extremely valuable in providing insight into deep theoretical results, they
can be a handicap if they do not capture the essence of a rigorous proof.
A development of random processes that is insufficiently mathematical
leaves the student ill prepared to generalize the techniques and results when
faced with a real-world example not covered in the text. For example, if
one is faced with the problem of designing signal processing equipment for
predicting or communicating measurements being made for the first time
by a space probe, how does one construct a mathematical model for the
physical process that will be useful for analysis? If one encounters a process
that is neither stationary nor ergodic, what techniques still apply? Can the
law of large numbers still be used to construct a useful model?
An additional problem with an insufficiently mathematical development
is that it does not leave the student adequately prepared to read modern
literature such as the many Transactions of the IEEE. The more advanced
mathematical language of recent work is increasingly used even in simple
cases because it is precise and universal and focuses on the structure com-
mon to all random processes. Even if an engineer is not directly involved
in research, knowledge of the current literature can often provide useful
ideas and techniques for tackling specific problems. Engineers unfamiliar
with basic concepts such as sigma-field and conditional expectation will find
many potentially valuable references shrouded in mystery.
The other category of courses and texts on random processes is the
typical mathematical approach, which requires an advanced mathemati-
cal background of real analysis, measure theory, and integration theory;
it involves precise and careful theorem statements and proofs, and it is
far more careful to specify precisely the conditions required for a result
to hold. Most engineers do not, however, have the required mathematical
background, and the extra care required in a completely rigorous develop-
ment severely limits the number of topics that can be covered in a typical
course — in particular, the applications that are so important to engineers
tend to be neglected. In addition, too much time can be spent with the
formal details, obscuring the often simple and elegant ideas behind a proof.
Often little, if any, physical motivation for the topics is given.
4 CHAPTER 1. INTRODUCTION
This book attempts a compromise between the two approaches by giving
the basic, elementary theory and a profusion of examples in the language
and notation of the more advanced mathematical approaches. The intent
is to make the crucial concepts clear in the traditional elementary cases,
such as coin flipping, and thereby to emphasize the mathematical structure
of all random processes in the simplest possible context. The structure is
then further developed by numerous increasingly complex examples of ran-
dom processes that have proved useful in stochastic systems analysis. The
complicated examples are constructed from the simple examples by signal
processing, that is, by using a simple process as an input to a system whose
output is the more complicated process. This has the double advantage
of describing the action of the system, the actual signal processing, and
the interesting random process which is thereby produced. As one might
suspect, signal processing can be used to produce simple processes from
complicated ones.
Careful proofs are constructed only in elementary cases. For example,
the fundamental theorem of expectation is proved only for discrete random
variables, where it is proved simply by a change of variables in a sum.
The continuous analog is subsequently given without a careful proof, but
with the explanation that it is simply the integral analog of the summation
formula and hence can be viewed as a limiting form of the discrete result.
As another example, only weak laws of large numbers are proved in detail
in the mainstream of the text, but the stronger laws are at least stated and
they are discussed in some detail in starred sections.
By these means we strive to capture the spirit of important proofs with-
out undue tedium and to make plausible the required assumptions and con-
straints. This, in turn, should aid the student in determining when certain
tools do or do not apply and what additional tools might be necessary when
new generalizations are required.
A distinct aspect of the mathematical viewpoint is the “grand exper-
iment” view of random processes as being a probability measure on se-
quences (for discrete time) or waveforms (for continuous time) rather than
being an infinity of smaller experiments representing individual outcomes
(called random variables) that are somehow glued together. From this point
of view random variables are merely special cases of random processes. In
fact, the grand experiment viewpoint was popular in the early days of ap-
plications of random processes to systems and was called the “ensemble”
viewpoint in the work of Norbert Wiener and his students. By viewing the
random process as a whole instead of as a collection of pieces, many basic
ideas, such as stationarity and ergodicity, that characterize the dependence
on time of probabilistic descriptions and the relation between time averages
and probabilistic averages are much easier to define and study. This also
