Probability, Random Variable, and Distribution

The goal of this note is to fully understand the concepts of the key elements consisting the probability space.

In later chapter Probability MeasureProbability Measure, we will discuss about how to understand $\sigma-$fieldfield and measuremeasure. Before that, we first review some of the key definitions we studied in Mathematical StatisticsMathematical Statistics.

Let a probability space be given for the rest of this section.

Here, you can simply understand the Borel sets as a collection of subsets of the real line. A detailed explanations follows in Borel MeasureBorel Measure. Now we define a probability (measure) on ,

A random variable is simply a mapping that maps each outcome to a real number where we can use well developed mathematical tools.

We now re-define some of the familiar concepts.

See Basic Definitions in Probability > ^24cd70Basic Definitions in Probability > Theorem 7 (properties of distribution function) for the proof.

The density function of a continuous variable is more tricky to define.

Note that the absolute continuity of is a result from a property of the measure, which itself is continuous. While we will further look into the definition later on, here, you can simply understand as a characteristic of the integral.

First, we follow the previous notions of the sample space and the set of all events denoted and , respectively. Obviously, is a collection of subsets of , a set of sets (i.e. family of sets). An typical example of an event set is the power set , which collects all the subsets of .

A sigma-algebra is a subset of the poser set , which is therefore a family of sets in , that satisfies certain properties.

Here, note that the term 'sigma-' in mathematics usually implies 'countable union' or 'countably infinite'. Thus, -algebra means it is closed under the countable union, i.e., it contains every countable union of itself.

Furthermore, since countable set is isomorphic to , meaning that there exists an one-to-one correspondence between the two, we can understand -algebra as the set consists the countably infinite number of the subsets of , where we can index the each of the subsets in natural number bases.

For some -algebra which is defined on , a measure is a function that assigns each events to a non-negative real numbers: Intuitively, this set function is a generalization and formalization of geometrical measures (length, area, and volume) and other common notions (magnitude, mass, and probability). More simply, we can understand the concept of a measure as a set function that measures the size of the given set.

Here, the countable additivity is the key property that makes a measure a generalized length.

Measure Theoretic Preliminaries > ^cdc44bMeasure Theoretic Preliminaries > Theorem 18 (properties of measure) for the proof.

From the definition of measuremeasure, by adding one more property, we finally have the definition of the probability measure.

Compared to the empty set, null set refers to the set which is practically non-existing, while empty set denotes the set which is actually empty. Here, the null means meaningless, insignificance, or negligible, rather then the absence.

Intuitively, 'almost surely' refers to the every point except the null set. This concept is similar to the integral where , and also to the probability where we does not care whether the end points of the set is included or not.

Note that in ^4fba4cDefinition 24 (almost surely converges), we do not care about the timing when enters to the neighborhood of and remains there forever. Thus we introduce further concepts.

Here, the term e.v. and i.o. emphasizes that in terms of convergence in a sequence, the only important thing is the long-term behavior, not the behavior in the first finite horizon of the time.

Usually, ^2ff43bRemark 26 (almost sure convergence, and ev and io) is another common way to define ^4fba4cDefinition 24 (almost surely converges). For the brief understanding, suppose pointwise and let . By the definition, for all .
Thus we have Alternatively, resulting in the definition of almost sure convergence. For the infinitely often part, it can be easily driven using De Morgan's law.

Intuitively, a measurable function is a function between the two measurable spacemeasurable space that preserves the structure of the spaces. Here, the inverse image of any measurable function is measurablemeasurable.

It is shown that it is equivalent to show the following conditions to prove the given function is measurable.

Proof.As 1 and 2 are complement to each other, similarly, 3 and 4 are complement to each other. By ^27dce2Definition 14 (sigma-algebra and measurable space), 1 and 2, and 3 and 4 are equivalent conditions (since the complement set of the element in algebra is also its element). Thus it is sufficient to show that 1 and 3 are equivalent condition, since 1 is a given definition from ^03f1b6Definition 27 (measurable function). Below, we first show that 1 implies 3, and then we show the converse.

() Assume for a function , we have Then we have since , and 1 holds for every . Since and field is closed under every intersection, we have .

() Assume for a function , we have Then we have and as field is closed under every union. □

Remark that ^03f1b6Definition 27 (measurable function) is analogue to Continuous Function > ^1bc20fContinuous Function > Definition 5 (continuous function between topological spaces), which gives the sense of the preserving properties of the measurable function.

For your information, measurable function can be defined in more fundamental way, not directly defining from its inverse-image. In that case, we can also derive the same property of ^03f1b6Definition 27 (measurable function).

Furthermore, measurable function can alternatively obtained from the pointwise convergence of the sequence of simple functions.

In real space, Borel algebra is the intersection of all sigma-algebrasigma-algebra that includes open sets in . In other words, it is the smallest sigma-algebra that can be defined on while including all the open sets (intervals) of . It is useful in the sense that it only contains the necessary elements to define a measure on the set of open sets, which makes it available to define a probability measureprobability measure.

While we defined Borel algebra under Euclidean space, it is more general to start on an arbitrary set. However, in probability theory, it is sufficient enough to define on , as it consists of all open and closed sets that can be defined on .

Note that the equivalence in ^f8eea7Definition 31 (random variable) holds by the properties of ^67466eDefinition 29 (Borel sets on Euclidean space). Random variable is a mapping that maps each elements in the sample space into a real space, which makes it available to use inequalities. Also, it restricts the sigma-field to the inverse image of the Borel sets, restraining the excessive abstractness of the sample space.

Remark that by the definition, is a ()measurable function. Moreover, if , then becomes a Borel function, since we have .

For your information, to generalize the definition into the multivariate random variable, we can simply define such that

Proof.Let are the pairwise disjoint Borel sets, then their inverse image are also pairwise disjoint. We have Thus by ^f02a69Definition 33 (probability distribution), we have where the third equality holds by ^58f06fDefinition 16 (measure and measure space), as denotes a probability measure. □

Thus, is also a probability space. Note that .

Using ^f02a69Definition 33 (probability distribution), we can re-define the independence of the random variable using measure theory.

Note that the independence between and is equivalent to the followings.