Transition probability function

22 Dec 2016

Namely, if we fix the first argument, the function

$$\begin{equation*} P_x : \mathcal{B} \rightarrow [0, 1] \end{equation*}$$

should be a probability measure on $S$ for all $x \in S$, and if we fix the second argument, the function

$$\begin{equation*} P(B) : S \rightarrow [0, 1] \end{equation*}$$

should be an $S$-valued random variable for all $B \in \mathcal{B}$.

The transition probability function returns the probability of ending up in $B$ when started from $x$.

Transition operator

Let $X : S \rightarrow S$ be a random variable with distribution $\pi$.

The distribution $\rho = \aP\pi$ defined in \eqref{operator} returns the expectation of $P(B)$ under the measure $\pi$ for very $B \in \mathcal{B}$

$$\begin{equation*} \rho(B) = \bE_\pi P(B). \end{equation*}$$

This is nothing deep, just a rewriting of \eqref{operator}.

Transition density function

If $P_x$ has the density $p_x$

$$\begin{equation*} \md P_x = p_x \,\md \mu \end{equation*}$$

and $\pi$ has the density $f$

$$\begin{equation*} \md \pi = f \,\md \mu, \end{equation*}$$

then we can rewrite \eqref{operator} in terms of the densities, interchange integrals, and discover that $\rho$ also has a density,

$$\begin{equation*} \md \rho = g \,\md \mu. \end{equation*}$$

Here is how to do it in detail. $P_x$ is a probability distribution, consequently

$$\begin{equation*} P_x(B) = \int_B \md P_x = \int_B p_x \,\md \mu. \end{equation*}$$

Substitute it into \eqref{operator},

$$\int_S P_x(B) \,\md \pi(x) = \int_S \Big( \int_B p_x(y) \,\md y \Big) \,f(x) \,\md x = \int_B \Big( \int_S p_x(y) \,f(x) \,\md x \Big) \,\md y.$$

We see that distribution $\rho = \mathcal{P}\pi$ has the density

$$\begin{equation} \label{conditional} g(y) = \int_S p_x(y) \,f(x) \,\md x. \end{equation}$$

Conclusion

We can simply work with densities and forget about distributions when densities exist.

Sometimes \eqref{conditional} is written as

$$\begin{equation*} p(y) = \int_S p(y | x) \,p(x) \,\md x. \end{equation*}$$

Be aware that the $p$ on the left and the two $p$’s on the right are all different functions. In this case, the letter in the argument carries the information about which function is which. This is not a common convention in mathematics, where an argument is usually just a dummy. However, this notation is rather convenient for manipulating densities, therefore widely used.