Transition probability function

Let $(S, \mathcal{B})$ be a given measurable space. Function $$ \begin{align*} P &:& S \times \mathcal{B} &\rightarrow [0, 1] \\ && (x, B) &\mapsto P_x(B) \end{align*} $$ is called the transition probability function if certain conditions are satisfied.

Namely, if we fix the first argument, the function

should be a probability measure on $S$ for all $x \in S$, and if we fix the second argument, the function

should be an $S$-valued random variable for all $B \in \mathcal{B}$.

The transition probability function returns the probability of ending up in $B$ when started from $x$.

Transition operator

Let $X : S \rightarrow S$ be a random variable with distribution $\pi$.

The transition probability function $P$ induces an operator $\mathcal{P}$ on distributions which is defined by $$ \newcommand{\md}{\mathrm{d}} \newcommand{\bE}{\mathbb{E}} \newcommand{\aP}{\mathcal{P}} \begin{equation} \label{operator} (\mathcal{P}\pi)(B) = \int_S P_x(B) \,\md \pi(x) \end{equation} $$ for all $B \in \mathcal{B}$. Operator $\mathcal{P}$ is called the transition operator.

The distribution $\rho = \aP\pi$ defined in \eqref{operator} returns the expectation of $P(B)$ under the measure $\pi$ for very $B \in \mathcal{B}$

This is nothing deep, just a rewriting of \eqref{operator}.

Transition density function

If $P_x$ has the density $p_x$

and $\pi$ has the density $f$

then we can rewrite \eqref{operator} in terms of the densities, interchange integrals, and discover that $\rho$ also has a density,

Here is how to do it in detail. $P_x$ is a probability distribution, consequently

Substitute it into \eqref{operator},

We see that distribution $\rho = \mathcal{P}\pi$ has the density

Function $p_x(y)$ is called the transition density function.


We can simply work with densities and forget about distributions when densities exist.

Sometimes \eqref{conditional} is written as

Be aware that the $p$ on the left and the two $p$’s on the right are all different functions. In this case, the letter in the argument carries the information about which function is which. This is not a common convention in mathematics, where an argument is usually just a dummy. However, this notation is rather convenient for manipulating densities, therefore widely used.