Feynman-Kac formula

05 Mar 2018

Consider a diffusion process

\begin{equation*} dX_t = \mu_t dt + \sigma_t dW_t \end{equation*}

with the infinitesimal generator

\begin{equation*} \newcommand{\L}{\mathcal{L}} \newcommand{\p}{\partial} \L = \mu_t \p_x + \frac{1}{2} \sigma_t^2 \p_x^2. \end{equation*}

For any functional of a trajectory $X_{t \to T}$, we can derive a differential equation that it satisfies using Ito’s lemma. In two important special cases, equations are particularly beautiful: linear cost (standard optimal control objective) and exponential cost (risk-sensitive optimal control objective). The theorem that relates the functionals with the differential equations is known as the Feynman-Kac theorem.

Linear cost functional

Let $g(s, X_s)$ denote the stage cost. Then the total cost of a trajectory that starts at time $t$ in state $X_t = x$ and finishes at time $T$ is given by

\begin{equation*} Y(t, x) = \int_t^T g(s, X_s) ds + \psi(X_T). \end{equation*}

Taking the expectation with respect to trajectories starting at $X_t = x$, we obtain the following functional

\begin{equation*} V(t, x) = E \left[ \int_t^T g(s, X_s) ds + \psi(X_T) \;\bigg|\; X_t = x \right]. \end{equation*}

By Ito’s lemma, for any nice function $Y$ of a diffusion $X$,

\begin{equation*} E[dY] = \left( \p_t EY + \L EY \right) dt. \end{equation*}

On the other hand, from the formula for $Y$,

\begin{equation*} E[dY] = -g dt. \end{equation*}

Therefore, $V(t, x)$ satisfies the following partial differential equation

\begin{equation*} -\p_t V = g + \L V \end{equation*}

with the terminal condition

\begin{equation*} V(T, x) = \psi(x). \end{equation*}

Exponential cost functional

By the same reasoning as above, functional

\begin{equation*} V(t, x) = E \left[ \psi(X_T) \exp \left( \int_t^T g(s, X_s) ds \right) \;\bigg|\; X_t = x \right] \end{equation*}

solves the partial differential equation

\begin{equation*} -\p_t V = g V + \L V \end{equation*}

with the terminal condition

\begin{equation*} V(T, x) = \psi(x). \end{equation*}

Note that this time around the equation is homogeneous, which is a bit nicer. For more details, read the lecture notes by Jonathan Goodman.