Tensor powers at the service of humanity

12 Dec 2016

There is a standard convention in mathematics to denote the set of functions from set $X$ to set $Y$ by $Y^X$. This looks a lot like $\mathbb{R}^n$, with the only difference that $n$ is a natural number whereas $X$ is a set. The motivation of this post is to treat $n$ as a set to uncover some parallels between functions and vector spaces.

$\mathbb{R}^n$ as a set of functions

Let $[n]$ denote a set of $n$ elements. One can think of it as the set $\{ 1, 2, \dots , n \}$. Be careful though, because $[n]$ is not a subset of $\mathbb{N}$—its elements are not natural numbers but some arbitrary objects, which we simply enumerated for convenience. For example, $[2] \cap [3] = \emptyset$ and $[2] \cup [3] = [5]$.

The key observation allowing us to unify notation is that we can view the vector space $\mathbb{R}^n$ as the set of functions from $[n]$ to $\mathbb{R}$,

$$\begin{equation*} \mathbb{R}^{[n]} = \left\{ [n] \rightarrow \mathbb{R} \right\}. \end{equation*}$$

Indeed, if $x \in \mathbb{R}^n$, you can query $x$ for its $k$-th coordinate, $x(k)$. That means $x$ can be viewed as a function $x : [n] \rightarrow \mathbb{R}$. The set of all such functions $\mathbb{R}^{[n]}$ is a vector space isomorphic to $\mathbb{R}^n$.

Let’s see this idea in action on several examples.

Direct product

Consider the direct product of vector spaces $\mathbb{R}^{[2]} \times \mathbb{R}^{[3]}$. It is easy to see that

$$\begin{equation*} \mathbb{R}^{[2]} \times \mathbb{R}^{[3]} \cong \mathbb{R}^{[2] \cup [3]} \cong \mathbb{R}^{[2+3]} = \mathbb{R}^{[5]}. \end{equation*}$$

The exponential notation is very suggestive, as you may have noticed. It allows one to manipulate vector spaces using intuition from natural numbers.

Tensor product

Consider the tensor product of vector spaces $\mathbb{R}^{[2]} \otimes \mathbb{R}^{[3]}$. It is easy to see that

$$\begin{equation*} \mathbb{R}^{[2]} \otimes \mathbb{R}^{[3]} \cong \mathbb{R}^{[2] \times [3]} \cong \mathbb{R}^{[2 \cdot 3]} = \mathbb{R}^{[6]}. \end{equation*}$$

In case of the tensor product, dimensionalities multiply. Interesting! We know that exponents multiply when one takes a power of a power (i.e., $(a^b)^c = a^{bc}$). Can we understand the tensor product in this way? It turns out we can. Check it out yourself.

Note that parentheses are important, because $(2^3)^4 = 2^{12}$ whereas $2^{(3^4)} = 2^{81}$.