Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds

Abstract

Sparsity-based representations have recently led to notable results in various visual recognition tasks. In a separate line of research, Riemannian manifolds have been shown useful for dealing with features and models that do not lie in Euclidean spaces. With the aim of building a bridge between the two realms, we address the problem of sparse coding and dictionary learning in Grassmann manifolds, i.e., the space of linear subspaces. To this end, we propose to embed Grassmann manifolds into the space of symmetric matrices by an isometric mapping. This in turn enables us to extend two sparse coding schemes to Grassmann manifolds. Furthermore, we propose an algorithm for learning a Grassmann dictionary, atom by atom. Lastly, to handle non-linearity in data, we extend the proposed Grassmann sparse coding and dictionary learning algorithms through embedding into higher dimensional Hilbert spaces. Experiments on several classification tasks (gender recognition, gesture classification, scene analysis, face recognition, action recognition and dynamic texture classification) show that the proposed approaches achieve considerable improvements in discrimination accuracy, in comparison to state-of-the-art methods such as kernelized Affine Hull Method and graph-embedding Grassmann discriminant analysis.

Appendix

In this appendix, we give proofs for the following theorems.

Theorem 2

Let ${\varvec{X}}$ be an $d \times d$ symmetric matrix with eigenvalue decomposition ${\varvec{X}} = {\varvec{U}} {\varvec{D}} {\varvec{U}}^T$, where ${\varvec{D}}$ contains the eigenvalues $\lambda _i$ of ${\varvec{X}}$ in descending order. Let ${\varvec{U}}_p$ be the $d \times p$ matrix consisting of the first p columns of ${\varvec{U}}$. Then $\widehat{{\varvec{U}}}_p = {\varvec{U}}_p {\varvec{U}}_p^T$ is the closest matrix in $\mathcal {PG}({p},{d})$ to ${\varvec{X}}$ (under the Frobenius norm).

Proof

Observe that $\Vert \widehat{{\varvec{V}}} - {\varvec{X}}\Vert _F^2 = \Vert \widehat{{\varvec{V}}} \Vert _F^2 + \Vert {\varvec{X}} \Vert _F^2 -2 \left<\!\right. \widehat{{\varvec{V}}}, {\varvec{X}} \left. \!\right>.$ Since $\Vert \widehat{{\varvec{V}}}\Vert _F$ (for $\widehat{{\varvec{V}}} \in \mathcal {PG}({p},{d})$) and $\Vert {\varvec{X}}\Vert _F$ are fixed, minimizing $\Vert \widehat{{\varvec{V}}} - {\varvec{X}}\Vert _F$ over $\widehat{{\varvec{V}}} \in \mathcal {PG}({p},{d})$ is the same as maximizing $\left<\!\right. \widehat{{\varvec{V}}}, {\varvec{X}} \left. \!\right>$. If $\widehat{{\varvec{V}}} = {\varvec{V}} {\varvec{V}}^T$, we may write $\left<\!\right. \widehat{{\varvec{V}}}, {\varvec{X}}\left. \!\right>= \mathop {{\mathrm{Tr}}}\nolimits ( {\varvec{V}} {\varvec{V}}^T {\varvec{X}}) = \mathop {{\mathrm{Tr}}}\nolimits ({\varvec{V}}^T {\varvec{X}} {\varvec{V}}),$ so it is sufficient to maximize $\mathop {{\mathrm{Tr}}}\nolimits ({\varvec{V}}^T {\varvec{X}} {\varvec{V}})$ over ${\varvec{V}} \in \mathcal {G}({p},{n})$.

If ${\varvec{X}} = {\varvec{U}} \mathrm{diag}(\lambda _1, \ldots , \lambda _d) {\varvec{U}}^T$, then ${\varvec{U}}_p^T {\varvec{X}} {\varvec{U}}_p = \mathrm{diag}(\lambda _1, \ldots , \lambda _p)$ and $\mathop {{\mathrm{Tr}}}\nolimits ({\varvec{U}}_p^T {\varvec{X}} {\varvec{U}}_p) = \sum _{i=1}^p \lambda _i$. On the other hand, let ${\varvec{W}} \in \mathcal {G}({p},{d})$. Then ${\varvec{W}}^T {\varvec{X}} {\varvec{W}}$ is symmetric of dimension $p\times p$. Let $\mu _1 \ge \mu _2 \ge \ldots \ge \mu _p$ be its eigenvalues and ${\varvec{a}}_i, \, i=1, \ldots , p$ the corresponding unit eigenvectors. Let ${\varvec{w}}_i = {\varvec{W}} {\varvec{a}}_i$. Then the ${\varvec{w}}_i$ are orthogonal unit vectors, and ${\varvec{w}}_i^T {\varvec{X}} {\varvec{w}}_i = \mu _i$.

For $k = 1$ to p, let $A_k$ be the subspace of $R^d$ spanned by ${\varvec{w}}_1, \ldots , {\varvec{w}}_k$ and $B_k$ be the space spanned by the eigenvectors ${\varvec{u}}_k, \ldots , {\varvec{u}}_d$ of ${\varvec{X}}$. Counting dimensions, $A_k$ and $B_k$ must have non-trivial intersection. Let ${\varvec{v}}$ be a non-zero vector in this intersection, and write ${\varvec{v}} = \sum _{i=1}^k \alpha _i {\varvec{w}}_i= \sum _{i=k}^d \beta _i {\varvec{u}}_i$. Then

$$\begin{aligned} \begin{aligned} \mu _k \le \frac{\sum _{i=1}^k \alpha _i^2 \mu _i}{\sum _{i=1}^k \alpha _i^2 } = \frac{{\varvec{v}}^T {\varvec{X}} {\varvec{v}}}{{\varvec{v}}^T{\varvec{v}}} = \frac{\sum _{i=k}^d \beta _i^2 \lambda _i}{\sum _{i=k}^d \beta _i^2 } \le \lambda _k ~. \end{aligned} \end{aligned}$$

(44)

Therefore $\mu _k \le \lambda _k$ and $\mathop {{\mathrm{Tr}}}\nolimits ({\varvec{W}}^T {\varvec{X}} {\varvec{W}}) = \sum _{i=1}^p \mu _i \le \sum _{i=1}^p \lambda _i = \mathop {{\mathrm{Tr}}}\nolimits ({\varvec{U}}^T {\varvec{X}} {\varvec{U}}) ~.$ $\square$

We acknowledge that this theorem is an adaptation of classical results in trace optimization Kokiopoulou et al. (2011) to the problem of interest in this paper and the proof is inspired by the Courant–Fischer Theorem on the Wikipedia page (Wikipedia 2015).

The chordal mean For two points (matrices) $\widehat{{\varvec{X}}}$ and $\widehat{{\varvec{Y}}}$ in $\mathcal {PG}({p},{d})$ the distance $\Vert \widehat{{\varvec{X}}} - \widehat{{\varvec{Y}}}\Vert _F$ is called the chordal distance between the two points. Given several points $\widehat{{\varvec{X}}}_i$, the $\ell _2$ chordal mean of $\{\widehat{{\varvec{X}}}_i\}_{i=1}^m$ is the element $\widehat{{\varvec{Y}}} \in \mathcal {PG}({p},{d})$ that minimizes $\sum _{i=1}^m \Vert \widehat{{\varvec{Y}}} - \widehat{{\varvec{X}}}_i\Vert _F^2$. There is a closed-form solution for the chordal mean of a set of points in a Grassman manifold.

Theorem 3

The chordal mean of a set of points $\widehat{{\varvec{X}}}_i \in \mathcal {PG}({p},{d})$ is equal to $\mathrm{Proj} (\sum _{i=1}^m \widehat{{\varvec{X}}}_i).$

Proof

The proof is analogous to the formula for the chordal mean of rotation matrices, given in Hartley et al. (2013). By the same argument as in Theorem 2, minimizing $\sum _{i=1}^m \Vert \widehat{{\varvec{X}}}_i - \widehat{{\varvec{Y}}}\Vert _F^2$ is equivalent to maximizing $\sum _{i=1}^m \left<\!\right. \widehat{{\varvec{X}}}_i, \widehat{{\varvec{Y}}} \left. \!\right>= \left<\!\right. \sum _{i=1}^m \widehat{{\varvec{X}}}_i, \widehat{{\varvec{Y}}} \left. \!\right>$. Thus, the required $\widehat{{\varvec{Y}}}$ is the closest point in $\mathcal {PG}({p},{d})$ to $\sum _{i=1}^m \,\widehat{{\varvec{X}}}_i$, as stated. $\square$