Epipolar Geometry

Assuming that cameras satisfy pinhole model, we have the following geometry constraints from two perspective views.

• Epipoles or epipolar points: $e_1, e_2$
• Epipolar lines: $l_1, l_2$
• Epipolar plane: the plane formed by optical centers $O_1, O_2$ and point $P$

Fundamental Matrix

Suppose that the transformation from frame $O_1$ to frame $O_2$ can be described by a rotation matrix$\boldsymbol{R}$ and a 3D translation vector $\boldsymbol{t}$. Let $\boldsymbol{P_1}$ denote the position of point $P$ in frame $O_1$, and $\boldsymbol{P_2}$ the position of point $P$ in frame $O_2$. We then have the relation

$$\begin{equation} \boldsymbol{P_2} = \boldsymbol{R} \boldsymbol{P_1} + \boldsymbol{t} \end{equation}$$

To derive the beautiful and concise epipolar constraint, which reflects the fact that $O_1, O_2$ and $P$ are co-planar, we first left-multiply both sides of (1) by$[\boldsymbol{t}]$ to obtain

$$\begin{equation} [\boldsymbol{t}] \boldsymbol{P_2} = [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} + [\boldsymbol{t}] \boldsymbol{t} \end{equation}$$

where$[\boldsymbol{t}]$ is the bracket notation (i.e. express cross product as a skew-symmetric matrix) of vector $\boldsymbol{t}$. The term $[\boldsymbol{t}] \boldsymbol{t}$ is always zero and can be crossed out. We then left-multiply both sides of (2) by $\boldsymbol{P_2}^{\mathrm{T}}$ to obtain

$$\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2} = \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} \end{equation}$$

The term$\boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2}$ is always zero, because $\boldsymbol{P_2}$ is perpendicular to$[\boldsymbol{t}] \boldsymbol{P_2}$ and the dot product of the two is always zero. Finally, we obtain the epipolar constraint

$$\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} = 0 \end{equation}$$

Let $\boldsymbol{F} = [\boldsymbol{t}] \boldsymbol{R}$ be the Fundamental Matrix, we can rewrite the epipolar constraint in a concise form

$$\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{P_1} = 0 \end{equation}$$

The epipolar constraint in this form relates the coordinates of the same point in two references frames.

Essential Matrix

Recall the pinhole camera model, we can project point $P$ onto the image planes in two views by

Taking the vector product with $t$, followed by the dot product with $p_2$ we obtain $p_2^{T} [t]_{\times}Rp_1 = 0$.

$$\begin{equation*} \boldsymbol{E}=\boldsymbol{t}^{\wedge} \boldsymbol{R}, \quad \boldsymbol{F}=\boldsymbol{K}^{-\mathrm{T}} \boldsymbol{E} \boldsymbol{K}^{-1}, \quad \boldsymbol{x}_{2}^{\mathrm{T}} \boldsymbol{E} \boldsymbol{x}_{1}=\boldsymbol{p}_{2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{p}_{1}=0 \end{equation*}$$

its normalized image coordinates (i.e. $x_1 = K^{-1} p_1$), and the