Epipolar Geometry

Assuming that cameras satisfy pinhole model, we have the following geometry constraints from two perspective views.

  • Epipoles or epipolar points: e1,e2e_1, e_2

  • Epipolar lines: l1,l2l_1, l_2

  • Epipolar plane: the plane formed by optical centers O1,O2O_1, O_2 and point PP

Fundamental Matrix

Suppose that the transformation from frame O1O_1 to frame O2O_2 can be described by a rotation matrixR\boldsymbol{R} and a 3D translation vector t\boldsymbol{t}. Let P1\boldsymbol{P_1} denote the position of point PP in frame O1O_1, and P2\boldsymbol{P_2} the position of point PP in frame O2O_2. We then have the relation

P2=RP1+t\begin{equation} \boldsymbol{P_2} = \boldsymbol{R} \boldsymbol{P_1} + \boldsymbol{t} \end{equation}

To derive the beautiful and concise epipolar constraint, which reflects the fact that O1,O2O_1, O_2 and PP are co-planar, we first left-multiply both sides of (1) by[t][\boldsymbol{t}] to obtain

[t]P2=[t]RP1+[t]t\begin{equation} [\boldsymbol{t}] \boldsymbol{P_2} = [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} + [\boldsymbol{t}] \boldsymbol{t} \end{equation}

where[t][\boldsymbol{t}] is the bracket notation (i.e. express cross product as a skew-symmetric matrix) of vector t\boldsymbol{t}. The term [t]t[\boldsymbol{t}] \boldsymbol{t} is always zero and can be crossed out. We then left-multiply both sides of (2) by P2T\boldsymbol{P_2}^{\mathrm{T}} to obtain

P2T[t]P2=P2T[t]RP1\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2} = \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} \end{equation}

The termP2T[t]P2\boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2} is always zero, because P2\boldsymbol{P_2} is perpendicular to[t]P2[\boldsymbol{t}] \boldsymbol{P_2} and the dot product of the two is always zero. Finally, we obtain the epipolar constraint

P2T[t]RP1=0\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} = 0 \end{equation}

Let F=[t]R\boldsymbol{F} = [\boldsymbol{t}] \boldsymbol{R} be the Fundamental Matrix, we can rewrite the epipolar constraint in a concise form

P2TFP1=0\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{P_1} = 0 \end{equation}

The epipolar constraint in this form relates the coordinates of the same point in two references frames.

Essential Matrix

Recall the pinhole camera model, we can project point PP onto the image planes in two views by

Taking the vector product with tt, followed by the dot product with p2p_2 we obtain p2T[t]×Rp1=0p_2^{T} [t]_{\times}Rp_1 = 0.

E=tR,F=KTEK1,x2TEx1=p2TFp1=0\begin{equation*} \boldsymbol{E}=\boldsymbol{t}^{\wedge} \boldsymbol{R}, \quad \boldsymbol{F}=\boldsymbol{K}^{-\mathrm{T}} \boldsymbol{E} \boldsymbol{K}^{-1}, \quad \boldsymbol{x}_{2}^{\mathrm{T}} \boldsymbol{E} \boldsymbol{x}_{1}=\boldsymbol{p}_{2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{p}_{1}=0 \end{equation*}

its normalized image coordinates (i.e. x1=K1p1x_1 = K^{-1} p_1), and the

Last updated