> For the complete documentation index, see [llms.txt](https://wiki.hanzheteng.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://wiki.hanzheteng.com/algorithm/cv/epipolar-geometry.md).

# Epipolar Geometry

Assuming that cameras satisfy pinhole model, we have the following geometry constraints from two perspective views. 

![](/files/5gxHsglxSLySguQal1Ue)

* Epipoles or epipolar points: $$e\_1, e\_2$$
* Epipolar lines: $$l\_1, l\_2$$
* Epipolar plane: the plane formed by optical centers $$O\_1, O\_2$$ and point $$P$$

### Fundamental Matrix

Suppose that the transformation from frame $$O\_1$$ to frame $$O\_2$$ can be described by a rotation matrix$$\boldsymbol{R}$$ and a 3D translation vector $$\boldsymbol{t}$$. Let $$\boldsymbol{P\_1}$$ denote the position of point $$P$$ in frame $$O\_1$$, and $$\boldsymbol{P\_2}$$ the position of point $$P$$ in frame $$O\_2$$. We then have the relation

$$
\begin{equation} \boldsymbol{P\_2} = \boldsymbol{R} \boldsymbol{P\_1} + \boldsymbol{t}  \end{equation}
$$

To derive the beautiful and concise epipolar constraint, which reflects the fact that $$O\_1, O\_2$$ and $$P$$ are co-planar, we first left-multiply both sides of (1) by$$\[\boldsymbol{t}]$$ to obtain

$$
\begin{equation} \[\boldsymbol{t}] \boldsymbol{P\_2} = \[\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P\_1} + \[\boldsymbol{t}] \boldsymbol{t}  \end{equation}
$$

where$$\[\boldsymbol{t}]$$ is the [bracket notation](https://en.wikipedia.org/wiki/Cross_product#Conversion_to_matrix_multiplication) (i.e. express cross product as a skew-symmetric matrix) of vector $$\boldsymbol{t}$$. The term $$\[\boldsymbol{t}] \boldsymbol{t}$$ is always zero and can be crossed out. We then left-multiply both sides of (2) by $$\boldsymbol{P\_2}^{\mathrm{T}}$$ to obtain

$$
\begin{equation} \boldsymbol{P\_2}^{\mathrm{T}} \[\boldsymbol{t}] \boldsymbol{P\_2} = \boldsymbol{P\_2}^{\mathrm{T}} \[\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P\_1}   \end{equation}
$$

The term$$\boldsymbol{P\_2}^{\mathrm{T}} \[\boldsymbol{t}] \boldsymbol{P\_2}$$ is always zero, because $$\boldsymbol{P\_2}$$ is perpendicular to$$\[\boldsymbol{t}] \boldsymbol{P\_2}$$ and the dot product of the two is always zero. Finally, we obtain the epipolar constraint

$$
\begin{equation}
\boldsymbol{P\_2}^{\mathrm{T}} \[\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P\_1}   = 0 \end{equation}
$$

Let $$\boldsymbol{F} = \[\boldsymbol{t}] \boldsymbol{R}$$ be the Fundamental Matrix, we can rewrite the epipolar constraint in a concise form

$$
\begin{equation}
\boldsymbol{P\_2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{P\_1}   = 0 \end{equation}
$$

The epipolar constraint in this form relates the coordinates of the same point in two references frames.

### Essential Matrix

Recall the pinhole camera model, we can project point $$P$$ onto the image planes in two views by 

Taking the vector product with $$t$$, followed by the dot product with $$p\_2$$ we obtain $$p\_2^{T} \[t]\_{\times}Rp\_1 = 0$$.

$$
\begin{equation\*}
\boldsymbol{E}=\boldsymbol{t}^{\wedge} \boldsymbol{R}, \quad \boldsymbol{F}=\boldsymbol{K}^{-\mathrm{T}} \boldsymbol{E} \boldsymbol{K}^{-1}, \quad \boldsymbol{x}*{2}^{\mathrm{T}} \boldsymbol{E} \boldsymbol{x}*{1}=\boldsymbol{p}*{2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{p}*{1}=0
\end{equation\*}
$$

 its normalized image coordinates (i.e. $$x\_1 = K^{-1} p\_1$$), and the