📖
Wiki
Back to my personal website
  • Home
  • Equipment and Devices
    • 3D Printer
    • Laser Cutter
    • Motion Capture System
    • Sensors
      • RGB-D Cameras
      • Velodyne LiDAR
      • Zed Camera
      • RealSense D435i
      • IMU
    • eGPU
    • Nvidia AGX Xavier
    • CPU Benchmark
    • Installation Checklist
  • Development
    • Linux
      • Shell
      • GDB
      • Git
      • Tmux
      • Network
      • Tricks
      • Debug FAQ
    • CMake
      • Catkin Tools
      • CMakeLists
      • CMake Variables
      • CMake Commands
      • CMake: find_package()
    • ROS
      • Gazebo
      • wstool
      • roslaunch
      • rosbag
      • multi-threaded spinner
    • ROS2
      • Convert ROS1 bag to ROS2 bag
    • C++
      • C++ 11
      • C++ Examples
      • C++ Debug
      • Factory Method
      • Timing
    • Google Tools
      • GLog
      • GFlags
      • GTest
      • Style Guide
      • Clang Format
    • PCL
      • Point Type
      • Methods
      • Architecture
      • Code Explained
    • Open3D
      • Python API
      • Registration
      • Visualization
      • Tools
    • OpenCV
      • Documentation
      • Modules
    • Other Libraries
      • Eigen
      • Ceres
      • g2o
      • GTSAM
    • Website
  • Algorithm
    • SLAM
      • K-D Tree
      • Octree
      • Bag of Words
      • Distance Measures
      • Coordinate Systems
      • LOAM
      • Iterative Closest Point
      • Generalized ICP
      • Mahalanobis Distance
    • Computer Science
      • Computational Model
      • Sorting
      • Analysis
      • Complexity Classes (P, NP)
      • Divide and Conquer
      • Greedy Algorithm
      • Dynamic Programming
      • Tree
      • Graph
    • Computer Vision
      • Camera Models
      • Distortion
      • Motion Models
      • Shutter
      • Image Sensors
      • Epipolar Geometry
      • Multiple-View Geometry
    • Datasets
      • RGB-D Datasets
      • Point Cloud Datasets
      • LiDAR SLAM Datasets
  • Math
    • Optimization
      • Convex Optimization
      • Descent Methods
    • Probability
      • Moment
      • Covariance Matrix
      • Stochastic Process
    • Topology
      • References
      • Concepts
      • Topological Spaces
      • Representation of Rotations
      • Representation of 3-sphere
    • Algebra
      • Linear Algebra
      • Matrix Factorization
      • Condition Number
      • Matrix Lie Group
    • Differential Geometry
      • Manifold
      • Submanifold
      • Quotient Manifolds
      • Tangent Space
  • Quadrotor
    • PX4 Development
    • Companion Computer
    • Drone Hardware
    • Propeller Lock
    • Debug
  • Favorites
    • Bookmarks
Powered by GitBook
On this page
  • Fundamental Matrix
  • Essential Matrix

Was this helpful?

  1. Algorithm
  2. Computer Vision

Epipolar Geometry

PreviousImage SensorsNextMultiple-View Geometry

Last updated 3 years ago

Was this helpful?

Assuming that cameras satisfy pinhole model, we have the following geometry constraints from two perspective views.

  • Epipoles or epipolar points: e1,e2e_1, e_2e1​,e2​

  • Epipolar lines: l1,l2l_1, l_2l1​,l2​

  • Epipolar plane: the plane formed by optical centers O1,O2O_1, O_2O1​,O2​ and point PPP

Fundamental Matrix

The epipolar constraint in this form relates the coordinates of the same point in two references frames.

Essential Matrix

Suppose that the transformation from frame O1O_1O1​ to frame O2O_2O2​ can be described by a rotation matrixR\boldsymbol{R}R and a 3D translation vector t\boldsymbol{t}t. Let P1\boldsymbol{P_1}P1​ denote the position of point PPP in frame O1O_1O1​, and P2\boldsymbol{P_2}P2​ the position of point PPP in frame O2O_2O2​. We then have the relation

P2=RP1+t\begin{equation} \boldsymbol{P_2} = \boldsymbol{R} \boldsymbol{P_1} + \boldsymbol{t} \end{equation}P2​=RP1​+t​​

To derive the beautiful and concise epipolar constraint, which reflects the fact that O1,O2O_1, O_2O1​,O2​ and PPP are co-planar, we first left-multiply both sides of (1) by[t][\boldsymbol{t}][t] to obtain

[t]P2=[t]RP1+[t]t\begin{equation} [\boldsymbol{t}] \boldsymbol{P_2} = [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} + [\boldsymbol{t}] \boldsymbol{t} \end{equation}[t]P2​=[t]RP1​+[t]t​​

where[t][\boldsymbol{t}][t] is the (i.e. express cross product as a skew-symmetric matrix) of vector t\boldsymbol{t}t. The term [t]t[\boldsymbol{t}] \boldsymbol{t}[t]t is always zero and can be crossed out. We then left-multiply both sides of (2) by P2T\boldsymbol{P_2}^{\mathrm{T}}P2​T to obtain

P2T[t]P2=P2T[t]RP1\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2} = \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} \end{equation}P2​T[t]P2​=P2​T[t]RP1​​​

The termP2T[t]P2\boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{P_2}P2​T[t]P2​ is always zero, because P2\boldsymbol{P_2}P2​ is perpendicular to[t]P2[\boldsymbol{t}] \boldsymbol{P_2}[t]P2​ and the dot product of the two is always zero. Finally, we obtain the epipolar constraint

P2T[t]RP1=0\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} [\boldsymbol{t}] \boldsymbol{R} \boldsymbol{P_1} = 0 \end{equation}P2​T[t]RP1​=0​​

Let F=[t]R\boldsymbol{F} = [\boldsymbol{t}] \boldsymbol{R}F=[t]R be the Fundamental Matrix, we can rewrite the epipolar constraint in a concise form

P2TFP1=0\begin{equation} \boldsymbol{P_2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{P_1} = 0 \end{equation}P2​TFP1​=0​​

Recall the pinhole camera model, we can project point PPP onto the image planes in two views by

Taking the vector product with ttt, followed by the dot product with p2p_2p2​ we obtain p2T[t]×Rp1=0p_2^{T} [t]_{\times}Rp_1 = 0p2T​[t]×​Rp1​=0.

E=t∧R,F=K−TEK−1,x2TEx1=p2TFp1=0\begin{equation*} \boldsymbol{E}=\boldsymbol{t}^{\wedge} \boldsymbol{R}, \quad \boldsymbol{F}=\boldsymbol{K}^{-\mathrm{T}} \boldsymbol{E} \boldsymbol{K}^{-1}, \quad \boldsymbol{x}_{2}^{\mathrm{T}} \boldsymbol{E} \boldsymbol{x}_{1}=\boldsymbol{p}_{2}^{\mathrm{T}} \boldsymbol{F} \boldsymbol{p}_{1}=0 \end{equation*}E=t∧R,F=K−TEK−1,x2T​Ex1​=p2T​Fp1​=0​

its normalized image coordinates (i.e. x1=K−1p1x_1 = K^{-1} p_1x1​=K−1p1​), and the

bracket notation